Now that the first quarter Key Performance Indicators (KPI) have been calculated and reviewed, and the purpose of the first quarter’s results have been identified, it is time to move on to the remaining quarters. Similar to the analysis for the first quarter, the second and third quarters should be analyzed in the same fashion. The data for the first through third quarters should be considered frozen and ready for summative analysis (Refer back to blog articles five through seven for guidance on determining the readiness of a “frozen” file.) The fourth quarter; however, will require a different process for review as it is still an active data set, since it is the middle the fourth quarter when this process has begun and does not end until the thirty mark on the timeline outlined in Blog #1. Therefore, the fourth quarter’s data is still being collected and verified.
After running the data for each KPI for the first quarter there are various items the team should consider:
Each of these considerations should be examined when completing the calculations for quarters two and three as well.
Nothing is perfect. There is always some sort of caveat about any data point. At this stage of the process, the team should be looking for limitations related to the collection rather than the significance, or lack thereof, of the data used to determine the value. Let’s look at an example to illustrate the difference. The temperature hit 76 degrees Fahrenheit. This temperature was highly unusual for the city that it was measured in as it was February and usually the temperature is 27 degrees Fahrenheit. What caused this unusual temperature record?
Similar to the process outlined for limitations, the focus at this stage should be on the data collection process rather than the context which will happen during the KPI evaluation process. At this stage it is important to consider related data points that bolster the meaning of the KPI data point. Consider the following KPI:
By the end of the 2017 fiscal year, 85% of vacancies that arise will be filled with candidates who have a college diploma.
For this example KPI, let’s say that the actual percentage for the second quarter was 90%. Additional deductions that can be made could include the percentage of:
The team should once again, review the data used to calculate the quarter specific data point to check to make sure that is reflective of the data set. Data quality checks that were referenced in KPI Assessment (KPIA) blog articles five through seven, should be used to once again conduct a verification for completeness.
4. Special Circumstances
When being interviewed, a common (and quite effective) question asked is: “is there anything else that I should know?” Often this question ends up being a safety net that catches information that is pertinent and useful but might not have been anticipated. When thinking about special circumstances that should be recorded about each quarter’s data this question comes in useful. The data dictionary for the data sets that were used to calculate the KPI(s) is a great resource to explore notations made during the data retrieval and analysis processes. These notations may be important to explain in greater detail as potential limitations or notes that will help later-on in the evaluation process. Items that are not already listed in the data dictionary can also be added. For example, based on notes from the data dictionary, there were some concerns about data quality during the first quarter. When the KPIs were calculated for each quarter, there was more variation with the first quarter data than the second or third. Upon further investigation, the team recalled that the team member that was usually in charge of the data collection of the KPI in question was on maternity leave during the first quarter. The person who was filling-in to manage data collection in the first quarter did not have as much experience as the team mate on maternity leave. The quality of the data was noticeably better once the team member on maternity leave returned to work. When the KPIE team constructs a narrative around the meaning and implications of the KPI values calculated during the assessment process, this type of special circumstance may become an important footnote to explain certain variations in the data.
Once the value has been determined for each KPI for the first, second, and third quarters, and the team has identified the limitations, deductions, completeness, and special circumstances, the team needs to come to consensus on the finality of the values calculated. For each quarter each team member should sign-off on the value calculated signifying that the prescribed business rules were followed, the values were cross referenced, and notes relevant to the KPI value(s) are recorded correctly.
Although the end of the fourth quarter has not yet occurred, there are steps to tentatively complete the KPIA process and commence with KPIE. In the next blog article, we will focus on the fourth quarter.
Blog #12 Questions: Should there be limitations to quarterly data?
Blog #13 Sneak Peek: Fourth Quarter