Executives often ask me for the impossible and what they want is best exemplified by an often used phrase, “A sip from the firehose.” In most organizations today there is no shortage of data and in the project and portfolio management realm in particular we have a plethora of systems and processes that can generate data but no manager can possibly sift through thousands or even millions of records in order to determine what decisions they should make.
The result is that most significant decisions about projects including the decision to start the project in the first place is made on an ad-hoc basis.
The technique to avoid this is well known. Organizations should choose “Key Performance Indicators” (KPIs )which will surface information that is critical to making those business decisions. But, how does one choose those KPIs? How do you know that the KPIs you’re using are producing the result you require? Let’s take a look.
Choosing KPIs
First of all, where do we look for choosing the right Key Performance Indicator and how will we measure it? These metrics will be used in a variety of ways. We can use these metrics for distinguishing one project from another. We can use them in the project selection process, the stage gate process, the resource priority allocation process and the project review process. There is no limit to the number of things we could measure.
One place to look it through the main elements of PMI’s PMBOK (Project Management Body of Knowledge). In each section you might have possible measure that would lead to better management of the project.
In the Scope area, for example, you might have scope creep or scope change as a measure. In the Time area you might look at delays in the schedule. In the Cost area you might look at resource costs or Return on Investment. In the Communications area you might look at the timeliness of schedule updates or the status of project reporting. In the HR area you might look at Resource Variance or Resource Load/Overload or Resource balance. In the Risk area you might look at the volume of risk items and their severity or urgency and so on and so on.
Not every metric will make sense in every context. For example, in a private organization, projected Return on Investment might be a critical measure. In a public organization this might be completely inappropriate. There, citizen satisfaction or alignment with legislative requirements might be much more appropriate.
Some of the criteria for a good KPI would be this:
* The KPI reflects some element of corporate strategy
So, measuring ontime delivery might make sense in most organizations, but where corporate strategy is more focused on quality or safety, focusing on ontime delivery might result in management making decisions that are diametrically opposed to the corporate mission.
* The KPI is actionable
All too often we see organizations with spectacular dash boards containing lovely graphics and charts and, when we ask what action is supposed to happen if this indicator turns red or if the needle on that guage goes to the danger zone, no one knows the answer. The whole point of a measuring and reporting on a KPI is to ultimately take action.
* The KPI should result in obtainable objectives
Creating a metric which no one ever has a chance of getting out of the “red” and into the “green” makes no sense. It results in urgent emergency action day after day after day and ultimately is self-defeating. The objective that the KPI represents (overload less than 110% of availability for example) must be attainable.
* The KPI should be measureable
It’s tempting to make KPIs which are wholly subjective, where someone just plugs in a number or a color. If this is based on a feeling rather than some empirical measure, the KPI typically has little value and yet the decisions made on the basis of the KPI may be just as significant as those based on other empical measures.
* The KPI should not be duplicated
Just because we have a dozen different ways to display late schedules, showing all those KPIs doesn’t add value. In fact, it can be confusing and disruptive.
* And finally, the KPI and the actions associated with it should be understood by all decision makers. Having a common understanding among all the decision makers of what a particular measure and indicator means is critical to using KPIs effectively.
KPI Challenges
There are a number of easy mistakes that people doing this exercise for the first time face. Here are some of the most common:
* “Great news, I have hundreds of KPIs!”
This isn’t great news at all. Remember, what we’re after is a sip from the firehose. If you open the tap too far; if you let too much water through the hose, you’ll knock management from its feet and they won’t be able to absorb the information. Typically we look for a handful of KPIs. As Chris Iervolino, an expert in Business Performance Management (BPM) said, “Somewhere in the extensive negotiations of creating the KPIs, the ‘K’ got lost.” So, just because you can measure a thing, you don’t need to measure it.
* “Great news, we have only one KPI!”
Having too few KPIs is also not useful. There are always a couple of opposing forces and results that can be identified in the project management business and the whole point here is to make business decisions. Those opposing forces need to be identified and displayed in some way.
* KPIs are too subjective
There’s a strong incentive to invent KPIs for which there is no measure; no metric. The result is an indicator behind which is just a subjective decisions based on someone’s feeling or intuition. While many business decisions are made on a manager’s intuition, when we put subjectivity into a Key Performance Indicator, we take a subjective perspective and display it as an empirical or data-sourced result. The effect of this can be negative.
* The KPI has questionable completeness or quality
Imagine that we have a Key Performance Indicator that shows our resource capacity. The graphic shows the organization’s current status of all work and all resource availability and projects the expected over or under load of resources against our project and nonproject tasks in the future. But the indicator only has current project data for 20% of the projects. Another 60% of the project data is at least 30 days old and 20% of the data is missing altogether. Imagine what business decision might be made on the basis of this indicator alone and how potentially damaging that would be. When KPIs are displayed, we always recommend that there is some indicator of the indicator’s quality and the completeness of the underlying data.
Wrap up
The great news about applying the Key Performance Indicator paradigm to project and portfolio management data is that there’s typically lots of great data to choose from and the nature of the project management process results in a lot of that data being of very high formalized quality. That’s perfect for a productive KPI exercise.
If you’re getting started on choosing your KPIs, the best advice I can give is to start small. Start with 3 or 4 indicators and get the measurements of them, the display of them and the actions associated to them right. The benefits of doing this can be profound and don’t worry, you can always expand the number of indicators in the future.
Happy measuring!
