PM Commentary by Stacy Goff
This is part two of our two-part post on Success Factors and Measures. Two independent events last month (an interview for a magazine article and a webinar) resonated around a frequently-discussed, but often disputed topic: What is project success, and how do you achieve it? The events covered two aspects of project success, the Success Factors (that lead to project success) and the Success Measures (used to evaluate success). This posting covers the Success Measures.
The Success Measures
Tim Jaques and Frank Salidis ran the latest webinar in the IPMA-USA 2010 Dialogue series the first week of July. The topic was Perspectives on Project Success: Excellence in Project Management. The well-presented and discussed Dialogue was excellent, but there is much more to the topic than an hour’s time. Some of the key points included the fact that the Triple Constraint is merely a project measure, and is certainly not as important to the end-user as such hard-to-measure items as customer satisfaction.
Other points included discussions about tangible and intangible value, including Return On Investment, Stakeholder satisfaction (beyond customers), and even enhanced PM intelligence. Perceived failures, at least according to project measures, may be successes by the time of product measurement. A key example provided was the Sydney Opera House. The distinction made: Project outputs versus project outcomes.
The webinar introduced this complex topic; the topic deserves a lot more discussion. After all, how can one hour-long Dialogue cure the years of neglect we’ve seen in measuring organization benefits from projects? Let us explore this topic a bit further, organized around several key points:
- The Most Important Measures
- Responsibility for Measures
- Reluctance to Measure
- Those Difficult Intangibles
- Timing of Measurement
The Most Important Measures
The most important project success measures are those factors that are important to the business or organization unit. Some organizations use the term Business Driver to focus the project on business results. One set of universal Business Drivers comes not from project management, but from Strategic Planning. Drawing from that discipline, the purpose of a problem-solving or opportunity-seizing project (select one) is to:
For a Problem-Solving Project
For an Opportunity-Seizing Project
The Business Driver expresses the Why of the project. It is either embedded in the first five words of a single Objective, or used in conjunction with a series of What-oriented Objectives. Experience has shown that a project that has multiple business drivers will tend to fail to meet them all. Thus, one business driver must be the focus of the project. Others may be “realized if possible” (the happy accident), but you should never try to achieve multiples. Note that while we have used the above universal list since our Strategic Planning days of the early 1980s, only recently has “Meet regulatory requirements” become such a frequently-used business driver.
Responsibility for Measures
Who is responsible for establishing the success measures, and then following up on them? It clearly must be (internal) line of business Customer Managers, in the group or groups the project affects, for several reasons. First, they are the ones who are in a position to evaluate and measure the project results in the organization. Secondly, the project team is usually long gone by the time of the final benefits measurements. A less obvious reason is this: Actions of internal Customer Managers have direct impact on whether the project is successful.
Even with Competent and Performing project managers and teams, stretched-thin internal Managers may fail to deliver. This happens when they cannot afford to provide access to the right customer staff during requirements elicitation, test planning and implementation, user documentation and training, and post-project organizational change management. This failure dooms the project from the start. Do you think the Manager who could have saved the doomed project is eager to measure the results? No, they were already frustrated because, despite their best intentions, their hands were tied.
In an IT Methodology, THE Guide, originally published by ProjectExperts and Requirements Solutions Group in the mid-1980s, we identified a key role for all projects, the Customer Representative. This was a Manager from the business area, who could speak for the project’s Customers. We cited managing Time and Cost as a responsibility of the Project Manager, and managing Scope and Quality as the responsibility of the Customer Representative. Both shared responsibility for managing Risks and Talent. The rationale: Even a competent project manager has difficulty balancing all the (as we called them) Vital Signs. As we’ve written elsewhere, in our popular Project Levers And Gauges article, successful project teams manage both the trailing indicators, and the leading ones. This was one way we assured that happened.
Reluctance to Measure
There are more reasons for a reluctance to measure success. In organizations that justify projects using Cost-Benefit Analysis, Break Even Time, or Return on Investment, there is often resistance to showing how many staff positions were “saved”. This popular measure is usually evaluated by counting the heads as they roll out the door. Few organizations have the internal measurements in place to show how all those “heads” were immediately re-purposed into other essential-but-resource-starved business functions. Too bad that cutting headcount is the only obvious measure to so many. A significant business training opportunity would be to show middle Managers how to value and verify business benefits from projects, including the intangible or hard-to-quantify ones.
Those Difficult Intangibles
Those Intangibles are difficult! Some of them are multi-variant, and may require alignment of all the planets in the solar system for them to come true. Others are multi-responsibility: No one Manager has control or even influence over all the variables needed to achieve them. Yet, we have seen $100M projects justified mostly by intangibles—and that is even outside Information Technology projects, where no one can agree what “user friendly” means. And, there we may have problems finding friendly users in the first place.
Timing of Measurement
Some organizations wait until long after the project is completed to measure its success, which makes some sense. After all, you cannot measure success until you’ve had some! And besides, maybe if we ignore measuring our results, Executive Management might forget that we promised this new product that the project launched will grow quarterly earnings 25% per quarter within 2 years after completion. Pretty embarrassing that the market changed significantly just before launch… Good thing CEOs change every three years in our industry…
Effective organizations begin evaluating the success measures at project initiation. After all, they need to know the baseline measures. Before Requirements are complete is a smart time to update the Success Measures that were originally stated at Portfolio Prioritization, but are now obsolete. And the effective organizations trace the indicators that will result in final success throughout the project, including at each Milestone or Stage Gate approval and in evaluating each requested change. This success-focus also establishes heightened Management vigilance for the out-of-control time-cost-quality-risk-scope status we’ve too often seen in the last third of some projects.
We said above that the excellent IPMA-USA Success Dialogue only began the true dialogue about Measuring Project Success. And our comments above may merely add more questions. It is up to you, dear reader, to add your perspective to this discussion. What is Project Success, from the Executive’s or Manager’s point of view? From the end-user’s view? From the Project Team’s view? How do you measure success? When do you measure it? Who is responsible? And, who has a success story you can share, about demonstrating and measuring project success?