Tag Archives: PMCert

Indicators of PM Competence

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

I haven’t been very reliable about posting here regularly. Part of the reason is that I often get involved in LinkedIn discussions about subjects that are near and dear to my heart … like project management competence. So in lieu of copying my comments in here, please take a minute to skim this thread:

What are the true indicators of PM competence?

And don’t forget to pay particular attention to my comments!

Your Comments?

Performance Based Competency

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

I’ve been getting a fair number of questions recently about “performance based competencies,” and it’s been quite a while since I posted anything here, so I decided to kill two birds with one stone and post something on the topic. The following text is adapted from the GAPPS Project Manager Framework.

Competent comes from the Latin root competere which means “to be suitable.” In today’s workplace, the term “competent” is generally used to describe someone who is sufficiently skilled to perform a specified task or to fill a defined position — a competent physician, a competent salesperson, a competent plumber. Organizations are interested in assessing the competency of individuals in order to guide employment and development decisions.

Broadly speaking, there are two major approaches to defining and assessing competency:

  • Attribute based wherein personal attributes such as knowledge, skills, and other characteristics are identified and assessed. Competence is inferred based on the presence of the necessary attributes.
  • Performance based wherein work outcomes and performance levels are identified and assessed. Competence is inferred based on the demonstrated ability to satisfy the performance criteria.

At IPMA-USA, we use the latter approach. Performance based competency assessment was invented by the US Army, and today, it is widely used throughout the world. For example, government endorsed standards and qualifications frameworks in Australia (Department of Education, Science and Training), New Zealand (New Zealand Qualifications Authority), South Africa (South African Qualifications Authority), and the United Kingdom (Qualifications and Curriculum Authority) are all focused primarily on performance based competency assessment.

Our performance based competency assessment looks at the following two questions:

  • What is usually done in this occupation, profession, or role by competent performers?
  • What standard of performance is usually considered acceptable to infer competence?

We answer these questions by defining:

  • Units of Competency

A Unit of Competency defines a broad area of professional or occupational performance that is meaningful to practitioners and which is demonstrated by individuals in the workplace.

  • Elements of Competency

Elements of Competency describe the key components of work performance within a Unit. They describe what is done by individuals in the workplace but do not prescribe how the work is done. For example, project managers must “define risks and risk responses for the project,” but they can do it themselves or delegate the work to others. In addition, there are many different tools and techniques that they could use.

  • Performance Criteria

Performance Criteria describe observable results and actions in the workplace from which competent performance can be inferred. Performance Criteria can be satisfied in many different ways; there are no mandatory approaches, tools, or methodologies.

  • Explanatory Statements

Explanatory Statements help to ensure consistent interpretation of the Elements and the Performance Criteria by expanding on critical or significant aspects of them to enable consistent application in different contexts.

This approach is both consistent with and compatible with generally accepted practice within the field of competency development and assessment.

The Units, Elements, and Performance Criteria are not linear or sequential: there is no requirement that the work be done in any particular sequence or that the Performance Criteria be satisfied in any particular order. In addition, some Performance Criteria can be satisfied with relatively little effort while others will require a substantial commitment from the project manager over the full length of the project.

Our Performance Criteria address threshold performance — demonstration of the ability to do something at a standard considered acceptable in the workplace. They do not measure superior performance — what the best project managers do. Superior performers should, however, be able to satisfy the threshold criteria without difficulty.

We assess against the minimum number of Performance Criteria needed to infer competence. As a result, a candidate must satisfy all of the Performance Criteria in the applicable Units in order to be viewed as competent. In addition, the Performance Criteria represent different levels of detail. The number of Performance Criteria in a Unit or Element is not proportional to the amount of time or effort that a project manager must spend in that area to be viewed as competent.

Your Comments?

More on Project Success

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

A slightly edited version of a recent LinkedIn post … this all started with a question that asked, “is scope, cost, and schedule enough to determine ‘project success.'” I responded with my usual position that “project success” is not an absolute, and that there were two dimensions to project success: project management success and product success. A couple of others continued to suggest using “project success” as the term for what I call “project management success.”

My most recent comment follows ..’

We’re close … my problem with “project success” is that the users/ owners/ clients/ customers/ sponsors for most projects could care less about how well managed the project was. They want their “thing” so that they can start using it to benefit the business. If we talk about “project success,” they are not going to understand us because the “project” to them is the “thing.”

In your shed examples [one built by hand and another built by a subcontractor], I disagree that different criteria are needed to measure project management success. For example, in both cases, we must select at least one measure of schedule success. There are many options:
— Delivery no later than the initial agreed date.
— Delivery within one week of the initial agreed date.
— Delivery at least one day before the delivery of the stuff going into the shed.
— Delivery no later than the final agreed date as adjusted due to scope changes.

One of these (or some other variant) would be agreed at the start. Once the criteria are agreed, it’s up to me as the PM to decide whether I should build the shed myself or contract it out.

Based on the success criteria, if you are a competent project manager, you may decide to include a contingency for a subcontractor’s heart attack. But if I’m the sponsor, I am not going to assess every decision you make: I am going to evaluate the results. Can you meet the schedule success criteria without being a good PM? Yes. Can you meet all of the PM success criteria without being a good PM? Unlikely without a co-dependent sponsor.

If you decide to manufacture the shed in bulk … that is a completely different endeavor with no relationship to the first one. The development of the product road map, design variants, and the rest is a project. The implementation of those plans is, indeed, product management and (as I have said from the beginning) has different success criteria. The product planning could have been part of the initial scope (with the first shed being a prototype), and we would still need separate sets of criteria for assessing the project management results and the product results.

But even without the bulk manufacturing … there are product/ benefit/ outcome criteria for the initial shed. The specs neglected to specify which model tractor, so the designer made an assumption, and the client signed off. Now the tractor is delivered, and it doesn’t fit. The shed is not fit for use (the “product” is a partial failure) despite the fact that it was well-managed.

One final repeat: the Project Manager is NOT responsible for delivering the product results. Just for being aware of what they are and doing their best to make sure that the project management success criteria support them.

Your Comments?

Art or Science?

PM Certification commentary by William Duncan, IPMA-USA Certification Director.
Once again, from a LinkedIn discussion … someone posted a question. “Is project management an art or a science?” There were nearly 30 comments, most of which argued “both,” and most of which leapt right into the discussion without bothering to define their terms. So here is my post:

As is my wont, I checked the dictionary …
There are two definitions of science: the study of the natural world, and an organized body of knowledge. Project management does NOT meet the first definition; it does satisfy the second. So we can say PM is a science, but not according to the most common definition.

As well, there are two definitions of art: producing works to be appreciated primarily for their beauty or emotional power, and skill at doing something. Project management does not meet either definition, although GOOD project management would satisfy the second. So again, we can say that PM is an art, but not according to the most common definition.

More to the point … what difference would knowing the answer make to you when managing a project? I suspect the answer is “none.”

On the other hand (and my biases are showing here) … would you rather hire a project manager who has proven their knowledge of the science of project management, or one who has demonstrated their grasp of the art? Would you rather hire a project manager who has passed a multiple choice exam? Or one whose performance has actually been assessed against a set of defined criteria?

Your Comments?

Can’t anybody here play this game?

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

My Dad always rooted for the underdog. As a result, I hated Casey Stengel and the Yankee’s with a passion … until Casey took over the hopelessly inept Mets, and then he became one of my favorites. The quote above dates from his experience managing the new New York team in its first year. If you google the phrase, you’ll also see that it is a popular heading for columnists ranting against what appears (to them at least) to be willful stupidity.

So … I’m a columnist … and here comes a rant …

Can’t anyone here manage this project?

Why is there so much apparent incompetence out there? I’ve been posting to a variety of LinkedIn discussions lately, and here are a couple of the things that people claim are common:

  • Project managers who reject changes that provide value.
  • Project managers who don’t know the difference between an estimate and a budget.
  • Project managers who don’t know the difference between scope and work.
  • Project managers who don’t understand that fixed price means fixed scope.
  • Project managers who don’t realize that a “construction project” is just one phase of an asset development project.
  • Project managers who think that schedule baselines can be established without considering resource availability.
  • Project managers who don’t know what float is.

I keep hearing that certification provides value by encouraging people to learn the fundamentals so that they can pass a test. Since many of the project managers referenced above seem to have been certified, what’s going on?

It’s really quite simple. Most project management certifications evaluate knowledge rather than performance. Let’s face it: getting the right answer on a multiple choice question is not nearly the same as being able to make the right choice on-the-job. Even those certifications that require experience generally don’t evaluate the quality of that experience. Would you want to hire a project manager who gained their experience delivering projects months late and 50% over budget?

To quote the Case again … “Don’t cut my throat. I may want to do that later myself.”

This is why we certify based on performance. Always have. Always will.

Your Comments?

Bad question, bad exam

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

I stopped by a website recently whose proprietor specializes in training people to take a certification exam offered by one of our competitors … “That Other Organization” or TOO. Here was the Question of the Month offered by this training provider:

“Ten stakeholders need to receive communications on an important change. You and your team of 40 are concerned that everyone needs to receive these changes. What is the formula to calculate communication channels on a project? How many communication channels are there?

Now, I realize that this is an open-ended question rather than multiple-choice, and so it may not be truly representative of what’s on the actual TOO exam. I also don’t know if this person has attended an item-writing session, so I don’t know what expertise they have to construct questions.

But this question does illustrate some of the reported problems with the construction of other project management certification exams. As a result, it also illustrates some of the problems that you will not have with our Level D exam.

First, the questions asks about an important concept: the fact that the number of communication channels goes up exponentially as the size of the project team increases. However, I could be well aware of this fact and still:

  • Not know the formula.
  • Have forgotten the formula.
  • Recall the formula incorrectly.
  • Do the calculation wrong.

So there is a good chance here that I know the concept and still get the answer wrong. By definition, that means this is a bad question.

Second, has anyone ever used this formula on a real project? I’m going to guess that the answer is no. At IPMA-USA, we try to avoid asking questions that are of academic interest only.

Next, the question itself has an excess of information that does not affect the answer. The first two sentences are completely irrelevant and simply slow the examinee down. Change the numbers from 10 and 40 to 3 and 7. Same question. I’m guessing that the item-writer was trying to provide context, but extra text does not equate to context. Context is when I give you information that is relevant to your choice of an answer.

Now let’s look at the question itself:

  • “Ten stakeholders.” Not clear whether these 10 are part of the team or not.
  • “You and your team of 40.” Are you part of the team? Is the team 40 people or 41?
  • Both NYT Style Guide and the Chicago Book of Style counsel against mixing number forms. So if the first two sentences were needed, they should read “10 … 40” or “Ten … forty” to make it easier to absorb the information.
  • “Everyone needs to receive these changes.” Who is “everyone”? The team? The 10 stakeholders? Some other group?
  • Let’s assume that we want to send the information to the 10 stakeholders. That means there areten channels since this is a one-way push, not a discussion. So the formula may be irrelevant. This starts to sound like a trick question.

Our promise to you: No trick questions. No excess verbiage. No useless knowledge.

Your Comments?

Designing a Certification Program: How to Certify (2)

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

Recapping from “How to Certify (1)” … certification is about assessing competence. Competence is intangible so it must be inferred from some kind of evidence. That evidence can be input-oriented or output-oriented. At IPMA-USA, we have decided to focus on outputs or results.

Why? Simple logic. If someone has already done the job successfully once, there is a much better chance that they can do it successfully again.

After looking at a number of alternatives, we decided to use a performance based competency standard (PBCS) as our measuring stick. Lots of reasons:

  • PBCS have been around for well over 20 years.
  • They are the preferred approach to competency assessment among most Human Resource Departments.
  • They are used in government-sanctioned competency assessment programs in countries such as Australia and South Africa.

We became a founding member of the Global Alliance for Project Performance Standards (GAPPS) and worked with GAPPS to develop a PBCS that would be compatible with IPMA’s Four-Level-Certification. Some of our assessors were initially skeptical, but they were won over the first time they did an assessment.

Explicit, pre-defined criteria provide a higher degree of validity than any other approach currently in use. A comprehensive set of questions enhances inter-rater reliability.

Not only that, but the candidates themselves were equally delighted. One of my favorite comments came from a Level B (Senior Project Manager) candidate: “Wow! I actually learned something in the process of being assessed!”

The GAPPS standard is available for FREE from www.globalpmstandards.org.

Your Comments?

Designing a Certification Program: How to Certify (1)

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

I’ve left this for last since it is the hardest issue to address. In fact, I have several books on my shelves that cover how to assess competence, so trying to answer that question in a few hundred words does seem more than a bit presumptuous. So please think of this entry as an introduction or an overview. If there is interest, I’ll get into more detail later.

First, remember that competence is intangible. You can’t see it, touch it, feel it, or smell it (although I have heard that incompetence has a distinctive aroma). Because competence is intangible, we must infer it from some type of evidence. Thus “how to certify” is fundamentally a question of what kinds of evidence will you require and how will you evaluate it.

In this regard, there are two broad schools of thought. One is input-oriented and the other is output-oriented. The input-oriented folks look at the characteristics of individuals who have been successful in a particular role, and then infer that someone else with those same characteristics will be successful as well. Here are some simple examples:

  • Most successful professional basketball players are tall, so we infer that height is an aspect of basketball competence.
  • People who do well on the Law School Aptitude Test (LSAT) tend to do well in law school, so we infer that someone who does well on the exam will be a competent law school student.
  • Exam-based certifications assert that competence is a function of knowledge, and they infer that someone who can pass a well-constructed test will be able to apply their knowledge in the real world.

As you can see, the input-oriented school tends to be predictive in nature. It asks, “are you likely to be successful?” rather than asking, “have you been successful?” Since there are so many potential factors involved in being competent as a project manager, we elected to adopt the output-oriented approach. We look for evidence of actual performance as a project manager.

Your Comments?

Designing a Certification Program: Why Certify?

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

This might sound like a simple question to answer, but it really isn’t. There are many possible reasons:

  • Because individuals want a credential that will help them get a new or better job.
  • Because individuals want something to brag about.
  • Because hiring managers want a tool to “screen out” some of the resumes they get.
  • Because line managers want a way to “prove” that their staff is skilled.
  • Because line managers want a way to “prove” that their PMs are more skilled than those of their peers.
  • Because line managers want a way to help them decide who to promote.
  • Because buyers of services want to know that their providers are competent.
  • Because sellers of services want a competitive edge.
  • Because professional organizations want to make money.
  • Because professional organizations want to limit entrance to the profession or discipline in order to increase the salaries of their members.

Some of these reasons are more altruistic; some are less. And each reason has the potential to influence the design of the program. For example, if you want to help someone get a new or better job, marketing the credential may be more important than the quality of the assessment process. If you want to assess promotion potential, you will be looking at different factors than if you are evaluating a candidate’s ability to do their current job.

At the same time, there is a lot of overlap and there are some feedback loops. For example, if the certification program does what it claims to do, then everybody wins:

  • Certified individuals will find it easier to find new and better jobs.
  • Hiring managers will feel comfortable screening-out non-certified individuals.
  • Buyers of services can be more confident about the level of competence of their providers’ staff.
  • The sponsoring organization can make a lot of money …

Sorry. Had to mention that last one. Project management certification today is big business. The world’s best known certification has generated an estimated $150,000,000 (yes, seven zeros) in “profits” for its sponsoring organization in the last 10-12 years. Two lesser known programs have generated substantial income for their sponsoring organizations. Several major international enterprises have developed their own internal certification programs because of the selling advantage it provides. And certification preparation is big business. Stop by any LinkedIn project management group and you will see that a huge percentage of the “discussions” are actually advertisements for certification preparation products.

So … why is IPMA-USA doing this? One major reason: we felt there was a crying need for a performance-based credential. Our founders were disappointed with the quality of the certification programs in the market, especially those available in the USA. We wanted to develop something that would actually assess a candidate’s performance, not just their knowledge and experience. We wanted to develop a credential that was based on verifying that the candidate had actually worked as a project manager. On verifying that the candidate could provide evidence of successful performance.

I think we’ve succeeded. We have clearly built a better mousetrap. Now the question is … does anyone else care? Here’s the question I’ve been asking: Would you rather go to an interview saying “I got a D- or better on my exam and no one evaluated my experience,” or “I’ve got two skilled project managers who questioned me in detail for 2 hours about over 50 explicit performance criteria and found me competent.”

Your Comments?

Defining a Certification Program: Where to Certify?

PM Certification commentary by William Duncan, IPMA-USA Certification Director.

There are three aspects of “where” that all have to do with marketing considerations.

The first aspect is, of course, geographic coverage: local, national, international, or global? The broader the coverage, the greater the expense. In addition, trying to offer a product or service globally also requires knowledge of local customs and languages. On the other hand, operating in only one country limits you to organizations that also operate only in your country. At IPMA-USA, we think we have the best of both worlds through our membership in IPMA and use of IPMA’s Four Level Certification program. IPMA provides the basic structure and relies on its National Member Associations to tailor the program to local needs. This way, IPMA-USA and PMCert can focus on what is needed and useful in the good ol’ U.S. of A. while still providing a credential that is recognized and valued around the world.

The second aspect of “where” also addresses geographic coverage, but on a more micro scale: do we offer assessment interviews in just a few cities and make our candidates travel to us? Or do we attempt to to provide local assessors in all major cities? The travel-to-us model has worked well among IPMA’s European members with their relatively small countries. In the USA, we have opted for the latter approach and are busy recruiting qualified assessors throughout the USA. Our plan is to have at least two assessors within a one-hour drive of 90% of the US population.

The final aspect of “where” has to do with industry emphasis. Focusing on just a few market sectors would mean lower costs and a simpler product. For example, there would be no need to address the fact that the oil and gas industry uses the term “contingency reserve” to describe what the aerospace and defense industry call “management reserve.” However, project management is used in pretty much every field of endeavor from medicine to construction to software development. It is as vital to success in the public sector as it is to success in the private sector. And one of IPMA-USA’s organizational objectives is to address the societal impact of poor project management, so we decided that we would not limit ourselves: our certification program is open to all industries and application areas despite the challenges that choice creates.

Your Comments?