• Guide the product manager and those who interact with him/her towards useful, productive focus and behavior.
• Clarify the goals, progress, and results. If you’re not sure what these are, you can’t be sure if they’ve been achieved.
• Define the product manager’s scope of responsibility and authority in the company.

Knowledge Metrics

One problem with measuring knowledge is the person measuring needs to know more than the person being measured – unlikely in the real-world. What’s needed are objective measures, ones that can be readily agreed upon.

What’s the most important “knowledge” for a product manager? It’s knowledge of the market. So the key “knowledge” question is: Does the product manager know what the market needs and how it will evolve? It’s a good, essential question. But, how to measure it?

There are four key components of target market knowledge that can be readily quantified and measured. These are market trends, customer needs, competitive landscape, and market opportunity size. Two quantitative metrics can be used here:

• A percentage metric reflecting the extent to which the product manager’s articulation of answers to each of these four questions is completed and clear to internal reviewers on the team.
• A percentage metric reflecting the extent to which this articulation has been validated by external reviewers (such as market research analysts, leaders and decision makers in target customers, friendly industry experts).

Do these metrics motivate the product manager to do the right thing? To score high, the product manager must do exactly what’s needed, and do it completely.

Do the metrics clarify goals and results? Each metric specifies related scope of knowledge and measurable achievement, defining the goals and enabling tracking of progress.

Do they clarify the product manager’s responsibility and authority? Within the boundary related to knowledge, they do. First, it’s the product manager’s responsibility to generate and organize all that knowledge and complete reviews and validations. Second, a product manager has the authority to engage internal and external reviewers.

Does the product manager have a realistic chance to get high scores? That depends mostly on the product managers own abilities – provided the related authority is there.

Creativity Metrics

As was the case with product manager knowledge, directly measuring creativity can’t be done objectively. But, it can be done indirectly.

It’s the job of a product manager to define the product. More often than not, it’s a task done with others. But, it’s always the responsibility of the product manager to articulate the product definition.

What’s involved? A product manager has done a good job defining the product if its features and timing fit the market opportunity, the feature-revenue dependency map is complete and understood, and product requirements are clear, prioritized and feasible.

The only reason a company develops a product is to make profits. Thus, the product definition must include a mapping between the product’s feature set and expected revenue. Successful products tend to have, at least initially, almost all of their features impacting revenue. In other words, almost every target customer wants the same feature set.

Once features are defined (including their timing and impact on revenue), product requirements can be generated. These must be clearly articulated, prioritized, and feasible. The right approach is to define requirements based on market needs first, independent of resources. Then, analyze and negotiate the intersection of requirements with available resources to generate the development plan. Two useful quantitative performance metrics are:

• Percentage of internal reviewers accepting product manager’s explanations of each of these three topics.
• Percentage of external reviewers confirming the validity of product definition, as articulated by the product manager.

Action Metrics

Measuring product manager’s performance in terms of actions taken is very different from applying the knowledge and creativity metrics. That’s because it’s not about correct and clearly articulated information anymore. It’s now about getting things done, with other people. Can a product manager’s performance in action be measured objectively?

To be effective, product managers must achieve three key results of their actions: an approved product development plan, effective product promotion, and product revenue growth. The related metrics are:

• Scope of agreement on the development plan, negotiated and achieved by due date with the responsible development and production managers. All agreed, on time, you get 100%. Anything less, you lose points.
• Completion and acceptance (actual usage) of information on product and product-based solutions for the purposes of promotion. The referees here are the marketing and sales managers. You get 100% if both of them say they got everything they need to achieve their own performance targets.
• Tracking between your revenue forecast and the actual revenue won. Tracking on target or better: you get 100% or more. Less, you lose points.

Do these metrics motivate the right people to do the right thing? They clearly motivate the product manager. Getting high scores means good things have been achieved. But, much depends here on other managers: development, production, marketing and sales. Are they motivated in the right direction by these metrics?

The answer is yes and no. That’s because each of these managers has their own performance metrics and those would always motivate them the most. To make it all work together, the general manager needs to make sure performance metrics of different managers work together rather than against each other.

The good news is the clarity of product management metrics helps define related responsibilities and metrics for other managers. It’s clearly the responsibility of the product manager to drive the creation of the development plan but it’s the responsibility of the development and production managers to do all they need to finalize the plan and then deliver.

It’s the same with marketing and sales managers, they have responsibility to negotiate and come to an agreement with the product manager as to the promotion tools and revenue expectations and then drive to deliver promotion and sales results.

Given these metrics, does the product manager have the authority to make sure other managers do the right thing? No. The metrics don’t give that authority. But, they do help clarify the scope of authority the product manager must have.

In Aggregate

One can derive a single aggregate metric for product management’s performance from the metrics above. It’s just using the right average and giving different weights to individual metrics while computing the average.

Conditions

“I can’t achieve my goals unless …” can be seen as an excuse or a real statement of fact, and must be addressed before getting going.

To know the market needs, the product manager must have access to both sources of information and to opinion givers and reviewers. That involves some budget and contact considerations. Budgets are always limited but there’s usually some money set aside and that amount becomes a factor. You have to do the most within the budget and clearly can’t spend more money than is budgeted. That may limit the total available access to research and reviewers but not the percentage of those available that you have succeeded with. So, while the access limits are firm, your performance within those limits depends only on you.

As for the internal reviewers, the usual problem is the time that a review takes. People often change their opinions: “Who, me? I never said that!” A product manager can only overcome the issues of “attention” and “memory” if product management-related reviews and negotiations/agreements are seen by others as high priority tasks.

Getting an agreement on a plan involves balancing objectives and resources, where the resources “belong” to someone else.

The product manager may have P&L responsibility for the product, which helps improve product management results in the company in two ways: First, it’s a confirmation of a product managers authority and helps in interactions with other managers in the company. Second, it’s a strong motivation for the product manager to take care of all issues impacting product success.

Summary

Product management is a complicated job with a broad reach with many interactions and dependencies. An inconvenient side effect of that complexity is opinions on product manager’s achievements tend to be subjective and vague.

Constructing and applying a set of objective product management performance metrics can be of great value, not only for the purpose of evaluation of product manager’s performance but also as a reference, a tool that helps align the efforts of multiple managers in the company. Effectively, while looking at one function in the company metrics can help increase productivity  of the entire team.

L. Kurt Reiss managed products at multiple startups and companies large and small, such as Interactive Supercomputing, Cascade Communications and Lucent Technologies. Focusing on technology ranging from niche products to a global market-leading product family, Kurt contributed to and led product management and marketing teams through all product cycle phases. He is the author of FOR PRODUCT SUCCESS—A Partnership of Two Managers and the What and Why of It. Contact: kurt@pathwaysfps.com

© Copyright 2010 by L. Kurt Reiss.