???????????

Paul Young  and Rich Mironov shared illuminating posts last week  on how to measure a product manager.  I also read about the same topic in the book, Inspired by Marty Cagan. That gave me a fillip to write on the same topic borrowing and adding on to their thoughts.

I have asked this question before to many people on many accounts to understand the parameters on which the performance of a product manager is measured.

Some of the questions that Rich wanted answers to were:

  • What metrics do we use for evaluating product managers, and how can we tell if they are doing a good job?
  • Are there PM KPIs*? Our agile development teams tell us that roadmaps are no longer needed, but our customers and sales teams still demand firm commitments.
  • How should we organize our product teams?  Senior/junior, technology vs. market segment, product owner vs. product manager?
  • What do career paths look like?

Though Paul admits that it’s a tough one to answer, he breaks the parameters into three areas:

  • Product-level metrics
  • Team-level metrics
  • Individual-level metrics

In the absence of clear metrics on team and individual, most organization rely on product-level metrics, which includes product revenuemargin or customer satisfaction.

These parameters make more sense because using them a  product manager is more motivated to lead the product toward success.  Unfortunately, product managers typically have very little control over the outcome of these metrics, or at best, indirect control.  Most times, the executive team would change the direction completely owing to a new competition or an acquisition.  This has a huge impact on the product manager’s metrics hence they don’t work most of the times.

The other issue according to Paul on  measuring product managers is the issue on timelines.

If you are measuring your team on product revenue, you can make an argument that it may take up  to 18-24 months to know if the product manager is doing a good job.  Consider this – if you hire a new product manager, before he or she can impact product revenue, the following will need to occur:

  • The new product manager will need to go into the market and research the market’s problems (up to 3 months)
  • The product manager will then write what they have learned into a business plan, and get it approved (1-2 months)
  • Then the product manager will write requirements for engineering (up to 3 months)
  • Then engineering will build something (6-9 months)
  • Then in most products there will be a Sales cycle (3-6 months)

Only then will we have the data to understand if the decisions made on the front-end of the process are “good.”  Hopefully we will be more agile and most faster, but the point remains – using product-level metrics to measure individual performance is insufficient. Hence Paul and Rich recommend a better way.

For measuring individual-level metrics, Rich has the following points to focus on:

  • My best indicators of a product manager’s success are subjective: Does she have a roadmap agreed to by Engineering and Marketing (as a proxy for consensus building)?
  • Does Engineering consider her a core part of their team (and not a technical lightweight)?
  • Do Marketing and Sales see her as the source of product facts, strategy, and market intelligence?
  • Do I trust her judgment, objectivity, whole product thinking, and personal integrity?
  • Does the team show more focus (and less panic) over time? Are other departments trying to recruit or borrow her?

Rich admits that the pointers are not as quantitative and scientific as he would like to be.

Paul lists out his own metrics. This is what he says:

   In my experience, the best product managers throw themselves against unanticipated product-specific market and organizational issues.  They practice higher-order problem solving, making progress fiendishly hard to measure.                           Great PMs fill the organizational gaps, work intramurally, and often hang back when credit is being awarded.   Their success can be invisible: the absence of confusion and unhealthy politicking. 

  I’m hesitant to assign numeric goals to my individual product managers.  If I give bonuses for perfect requirements, I’ll get 100-page MRDs (instead of time spent with customers).  If I reward pure revenue performance, my folks might                       spend all of their time on the road (and neglect next quarter’s planning). – See more at: http://www.mironov.com/pm-kpi/#sthash.xUzCPoO3.dpuf

For measuring individual-level metrics, Paul lists out the following for a product manager to be market-driven:

  1. Setting a quota for market visits (10/quarter is a good start),
  2. Being able to defend an updated business plan in front of the senior management and their peers every quarter, without using the phrases “I think,” or “In my opinion”(which requires higher order thought and research).
  3. Keeping their finger on the pulse of the business by defining and measuring the right product level metrics and communicating them in the form of a dashboard report monthly.

Paul favors these metrics because they require a product manager to get out from behind their desk and be outside-in focused, and the product manager can control them, as opposed to revenue, margin or CSAT which the product manager cannot control.

How do we measure product managers? By Marty Cagan in the book Inspired

  • The true measure of the product manager is the success of his or her product.
  • Another metric available is : Net Promoter Score.

Here’a how it works: You ask your customers how likely they would be to recommend your products to others, on a scale of 10-0. Those who rate 9-10 are considered “promoters” (they’re out there telling their friends how much they love your product, and are actually evangelizing for you); those who rate 7-8 are lukewarm or neutral; and those who rate 0-6, the “detractors,” that is, they not only won’t recommend your product, they may even actively warn their friends or the public against your product. If you take the percentage of promoters and subtract the percentage of detractors, you get the NPS, which essentially tells you if you have more people cheering for you or against you.

Some companies that rate very high on NPS are companies like Apple, Amazon, Google, and eBay.