Measures: Will You Get What You Want?

May 10, 2010

I recently spoke with a new client who shared with me their innovation measures.  When I looked at their measurement system, I immediately saw flaws.

But before addressing these imperfections, let me first provide you my perspective on innovation measures.

In general, there are three types of measures associated with “challenge-based” innovation (be sure to read this article if you are unfamiliar with the concept of challenge-based innovation):

  1. Process Measures - These measure the activity associated with your challenges (e.g., 500 registered solvers, 40 submissions per challenge, 80 votes per challenge, etc)
  2. Solve-Rate Measures - These subjectively measure how well you solved your challenges (e.g., 82% of challenges were partially solved, 61% of challenges were completely solved, etc)
  3. Value Measures – These measure the actual value accrued (e.g., increased revenues by $25M, reduced costs by $35M, etc)

The last measure (value) is where the rubber meets the road.  This is your ultimate goal.  But sometimes, value realization can take years (or in the case of pharmaceutical companies, decades).  Therefore, the second measure (solve-rate) is a good way to monitor progress with your program.  But what about process measures?

Process measures are leading indicators that can be useful in measuring trends over time for things like community engagement, effectiveness of internal communications, and quality of challenges.

Let’s look at one common process measure: the number of ideas/solutions submitted for a given challenge.  This was one of the measures that my new client used.

Imagine that you are using crowdsourcing to find a solution to a challenge.  You post the challenge on your website or intranet.  A month later you check to see how many responses you get.  In this scenario…

Which is better:

  • getting 100 ideas/solutions?
    or
  • getting only 2 ideas/solutions?

Most people intuitively think that 100 solutions is better than 2.  In fact, most organizations believe that more ideas equates to greater success.  The reality is, however, that 100 is not necessarily better than 2.

Let me re-frame the question…

Which is better:

  • getting 100 ideas where only 2 of them were exactly what you needed and the other 98 were duds?
    or
  • getting 2 ideas where both were exactly what you needed?

Now the correct answer is a bit more obvious.  In this situation, the latter is probably better.  The amount of work needed to sift through the solutions is a lot less when you have only 2 submissions.  Imagine if you received 10,000 ideas of which only 2 were good.  You can see now that the effort to find the best solutions/ideas might be overwhelming.

Although activity is good, too many submissions can indicate that you have a poorly defined challenge.  Therefore the ratio of good ideas to duds might be a more interesting measure.

The key is, make sure you understand the unintended consequences of your measurement system, especially when it comes to process measures. If done properly, process measures can help you drive higher solve rates (measure #2). And often, higher solve rates lead to greater value (measure #3) in the long run.  But not always.

High solve rates with low value can also indicate problems with your innovation program:

  • Poor implementation – You are unable to convert solutions into finished products/services
  • Poor commercialization – Your solutions do not meet the needs of the market/customers and therefore do not generate revenue
  • Poor relevance – Your challenges, although solved, are not important enough to “move the needle” of the organization’s innovation efforts

Measures are important for helping tracking your innovation efforts.  And they can help diagnose potential issues.  But it is important to measure the right things.

There is an old expression: “You will get what you measure.”

But the bigger question is, “Will you get what you want?”

Innovation and the Performance Paradox

January 9, 2008

The Innovation Performance Paradox CurveToday, my article on “The Performance Paradox: When Less is More” was published by the American Management Association.

You may recall that I introduced this concept in a blog entry last month.

What is the Performance Paradox?

The more fixated on your goal you become, the greater your chance of success, right? Yes, but only to a certain extent. It turns out that when people are too fixated on the future, their creativity and overall performance actually diminish.

Read the entire Performance Paradox article on the AMA website.

How to Swim Faster

July 3, 2007

At a recent workshop on creativity, I discussed “the performance paradox” – the concept that trying harder produces poorer results.

Afterwards, one executive in the audience came up to me and told me his own story.  He said…

“When I was a kid, I went to summer camp.  One of our daily activities was swimming.  We were told to swim our laps as fast as possible.  As we did, the camp counselors timed our speed.  We did this over and over, each and every day.

“As expected, our lap times improved the more we practiced.  However, about half-way through the summer, our improvements stopped.  No matter how hard we tried, we could not go any faster.

“It was at that point that the counselors told us they would no longer evaluate us on our speed.  Instead they were going to rate us based on the quality of our stroke.  We discovered afterwards that we were still being timed.  Surprisingly, by focusing on style rather than speed, we all went significantly faster.  When we stopped trying to go faster, we went faster.”

Reduced performance is often the result of focusing on a “goal” rather than being “present.”

In what areas of life can you improve YOUR performance by focusing on what is in front of you rather than worrying about the result?

Where, in the past, have you improved your performance by being present?

How to Perform at Optimal Levels

May 6, 2007

In the early 1900s, Robert Yerkes and J. D. Dodson developed the aptly named Yerkes-Dodson Law. The premise is that performance increases relative to motivation (they call it “arousal”) only to a point, after which performance drops. It is typically drawn as an inverted U-shaped curve.

You will notice that I superimposed three “goal” concepts on this graph to give you a sense of how they (roughly) relate.

If you are goal-less, you have no sense of direction and no motivation. Therefore, your performance is low. This is not surprising.

As your motivation increases, your performance increases. Being goal-free – having a sense of direction and purpose, without specific deadlines and limitations – can increase performance…to a point.

Then, as you become goal-driven, performance paradoxically decreases. Goals increase stress and focus you on the future rather than the present.

This phenomenon has been documented in numerous places throughout this blog. Race-car pit crews who increase performance when they are not worried about the stop watch. Students who perform better on exams when they are not as focused on grades. Sales people who sell more when they are not driven by sales targets.

Yerkes and Dodson suggest that different types of tasks require different levels of arousal (to use their word). To improve concentration, intellectually challenging tasks require lower levels of arousal for optimal performance while physically demanding tasks require higher levels. This may explains why professional athletes tend to be more goal-driven. However, even then, goals can limit performance. Listen to my interview with Dr. Doug Gardner, former sports psychology consultant to the Boston Red Sox.

Goal-Free Living is NOT about eliminating your goals. You can have goals and still perform at optimal levels. They key is to have the RIGHT goals (ones that “pull” you forward and don’t create stress) and be PRESENT to what you are doing (being detached from the desired outcomes).

Do you have examples of where you performed at optimal levels by freeing yourself from the stranglehold of rigid goals?

Go 95%; Achieve Optimal Performance

October 3, 2005

I just read a blog entry on CanOWorms that discusses the concept of 95% perfection. The general idea is that in sports, we achieve optimal performance when we put 95% effort into what we do. My own experiences — personal and professional — support this premise.

A few years ago, I worked with a Formula One team (auto racing). Their pit crews have long been admired for their ability to fuel a car, change the tires (back before rule changes that disallowed tire changes during refueling), and do the required maintenance in a matter of seconds. There are 19 people in a pit crew. To find the optimal configuration of the team, they move each of the crew members around until they get the best combination. And then they practice more. All of this is under while being measured with a stop watch. Eventually the team can go no faster; they hit a performance plateau no matter how hard they try. Once, as an experiment, the pit crew members were told that they were NOT going to be timed; that they should just go as fast as possible without going full out (95%?). The result? The pit crew shaved several tenths of a second off their best time – although pit crew members “felt” that they went slower.

When we remove the time pressures of traditional goals and the mental pressure to go full out (100%), our efforts flow more effortlessly and we perform at optimum levels. Whether it be in sports or in life, when we play, have fun, and allow life to unfold naturally – rather than forcing it – we operate at a higher level of performance AND do it with greater ease.