Measures: Will You Get What You Want?

I recently spoke with a new client who shared with me their innovation measures.  When I looked at their measurement system, I immediately saw flaws.

But before addressing these imperfections, let me first provide you my perspective on innovation measures.

In general, there are three types of measures associated with “challenge-based” innovation (be sure to read this article if you are unfamiliar with the concept of challenge-based innovation):

1. Process Measures - These measure the activity associated with your challenges (e.g., 500 registered solvers, 40 submissions per challenge, 80 votes per challenge, etc)
2. Solve-Rate Measures - These subjectively measure how well you solved your challenges (e.g., 82% of challenges were partially solved, 61% of challenges were completely solved, etc)
3. Value Measures – These measure the actual value accrued (e.g., increased revenues by $25M, reduced costs by $35M, etc)

The last measure (value) is where the rubber meets the road.  This is your ultimate goal.  But sometimes, value realization can take years (or in the case of pharmaceutical companies, decades).  Therefore, the second measure (solve-rate) is a good way to monitor progress with your program.  But what about process measures?

Process measures are leading indicators that can be useful in measuring trends over time for things like community engagement, effectiveness of internal communications, and quality of challenges.

Let’s look at one common process measure: the number of ideas/solutions submitted for a given challenge.  This was one of the measures that my new client used.

Imagine that you are using crowdsourcing to find a solution to a challenge.  You post the challenge on your website or intranet.  A month later you check to see how many responses you get.  In this scenario…

Which is better:

• getting 100 ideas/solutions?
  or
  
• getting only 2 ideas/solutions?

Most people intuitively think that 100 solutions is better than 2.  In fact, most organizations believe that more ideas equates to greater success.  The reality is, however, that 100 is not necessarily better than 2.

Let me re-frame the question…

Which is better:

• getting 100 ideas where only 2 of them were exactly what you needed and the other 98 were duds?
  or
• getting 2 ideas where both were exactly what you needed?

Now the correct answer is a bit more obvious.  In this situation, the latter is probably better.  The amount of work needed to sift through the solutions is a lot less when you have only 2 submissions.  Imagine if you received 10,000 ideas of which only 2 were good.  You can see now that the effort to find the best solutions/ideas might be overwhelming.

Although activity is good, too many submissions can indicate that you have a poorly defined challenge.  Therefore the ratio of good ideas to duds might be a more interesting measure.

The key is, make sure you understand the unintended consequences of your measurement system, especially when it comes to process measures. If done properly, process measures can help you drive higher solve rates (measure #2). And often, higher solve rates lead to greater value (measure #3) in the long run.  But not always.

High solve rates with low value can also indicate problems with your innovation program:

• Poor implementation – You are unable to convert solutions into finished products/services
• Poor commercialization – Your solutions do not meet the needs of the market/customers and therefore do not generate revenue
• Poor relevance – Your challenges, although solved, are not important enough to “move the needle” of the organization’s innovation efforts

Measures are important for helping tracking your innovation efforts.  And they can help diagnose potential issues.  But it is important to measure the right things.

There is an old expression: “You will get what you measure.”

But the bigger question is, “Will you get what you want?”

There is No GPS for Innovation

OK, after 2 weeks of sleep deprivation due to manuscript deadlines, I am now back in action here.  The final version of the manuscript went to the publisher on Saturday.  I then played Personality Poker in Memphis with nearly 100 representatives from Penguin’s gift sales on Sunday.   These individuals sell books into non-traditional bookstores, gift stores, hospital gift shops, department stores, casino, and similar places.

Last weekend, I played Personality Poker with a couple hundred people at a conference in Canada.

After the event,  over a dozen of us decided to go to dinner together.  Half the people fit into taxis.   After the taxis departed from the hotel, the remaining individuals went in two cars, one of which I drove.  We had the address and a map. I, being Mr. Technology, plugged the address into the GPS.  The other individual had the map, but also relied on directions he received from the front desk.  I didn’t bother getting directions since I had the navigation system.

I was the first car out of the parking lot. After exiting the hotel, I turned left, just as the GPS told me to do.  The other car followed, but not for long.  David, the other driver flashed his lights.  I kept driving.  After a minute I realized David was no longer behind me.  Instead of believing that I might be going in the wrong direction, I just assumed that the GPS was taking me there via a shortcut.

After taking a series of turns – left, right, left, right, left, right – the final turn led us to a dead end.  In fact, this road was nothing more than a large pile of dirt.  So much for taking a shortcut.

Since my technology was not going to get us there, we needed to rely on the map.  Unfortunately, the map provided by the hotel only had the restaurant marked off.  The hotel was not to be found.  The reason we could not find the hotel on the map was because the map did not extend far enough to include it.

There we were, in the middle of nowhere, with a map that told us nothing – and a GPS that told us even less.

This got me thinking. 

How often do we drive our innovation programs the same way I drove to the restaurant that night?

We create our plans for innovation and we start driving.  There might be signals along the way (like the flashing lights of the car behind us) that something is not right.  In the case of innovation, it might be signals from the customers, buyers, or vendors telling us we are going the wrong way.  But all too often, we continue to drive forward, arrogantly believing we are right and that those signs are all wrong.

No matter how great your plans are, you need to keep your eyes open.  Look for signs. Don’t assume others are wrong.  Maybe your blueprint/map is incorrect.

Or, as Scott Cook from Intuit so eloquently said, “For every one of our failures, we had spreadsheets that looked awesome.”

There are no accurate GPS systems in the world of innovation.  Your ability – and willingness – to adapt, evolve, and change your plans is critical to a successful innovation program.

If you don’t watch out for the signs and you blindly follow your plans, your innovation program will probably lead you to a huge pile of, um, dirt.

P.S. We did eventually get to the hotel.  We did what any sane person would do…we asked for directions.

NESTA Open Innovation Conference Videos

Last week I spoke at an event hosted by NESTA – the UK’s National Endowment for Science, Technology, and the Arts.

The day focused on Open Innovation and had some spectacular speakers including Cheryl Perkins (the former Chief Innovation Officer for Kimberly-Clark), Karim Lakhani (an open innovation guru from Harvard Business School), Stefan Lindegaard (a well-known expert on open innovation), and Helmut Traitler (from Nestle).

You can watch my opening remarks here. (8 minutes)

On the NESTA website, you can watch all of the other videos including my panel discussion, some Q&A, the morning panel, and more.

NOTES: A few quick comments on my opening remarks video: 1) For those in the US…Pop Idol is the UK version of American Idol. 2) I was a little over eager with InnoCentive “Solver” count. We have over 200K, but not quite a quarter of a million…yet. 3) There are many versions of the Edison quote…the one I prefer is, “I have not failed 700 times. I have not failed once. I have succeeded in proving that those 700 ways will not work. When I have eliminated the ways that will not work, I will find the way that will work.” Admittedly, I’m not sure what he actually said since I wasn’t there.

Great Event in Canada

This weekend I am speaking at a very cool event!  And I would love to have you join me.

What: The Ultimate Success Event
When: Saturday April 17th, 2010 from 8am to 6:30pm
Where: Chateau Cartier Hotel close to downtown Ottawa Canada

But, to kick things off, the weekend starts Friday night with a private party that is totally free.

Peggy McColl is releasing her 7th book, “Viral Explosions,” and Friday is her book launch party.  I have known Peggy for years, and she is an amazing woman with life changing books.

The party will be a blast.  I will of course be there.  There will be lots of entertainment, free hors d’oeuvres, beverages, and give-aways galore.

Then…after celebrating the launch of “Viral Explosions” on Friday night, Saturday is a full day celebration of your success.

The Ultimate Success Event, Saturday April 17, will be a jam packed, seminar featuring a stellar line up of 13 luminaries in the field of personal and business success coaching, including yours truly.  I’ll be playing Personality Poker with the 250 people in attendance.  Other speakers will talk about marketing, sales, personality development and much much more.

Click here to get ALL the details of both the free Book Launch Party for “Viral Explosions”…AND the Ultimate Success Event.

I hope to see you there!

L Vaughn Spencer

I have been ridiculously busy with the manuscript for the new book.  As a result, my blogging has slowed down.  So, for today’s entry, I want to share with you an interview between a good friend of mine and The Economist.  I’m sure you will agree that L-Vo, as he likes to be called, is brilliant.

« Previous Page — Next Page »