The Value of Innovation
Description
More details
Other editions
Additional editions
Persons
Jack J. Phillips, PhD is Chairman of ROI Institute and a world-renowned expert on measurement and evaluation. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than a hundred books and hundreds of articles. His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN.
Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journals.
Content
Preface xix
Acknowledgements xxiii
About the Authors xxv
1 The Importance and Challenges of Innovation 1
Innovation Hype 2
Articles 2
Books 2
Jobs 3
Speeches 3
Experience 3
The Realities of Innovation 4
Innovation is Not New 4
Innovation is Necessary for Survival 5
Innovation is Equated with Success 5
Innovation is Truly Global 6
Consumers and Investors Expect Innovation 6
Innovation is Often Disruptive 6
Innovation is Not a Single Event 7
Little Ideas Often Make a Big Difference 7
Innovation Comes in Many Types and Forms 8
Innovation Spans Many Different Horizons 8
Trouble in Paradise: The Misconceptions 9
Misconception 1: Small Companies are More Innovative 10
Misconception 2: Uncontested Markets are Good for Innovation 10
Misconception 3: Spending More on R&D Increases Innovation 10
Misconception 4: Companies Need More Radical Innovation 10
Misconception 5: Open Innovation Turbocharges R&D 11
Misconception 6: R&D Needs to be More Relevant 11
Misconception 7: Wall Street Rewards Innovation 11
Innovation Challenges 12
Innovation is Expensive 12
Managing Innovation is Difficult 13
An Innovation Culture is Necessary for Success 13
Innovation Requires Many Personas 14
Innovation Success Rates Need to Improve 16
The Value of Innovation is Unclear 16
Final Thoughts 17
2 Status and Concerns about Innovation Measurement 19
Innovation: Definition, Models, and Measures 20
Sources of Innovation 21
Measurement Shifts 23
Measurement Shifts are Common 23
Value Perception. 24
The Search for Money 24
Hoping, Knowing, Proving, and Showing Value 25
Innovation is Systematic 25
Macro View of Measurement 27
Industry Level Measures 29
Company Level 30
Concerns about Company Level Measures 30
Micro View of Measurement 32
Final Thoughts 34
3 The Case for a New System 35
Innovation: A Cost or an Investment? 36
The Value of Innovation: A Summary 38
Intangibles and the Fear of not Investing 38
Relationship Between Variables 39
ROI Studies 41
Types of Data 41
Inputs 41
Reaction and Planned Action 42
Learning 43
Application and Implementation 43
Impact 44
Return on Investment 44
How Does Your Current System Stack Up? 45
Focus of Use 45
Standards 47
Contents vii
Types of Data 47
Dynamic Adjustments 47
Connectivity 48
Approach 48
Conservative Nature 48
Simplicity 48
Theoretical Foundation 49
Acceptance 49
Using Design Thinking to Deliver and Measure Results 49
Start with Why: Aligning Projects with the Business 50
Make it Feasible: Selecting the Right Solution 51
Expect Success: Designing for Results 52
Make it Matter: Designing for Input, Reaction, and Learning 52
Make it Stick: Designing for Application and Impact 53
Make it Credible: Measuring Results and Calculating ROI 53
Tell the Story. Communicating Results to Key Stakeholders 54
Optimize the Results: Using Black Box Thinking to Increase Funding 54
Requirements for the Value of Innovation: A Measurement Process 55
ROI Measurement Methodology 56
Terminology: Projects, Solutions, Participants . . . 57
Final Thoughts 57
4 Introducing the ROI Methodology 59
The ROI Methodology 60
Types of Data 60
The Initial Analysis 63
The ROI Process Model 65
Planning the Evaluation 66
Evaluation Purpose 66
Feasibility 67
Data Collection Plan 68
ROI Analysis Plan 68
Project Plan 71
Collecting Data 71
Isolating the Effects of the Project 72
Converting Data to Monetary Values 72
Identifying Intangible Benefits 73
Tabulating Project Costs 74
Calculating the Return on Investment 74
Reporting 75
Operating Standards and Philosophy 75
Implementing and Sustaining the Process 76
Benefits of This Approach 76
Aligning with Business 77
Validating the Value Proposition 77
Improving Processes 77
Enhancing the Image and Building Respect 78
Improving Support 78
Justifying or Enhancing Budgets 78
Building a Partnership with Key Executives 79
Earning a Seat at the Table 79
Final Thoughts 79
5 Aligning Innovation Projects to the Organization 81
Creating Business Alignment 83
The Purpose of Alignment 83
Disciplined Analysis 84
Determining the Potential Payoff 86
Obvious Versus not-so-obvious Payoff 87
The Cost of a Problem 89
The Value of an Opportunity 90
To Forecast or not to Forecast? 90
Determining Business Needs 90
The Opportunity 91
Hard Data Measures 91
Soft Data Measures 92
Tangible versus Intangible Benefits: A Better Approach 93
Impact Data Sources 94
Determining Performance Needs 95
Analysis Techniques 95
A Sensible Approach 95
Determining Learning Needs 96
Determining Preference Needs 97
Case Study: Southeast Corridor Bank 98
Payoff Needs 98
Business Needs 99
Performance Needs 99
The Solution 100
Learning Needs 101
Preference Needs 102
Developing Objectives for Innovation Projects 102
Reaction Objectives 102
Learning Objectives 103
Application and Implementation Objectives 103
Impact Objectives 104
ROI Objectives 105
Final Thoughts 106
6 Collecting Data Along Chain of Impact with a Toolbox of Methods 107
Questionnaires and Surveys 109
Types of Questions and Statements 109
Design Issues 110
A Detailed Example 111
Improving the Response Rate for Questionnaires and Surveys 113
Using Interviews 120
Types of Interviews 121
Interview Guidelines 121
Using Focus Groups 122
Applications for Evaluation 122
Guidelines 123
Measuring with Tests 124
Measuring with Simulation 124
Task Simulation 124
Role-Playing/Skill Practice 125
Using Observation 125
Guidelines for Effective Observation 125
Observation Methods 127
Using Action Plans 128
Using Action Plans Successfully 129
Advantages/Disadvantages of Action Plans 132
Using Performance Contracts 133
Monitoring Business Performance Data 134
Existing Measures 134
Developing New Measures 135
Selecting the Appropriate Method for Each Level 135
Type of Data 135
Participants' Time for Data Input 136
Manager Time for Data Input 136
Cost of Method 137
Disruption of Normal Work Activities 137
Accuracy of Method 137
Utility of an Additional Method 137
Cultural Bias for Data Collection Method 138
Final Thoughts 138
7 Measuring Reaction and Perceived Value 139
Why Measure Reaction and Perceived Value? 140
Customer Satisfaction 141
Immediate Adjustments 141
Predictive Capability 141
Important but not Exclusive 142
Sources of Data 143
Participants 143
Participant Managers 143
Other Team Members 143
Internal or External Customers 144
Project Leaders and Team Members 144
Sponsors and Senior Managers 144
Records and Previous Studies 144
Areas of Feedback 145
Data Collection Timing 146
Data Collection Methods 146
Questionnaires and Surveys 146
Interviews 147
Focus Groups 147
Using Reaction Data 147
Final Thoughts 148
8 Measuring Learning 149
Why Measure Learning and Confidence? 150
The Importance of Intellectual Capital 151
The Learning Organization 152
The Compliance Issue 152
The Use and Development of Competencies 152
The Role of Learning in Innovation Projects 153
The Challenges and Benefits of Measuring Learning 153
Challenges 154
The Benefits of Measuring Learning 154
Measurement Issues 155
Project Objectives 155
Typical Measures 155
Timing 156
Data Collection Methods 157
Questionnaires and Surveys 157
Performance Tests 157
Technology and Task Simulations 158
Case Studies 159
Role-Playing and Skill Practice 159
Informal Assessments 159
Administrative Issues 160
Reliability and Validity 160
Consistency 161
Pilot Testing 161
Scoring and Reporting 161
Using Learning Data 162
Final Thoughts 162
9 Measuring Application and Implementation 163
Why Measure Application and Implementation? 165
Information Value 165
Project Focus 166
Problems and Opportunities 166
Reward Effectiveness 167
Challenges 167
Linking with Learning 168
Building Data Collection into the Project 168
Ensuring a Sufficient Amount of Data 168
Addressing Application Needs at the Outset 169
Measurement Issues 169
Methods 169
Objectives 170
Areas of Coverage 170
Data Sources 170
Timing 170
Responsibilities 171
Data Collection Methods 171
Using Questionnaires to Measure Application and Implementation 172
Using Interviews, Focus Groups, and Observation 172
Using Action Plans 172
Barriers to Application 174
Application Data Use 174
Final Thoughts 175
10 Measuring Impact 177
Why Measure Business Impact? 178
Higher-Level Data 178
A Business Driver for Projects 179
"The Money" for Sponsors 179
Easy to Measure 180
Collecting Effective Impact Measures 180
Data Categories 180
Metric Fundamentals 181
Identifying Specific Measures Linked to Projects 182
Business Performance Data Monitoring 183
Identify Appropriate Measures 184
Convert Current Measures to Usable Ones 184
Develop New Measures 184
Data Collection Methods 185
Using Action Plans to Develop Business Impact Data 185
Using Performance Contracts to Measure Business Impact 187
Using Questionnaires to Collect Business Impact Measures 189
Measuring the Hard to Measure 190
Everything Can Be Measured 190
Perceptions are Important 191
Every Measure Can Be Converted to Money, but not Every Measure Should Be 191
Special Emphasis on Intangibles 192
Final Thoughts 192
11 Isolating the Effects of Innovation 193
Why the Concern over this Issue? 196
Reality 196
Myths 196
Preliminary Issues 198
Chain of Impact 198
Identify other Factors: A First Step 199
Isolation Methods 200
Control Groups 200
Trend Line Analysis 203
Mathematical Modeling 205
Estimates 206
Participants' Estimate of Impact 206
Manager's Estimate of Impact 209
Customer Estimates of Project Impact 209
Internal or External Expert Estimates 210
Estimate Credibility: The Wisdom of Crowds 210
Calculate the Impact of other Factors 212
Select the Technique 213
Final Thoughts 214
12 Converting Data to Money 215
Why Convert Data to Monetary Values? 217
Value Equals Money 217
Impact is More Understandable 217
Converting to Monetary Values is Similar to Budgeting 218
Monetary Value is Vital to Organizational Operations 218
Monetary Values are Necessary to Understand
Problems and Cost Data 219
Key Steps in Converting Data to Money 219
Standard Monetary Values 222
Converting output Data to Money 222
Calculating the Cost of Quality 223
Converting Employee Time Using Compensation 227
Finding Standard Values 228
When Standard Values are not Available 229
Using Historical Costs from Records 229
Time 229
Availability 230
Access 230
Accuracy 230
Using Input from Experts 230
Using Values from External Databases 231
Linking with other Measures 232
Using Estimates from Participants 233
Using Estimates from the Management Team 233
Using Project Staff Estimates 234
Technique Selection and Finalizing Value 234
Choose a Technique Appropriate for the Type of Data 235
Move from Most Accurate to Least Accurate 235
Consider Source Availability 235
Use the Source with the Broadest Perspective on the Issue 236
Use Multiple Techniques When Feasible 236
Apply the Credibility Test 236
Consider the Possibility of Management Adjustment 238
Consider the Short-Term/Long-Term Issue 238
Consider an Adjustment for the Time Value of Money 239
Final Thoughts 239
13 Addressing Intangibles 241
Why Intangibles are Important 244
Intangibles are the Invisible Advantage 244
We are Entering the Intangible Economy 245
More Intangibles are Converted to Tangibles 245
Intangibles Drive Innovation Projects 246
The Magnitude of the Investment 246
Measuring and Analyzing Intangibles 246
Measuring the Intangibles 247
Converting to Money 249
Identifying and Collecting Intangibles 251
Analyzing Intangibles 252
Final Thoughts 253
14 Measuring ROI 255
Why Monitor Costs and Measure ROI? 258
Fundamental Cost Issues 259
Fully Loaded Costs 259
Costs Reported without Benefits 260
Develop and Use Cost Guidelines 261
Sources of Costs 262
Prorated versus Direct Costs 262
Employee Benefits Factor 263
Specific Costs to Include 263
Initial Analysis and Assessment 264
Development of Project Solutions 264
Acquisition Costs 264
Implementation Costs 264
Maintenance and Monitoring 265
Support and Overhead 265
Evaluation and Reporting 265
The ROI Calculation 265
Benefits/Costs Ratio 266
ROI Formula 267
ROI Objective 269
Other ROI Measures 270
Payback Period (Breakeven Analysis) 270
Discounted Cash Flow 271
Internal Rate of Return 271
Final Thoughts 272
15 Forecasting Value, Including ROI 273
Why Forecast ROI? 278
Expensive Projects 279
High Risks and Uncertainty 279
Postproject Comparison 279
Compliance 280
The Trade-offs of Forecasting 280
Preproject ROI Forecasting 282
Basic Model 282
Basic Steps to Forecast ROI 283
Sources of Expert Input 287
Securing Input 287
Conversion to Money 288
Estimate Project Costs 288
Case Study 289
Forecasting with a Pilot Program 293
Forecasting with Reaction Data 293
Case Study: Forecasting ROI from Reaction Data 294
Use of the Data 295
Forecasting Guidelines 296
Final Thoughts 299
16 Reporting Results 301
The Importance of Communicating Results? 303
Communication is Necessary to Make Improvements 303
Communication is Necessary to Explain the Contribution 303
Communication is a Politically Sensitive Issue 304
Different Audiences Need Different Information 304
Principles of Communicating Results 304
Communication Must Be Timely 305
Communication Should Be Targeted to Specific Audiences 305
Media Should Be Carefully Selected 305
Communication Should Be Unbiased and Modest in Tone 305
Communication Must Be Consistent 306
Make the Message Clear 306
Testimonials Must Come from Respected Individuals 306
The Audience's Bias of the Project Will Influence the Communication Strategy 306
Storytelling is Essential 307
The Process for Communicating Results 307
The Need for Communication 308
The Communication Plan 309
The Audience for Communications 309
Basis for Selecting the Audience 311
Information Development: The Impact Study 312
Media Selection 312
Meetings 312
Interim and Progress Reports 314
Routine Communication Tools 315
E-mail and Electronic Media 316
Project Brochures and Pamphlets 316
Case Studies 316
Delivering the Message 316
Routine Feedback on Project Progress 317
Storytelling 319
Presentation of Results to Senior Management 320
Reactions to Communication 322
Final Thoughts 322
17 Implementing and Sustaining ROI 323
Why is this Important? 324
Resistance is Always Present 326
Implementation is the Key to Success 326
Consistency is Needed 326
Efficiency 326
Value is Maximized 326
Implementing the Process: Overcoming Resistance 327
Review Current Results 328
Developing Roles and Responsibilities 328
Identifying a Champion 329
Developing the ROI Leader 329
Establishing a Task Force 329
Assigning Responsibilities 330
Establishing Goals and Plans 331
Setting Evaluation Targets 331
Developing a Plan for Implementation 332
Revising or Developing Policies and Guidelines 332
Preparing the Project Team 334
Involving the Project Team 334
Using ROI as a Learning and Project Improvement Tool 334
Teaching the Team 334
Initiating ROI Studies 335
Selecting the Initial Project 335
Developing the Planning Documents 335
Reporting Progress 336
Establishing Discussion Groups 336
Preparing the Sponsors and Management Team 336
Removing Obstacles 337
Dispelling Myths 337
Delivering Bad News 338
Using the Data 338
Monitoring Progress 339
Final Thoughts 340
References 343
Index 351
Chapter 1
The Importance and Challenges of Innovation
Clifton Leaf, editor-in-chief of Fortune magazine borrowed a stranger's silver and orange bicycle and rode it two kilometers. When he was finished riding, he leaned the bike against a street lamp at a city intersection. Clifton was benefiting from the remarkable business model of Mobike, a Beijing-based start-up whose more than 100 million registered users do much the same thing an average of 20 times a day. This is more than three times the rate of use by other types of ride-share bikes. Many cities have bike-sharing where users typically pay to release the bike from a docking station and return it to another docking station within a particular timeframe. Mobike has eliminated the cumbersome docking process entirely. A user downloads an app, finds a bike nearby, and scans a QR code to unlock it. After using the bike, the user drops off the bike wherever they would like, because GPS and other wireless technologies are built into the bike's chassis, allowing the company to track its whereabouts. A smart-locking system bolts the rear tire in place until the next user shows up [1].
The old dock-based sharing systems are like the first-generation PCs. According to Davis Wang, CEO and co-founder of Mobike, while first-generation PCs were attached to desktops, Mobikes, in contrast, are like smartphones-you can take them anywhere you would like. Today, Mobike has more users than Uber, although it has been in business only two years. While few consumers outside of China have likely even heard of this ride-sharing business, the company has already expanded to Singapore, the United Kingdom, and the United States.
Innovation Hype
Innovation is everywhere and is constantly being brought to our attention. In the media, stories, ads, interviews, chats, blogs, and references dominate. It's hard to read articles, stories, or see advertisements without the term innovation being mentioned. Everything we see is innovation and everything that is developed is innovative. We even have innovative leaders now.
Articles
Innovation articles are everywhere, dominating the print space. Fast Company magazine, which has been a driving force for promoting innovation, devotes a good portion of its magazine to the subject of innovation, as do other major publications such as Harvard Business Review, Wall Street Journal, Bloomberg Businessweek and Forbes, among others. There is even a Chief Innovation Officer Magazine dedicated to those who are leading this effort.
Books
Innovation books now dominate the business book scene. A few years back, that wasn't the case. Famed management guru Peter Drucker wrote one of the earliest books on innovation in 1985 when he authored Innovation and Entrepreneurship: Practice and Principles.[2] Tom Peters, legendary best-selling business book author, is another successful innovation author who jumped into the arena with his book, The Circle of Innovation: You Can't Shrink Your Way to Greatness. This book put innovation on the map for many readers and business people, as he complained that not enough space had been devoted to this important area. Perhaps the person who has contributed the most is Tom Kelley, a partner in IDEO, one of the world's leading innovation and design firms, who contributed The Art of Innovation, The Ten Faces of Innovation, and Creative Confidence, the latter co-authored with David Kelley. The person who has contributed the most about disruption from innovation is Clayton M. Christensen. His classic book, The Innovators Dillema: When New Technologies Cause Great Firms to Fail, showed not only the power of innovation but also the many outcomes of the innovation process. Even Google founder Eric Schmidt got into the innovation fray with his book, How Google Works. CEOs have also popped into the arena with books ranging from Phil Knight (CEO of Nike), author of Shoe Dog, to Steve Case (former CEO of AOL Time Warner), who wrote The Innovation Blind Spot and The Third Wave. Clearly, the subject of innovation is now one of the most documented and written about areas in the business book market. For example, at Stanford University Press, in the business book category, six of the 23 titles for 2017 were in the innovation area. Amazon now has over 100 pages of books in the innovation category.
Jobs
Job titles have emerged for those responsible for innovation as the responsibility is shifting to one or more individuals. The title of Chief Innovation Officer, VP of Innovation, Director of Business Innovation, and Chief Innovation and Engagement Officer point to a growing desire to have leaders of innovation within organizations. Further down the organizational chain, we have innovation managers, innovation directors, and even innovation champions. A website, www.cio.com, is dedicated to providing resources to these individuals in this job category.
Speeches
You would be hard pressed to see a speech from a top executive recapping the events and success of the organization without a mention of innovation. State-of-the-company addresses and end-of-year summaries are almost always laced with innovation progress, updates, and issues. Politicians from presidents to mayors have proclaimed innovation to be an important part of their campaigns and strategies and have created more innovation for the government. Dubai has made a commitment to be the most innovative city. In Malaysia, there is an appointed Minister of Innovation.
Experience
You don't have to be reading books, articles, or listening to speeches to be innovative. We all experience innovation every day. Imagine the complete Uber experience compared to the use of a taxi, or the timeliness and efficiency of shopping with Amazon.com, or the amazing innovation that goes into Google's ability to find all the information we would need, not to mention the innovative efforts of organizations like Facebook, Netflix, and Apple. In our daily lives, we are experiencing innovation in a big way-in almost everything we do.
We love innovation because it often brings convenience. Sometimes it makes an entire process feel seamless, as in the Mobike example presented earlier in the chapter. In many cases, it reduces the cost of what we are doing or buying, such as with Amazon. Sometimes it is environmentally friendly, like the use of a ride-sharing program. At other times it makes us healthy, such as the medical devices that keep us alive. Few topics have enjoyed as much hype as innovation, and we've come to expect innovation and appreciate it.
The Realities of Innovation
Although there is much hype surrounding innovation, there are some realities of innovation that point to opportunities and concerns.
Innovation is Not New
Innovation has been around for many years with inventions traced to the ancient Chinese and Greek cultures. Innovation has been progressing for a very long time, but it has mushroomed in the last 50 years. Now, each year it seems to increase at a more rapid rate. Sometimes innovation is older than it looks. For example, Lockheed Martin developed a cargo aircraft, the C5A. At the time of its development, it was the world's largest airplane, much larger than the passenger aircraft Boeing 747. The C5A is huge, but also very versatile with the capability of tanks and large trucks driving through the front of the plane and out the rear. It has the capability to land and take off on unimproved runways. More impressively, it has the capability of taking off, flying, and landing entirely on autopilot. In 1969, over the mountains of North Georgia, the C5A made its first flight completely automated from takeoff to landing. In most of today's aircraft, autopilots are common. Still, many are surprised to know that the capability exists to take off, fly the entire plane, and land without a pilot. What makes this story interesting is that this capability was demonstrated almost 50 years ago. So innovation is not new, but the rate of change is rapidly increasing.
Innovation is Necessary for Survival
Figure 1.1 lists several companies that have had a near-death experience or have actually died, based primarily on their failure to innovate. Unfortunately, there are many familiar suspects on the list. In 1935, the expected life span of a company was 90 years. By 2005 this had fallen to 15 years. Yet at the same time, many companies have survived for over 100 years because they have constantly innovated.[3] For example, Johnson & Johnson, IBM, American Express, 3M, P&G, and Goodrich are on average about 100 years old, and they are still going strong. Innovation is essential; without it, it is almost certain that the company will not exist.
Figure 1.1 What is True of All These Companies?
Adapted from: Lead and Disrupt: How to Solve the Innovator's Dilemma (Charles A. O'Reilly III and Michael L. Tushman)
Innovation is Equated with Success
If something is successful, it must be innovative, and usually is. For example, on a recent flight on Delta Airlines, we noticed that Sky magazine, Delta's in-flight publication, had a feature on the founder of Delta with the title, "Spirit of Innovation." In this article, Delta was celebrating its success since its founding in 1929. Delta is now one of the largest, most admired, and most profitable airlines with 80,000 employees. They owe much of their...
