The Value of Innovation
Knowing, Proving, and Showing the Value of Innovation and Creativity
1st Edition
Published on 30. January 2018
Book
Hardback
390 pages
978-1-119-24237-6 (ISBN)
Description
Innovation is the life blood of practically every organization. Innovation drives growth, development, and prosperity for many organizations and geographical areas. Sometimes, innovation thrives within a certain geographical location or in certain organizations that are known for their innovative approaches. This outstanding new volume will demonstrate how to measure the success of innovation in all types of organizations.
In the last decade, there have been tremendous investments in creativity and innovations sponsored by companies, cities, states, countries, universities, NGO's, and even non-profits. With the magnitude of emphasis on creativity and innovation, the sponsors and key stakeholders will demand to know the value of these programs. The Value of Innovation: Measuring the Impact and ROI in Creativity and Innovation Programs will show step-by-step how to measure the impact and the ROI of innovation and creativity programs. The process collects six types of data: reaction, learning, application, impact, ROI, and intangibles. Data are collected analyzed and reported using a systematic, logic model. Conservative standards create results that are both CEO and CFO friendly. This proven process has been used now in 5000 organizations and this new book adapts the method directly to this critical area of innovation, showing examples and case studies.
In the last decade, there have been tremendous investments in creativity and innovations sponsored by companies, cities, states, countries, universities, NGO's, and even non-profits. With the magnitude of emphasis on creativity and innovation, the sponsors and key stakeholders will demand to know the value of these programs. The Value of Innovation: Measuring the Impact and ROI in Creativity and Innovation Programs will show step-by-step how to measure the impact and the ROI of innovation and creativity programs. The process collects six types of data: reaction, learning, application, impact, ROI, and intangibles. Data are collected analyzed and reported using a systematic, logic model. Conservative standards create results that are both CEO and CFO friendly. This proven process has been used now in 5000 organizations and this new book adapts the method directly to this critical area of innovation, showing examples and case studies.
More details
Language
English
Place of publication
United States
Publishing group
John Wiley & Sons Inc
Target group
Professional and scholarly
Product notice
sewn/stitched
Cloth over boards
Dimensions
Height: 236 mm
Width: 158 mm
Thickness: 32 mm
Weight
682 gr
ISBN-13
978-1-119-24237-6 (9781119242376)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
Additional editions
Jack J. Phillips | Patricia Pulliam Phillips
The Value of Innovation
Knowing, Proving, and Showing the Value of Innovation and Creativity
E-Book
12/2017
1st Edition
Wiley
€73.99
Available for download
Jack J. Phillips | Patricia Pulliam Phillips
The Value of Innovation
Knowing, Proving, and Showing the Value of Innovation and Creativity
E-Book
12/2017
1st Edition
Wiley
€70.99
Available for download
Persons
Jack J. Phillips, PhD is Chairman of ROI Institute and a world-renowned expert on measurement and evaluation. Phillips provides consulting services for Fortune 500 companies and workshops for major conference providers worldwide. Phillips is also the author or editor of more than a hundred books and hundreds of articles. His work has been featured in the Wall Street Journal, Bloomberg Businessweek, Fortune, and on CNN.
Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journals.
Patricia Pulliam Phillips, PhD is an internationally recognized author, consultant, and President and CEO of ROI Institute, Inc. Phillips provides consulting services to organizations worldwide. She helps organizations build capacity in the ROI Methodology by facilitating the ROI certification process and teaching the ROI Methodology through workshops and graduate level courses. She has been an author or editor of more than a hundred books and numerous journals.
Content
Preface xix
Acknowledgements xxiii
About the Authors xxv
1 The Importance and Challenges of Innovation 1
Innovation Hype 2
Articles 2
Books 2
Jobs 3
Speeches 3
Experience 3
The Realities of Innovation 4
Innovation is Not New 4
Innovation is Necessary for Survival 5
Innovation is Equated with Success 5
Innovation is Truly Global 6
Consumers and Investors Expect Innovation 6
Innovation is Often Disruptive 6
Innovation is Not a Single Event 7
Little Ideas Often Make a Big Difference 7
Innovation Comes in Many Types and Forms 8
Innovation Spans Many Different Horizons 8
Trouble in Paradise: The Misconceptions 9
Misconception 1: Small Companies are More Innovative 10
Misconception 2: Uncontested Markets are Good for Innovation 10
Misconception 3: Spending More on R&D Increases Innovation 10
Misconception 4: Companies Need More Radical Innovation 10
Misconception 5: Open Innovation Turbocharges R&D 11
Misconception 6: R&D Needs to be More Relevant 11
Misconception 7: Wall Street Rewards Innovation 11
Innovation Challenges 12
Innovation is Expensive 12
Managing Innovation is Difficult 13
An Innovation Culture is Necessary for Success 13
Innovation Requires Many Personas 14
Innovation Success Rates Need to Improve 16
The Value of Innovation is Unclear 16
Final Thoughts 17
2 Status and Concerns about Innovation Measurement 19
Innovation: Definition, Models, and Measures 20
Sources of Innovation 21
Measurement Shifts 23
Measurement Shifts are Common 23
Value Perception. 24
The Search for Money 24
Hoping, Knowing, Proving, and Showing Value 25
Innovation is Systematic 25
Macro View of Measurement 27
Industry Level Measures 29
Company Level 30
Concerns about Company Level Measures 30
Micro View of Measurement 32
Final Thoughts 34
3 The Case for a New System 35
Innovation: A Cost or an Investment? 36
The Value of Innovation: A Summary 38
Intangibles and the Fear of not Investing 38
Relationship Between Variables 39
ROI Studies 41
Types of Data 41
Inputs 41
Reaction and Planned Action 42
Learning 43
Application and Implementation 43
Impact 44
Return on Investment 44
How Does Your Current System Stack Up? 45
Focus of Use 45
Standards 47
Contents vii
Types of Data 47
Dynamic Adjustments 47
Connectivity 48
Approach 48
Conservative Nature 48
Simplicity 48
Theoretical Foundation 49
Acceptance 49
Using Design Thinking to Deliver and Measure Results 49
Start with Why: Aligning Projects with the Business 50
Make it Feasible: Selecting the Right Solution 51
Expect Success: Designing for Results 52
Make it Matter: Designing for Input, Reaction, and Learning 52
Make it Stick: Designing for Application and Impact 53
Make it Credible: Measuring Results and Calculating ROI 53
Tell the Story. Communicating Results to Key Stakeholders 54
Optimize the Results: Using Black Box Thinking to Increase Funding 54
Requirements for the Value of Innovation: A Measurement Process 55
ROI Measurement Methodology 56
Terminology: Projects, Solutions, Participants . . . 57
Final Thoughts 57
4 Introducing the ROI Methodology 59
The ROI Methodology 60
Types of Data 60
The Initial Analysis 63
The ROI Process Model 65
Planning the Evaluation 66
Evaluation Purpose 66
Feasibility 67
Data Collection Plan 68
ROI Analysis Plan 68
Project Plan 71
Collecting Data 71
Isolating the Effects of the Project 72
Converting Data to Monetary Values 72
Identifying Intangible Benefits 73
Tabulating Project Costs 74
Calculating the Return on Investment 74
Reporting 75
Operating Standards and Philosophy 75
Implementing and Sustaining the Process 76
Benefits of This Approach 76
Aligning with Business 77
Validating the Value Proposition 77
Improving Processes 77
Enhancing the Image and Building Respect 78
Improving Support 78
Justifying or Enhancing Budgets 78
Building a Partnership with Key Executives 79
Earning a Seat at the Table 79
Final Thoughts 79
5 Aligning Innovation Projects to the Organization 81
Creating Business Alignment 83
The Purpose of Alignment 83
Disciplined Analysis 84
Determining the Potential Payoff 86
Obvious Versus not-so-obvious Payoff 87
The Cost of a Problem 89
The Value of an Opportunity 90
To Forecast or not to Forecast? 90
Determining Business Needs 90
The Opportunity 91
Hard Data Measures 91
Soft Data Measures 92
Tangible versus Intangible Benefits: A Better Approach 93
Impact Data Sources 94
Determining Performance Needs 95
Analysis Techniques 95
A Sensible Approach 95
Determining Learning Needs 96
Determining Preference Needs 97
Case Study: Southeast Corridor Bank 98
Payoff Needs 98
Business Needs 99
Performance Needs 99
The Solution 100
Learning Needs 101
Preference Needs 102
Developing Objectives for Innovation Projects 102
Reaction Objectives 102
Learning Objectives 103
Application and Implementation Objectives 103
Impact Objectives 104
ROI Objectives 105
Final Thoughts 106
6 Collecting Data Along Chain of Impact with a Toolbox of Methods 107
Questionnaires and Surveys 109
Types of Questions and Statements 109
Design Issues 110
A Detailed Example 111
Improving the Response Rate for Questionnaires and Surveys 113
Using Interviews 120
Types of Interviews 121
Interview Guidelines 121
Using Focus Groups 122
Applications for Evaluation 122
Guidelines 123
Measuring with Tests 124
Measuring with Simulation 124
Task Simulation 124
Role-Playing/Skill Practice 125
Using Observation 125
Guidelines for Effective Observation 125
Observation Methods 127
Using Action Plans 128
Using Action Plans Successfully 129
Advantages/Disadvantages of Action Plans 132
Using Performance Contracts 133
Monitoring Business Performance Data 134
Existing Measures 134
Developing New Measures 135
Selecting the Appropriate Method for Each Level 135
Type of Data 135
Participants' Time for Data Input 136
Manager Time for Data Input 136
Cost of Method 137
Disruption of Normal Work Activities 137
Accuracy of Method 137
Utility of an Additional Method 137
Cultural Bias for Data Collection Method 138
Final Thoughts 138
7 Measuring Reaction and Perceived Value 139
Why Measure Reaction and Perceived Value? 140
Customer Satisfaction 141
Immediate Adjustments 141
Predictive Capability 141
Important but not Exclusive 142
Sources of Data 143
Participants 143
Participant Managers 143
Other Team Members 143
Internal or External Customers 144
Project Leaders and Team Members 144
Sponsors and Senior Managers 144
Records and Previous Studies 144
Areas of Feedback 145
Data Collection Timing 146
Data Collection Methods 146
Questionnaires and Surveys 146
Interviews 147
Focus Groups 147
Using Reaction Data 147
Final Thoughts 148
8 Measuring Learning 149
Why Measure Learning and Confidence? 150
The Importance of Intellectual Capital 151
The Learning Organization 152
The Compliance Issue 152
The Use and Development of Competencies 152
The Role of Learning in Innovation Projects 153
The Challenges and Benefits of Measuring Learning 153
Challenges 154
The Benefits of Measuring Learning 154
Measurement Issues 155
Project Objectives 155
Typical Measures 155
Timing 156
Data Collection Methods 157
Questionnaires and Surveys 157
Performance Tests 157
Technology and Task Simulations 158
Case Studies 159
Role-Playing and Skill Practice 159
Informal Assessments 159
Administrative Issues 160
Reliability and Validity 160
Consistency 161
Pilot Testing 161
Scoring and Reporting 161
Using Learning Data 162
Final Thoughts 162
9 Measuring Application and Implementation 163
Why Measure Application and Implementation? 165
Information Value 165
Project Focus 166
Problems and Opportunities 166
Reward Effectiveness 167
Challenges 167
Linking with Learning 168
Building Data Collection into the Project 168
Ensuring a Sufficient Amount of Data 168
Addressing Application Needs at the Outset 169
Measurement Issues 169
Methods 169
Objectives 170
Areas of Coverage 170
Data Sources 170
Timing 170
Responsibilities 171
Data Collection Methods 171
Using Questionnaires to Measure Application and Implementation 172
Using Interviews, Focus Groups, and Observation 172
Using Action Plans 172
Barriers to Application 174
Application Data Use 174
Final Thoughts 175
10 Measuring Impact 177
Why Measure Business Impact? 178
Higher-Level Data 178
A Business Driver for Projects 179
"The Money" for Sponsors 179
Easy to Measure 180
Collecting Effective Impact Measures 180
Data Categories 180
Metric Fundamentals 181
Identifying Specific Measures Linked to Projects 182
Business Performance Data Monitoring 183
Identify Appropriate Measures 184
Convert Current Measures to Usable Ones 184
Develop New Measures 184
Data Collection Methods 185
Using Action Plans to Develop Business Impact Data 185
Using Performance Contracts to Measure Business Impact 187
Using Questionnaires to Collect Business Impact Measures 189
Measuring the Hard to Measure 190
Everything Can Be Measured 190
Perceptions are Important 191
Every Measure Can Be Converted to Money, but not Every Measure Should Be 191
Special Emphasis on Intangibles 192
Final Thoughts 192
11 Isolating the Effects of Innovation 193
Why the Concern over this Issue? 196
Reality 196
Myths 196
Preliminary Issues 198
Chain of Impact 198
Identify other Factors: A First Step 199
Isolation Methods 200
Control Groups 200
Trend Line Analysis 203
Mathematical Modeling 205
Estimates 206
Participants' Estimate of Impact 206
Manager's Estimate of Impact 209
Customer Estimates of Project Impact 209
Internal or External Expert Estimates 210
Estimate Credibility: The Wisdom of Crowds 210
Calculate the Impact of other Factors 212
Select the Technique 213
Final Thoughts 214
12 Converting Data to Money 215
Why Convert Data to Monetary Values? 217
Value Equals Money 217
Impact is More Understandable 217
Converting to Monetary Values is Similar to Budgeting 218
Monetary Value is Vital to Organizational Operations 218
Monetary Values are Necessary to Understand
Problems and Cost Data 219
Key Steps in Converting Data to Money 219
Standard Monetary Values 222
Converting output Data to Money 222
Calculating the Cost of Quality 223
Converting Employee Time Using Compensation 227
Finding Standard Values 228
When Standard Values are not Available 229
Using Historical Costs from Records 229
Time 229
Availability 230
Access 230
Accuracy 230
Using Input from Experts 230
Using Values from External Databases 231
Linking with other Measures 232
Using Estimates from Participants 233
Using Estimates from the Management Team 233
Using Project Staff Estimates 234
Technique Selection and Finalizing Value 234
Choose a Technique Appropriate for the Type of Data 235
Move from Most Accurate to Least Accurate 235
Consider Source Availability 235
Use the Source with the Broadest Perspective on the Issue 236
Use Multiple Techniques When Feasible 236
Apply the Credibility Test 236
Consider the Possibility of Management Adjustment 238
Consider the Short-Term/Long-Term Issue 238
Consider an Adjustment for the Time Value of Money 239
Final Thoughts 239
13 Addressing Intangibles 241
Why Intangibles are Important 244
Intangibles are the Invisible Advantage 244
We are Entering the Intangible Economy 245
More Intangibles are Converted to Tangibles 245
Intangibles Drive Innovation Projects 246
The Magnitude of the Investment 246
Measuring and Analyzing Intangibles 246
Measuring the Intangibles 247
Converting to Money 249
Identifying and Collecting Intangibles 251
Analyzing Intangibles 252
Final Thoughts 253
14 Measuring ROI 255
Why Monitor Costs and Measure ROI? 258
Fundamental Cost Issues 259
Fully Loaded Costs 259
Costs Reported without Benefits 260
Develop and Use Cost Guidelines 261
Sources of Costs 262
Prorated versus Direct Costs 262
Employee Benefits Factor 263
Specific Costs to Include 263
Initial Analysis and Assessment 264
Development of Project Solutions 264
Acquisition Costs 264
Implementation Costs 264
Maintenance and Monitoring 265
Support and Overhead 265
Evaluation and Reporting 265
The ROI Calculation 265
Benefits/Costs Ratio 266
ROI Formula 267
ROI Objective 269
Other ROI Measures 270
Payback Period (Breakeven Analysis) 270
Discounted Cash Flow 271
Internal Rate of Return 271
Final Thoughts 272
15 Forecasting Value, Including ROI 273
Why Forecast ROI? 278
Expensive Projects 279
High Risks and Uncertainty 279
Postproject Comparison 279
Compliance 280
The Trade-offs of Forecasting 280
Preproject ROI Forecasting 282
Basic Model 282
Basic Steps to Forecast ROI 283
Sources of Expert Input 287
Securing Input 287
Conversion to Money 288
Estimate Project Costs 288
Case Study 289
Forecasting with a Pilot Program 293
Forecasting with Reaction Data 293
Case Study: Forecasting ROI from Reaction Data 294
Use of the Data 295
Forecasting Guidelines 296
Final Thoughts 299
16 Reporting Results 301
The Importance of Communicating Results? 303
Communication is Necessary to Make Improvements 303
Communication is Necessary to Explain the Contribution 303
Communication is a Politically Sensitive Issue 304
Different Audiences Need Different Information 304
Principles of Communicating Results 304
Communication Must Be Timely 305
Communication Should Be Targeted to Specific Audiences 305
Media Should Be Carefully Selected 305
Communication Should Be Unbiased and Modest in Tone 305
Communication Must Be Consistent 306
Make the Message Clear 306
Testimonials Must Come from Respected Individuals 306
The Audience's Bias of the Project Will Influence the Communication Strategy 306
Storytelling is Essential 307
The Process for Communicating Results 307
The Need for Communication 308
The Communication Plan 309
The Audience for Communications 309
Basis for Selecting the Audience 311
Information Development: The Impact Study 312
Media Selection 312
Meetings 312
Interim and Progress Reports 314
Routine Communication Tools 315
E-mail and Electronic Media 316
Project Brochures and Pamphlets 316
Case Studies 316
Delivering the Message 316
Routine Feedback on Project Progress 317
Storytelling 319
Presentation of Results to Senior Management 320
Reactions to Communication 322
Final Thoughts 322
17 Implementing and Sustaining ROI 323
Why is this Important? 324
Resistance is Always Present 326
Implementation is the Key to Success 326
Consistency is Needed 326
Efficiency 326
Value is Maximized 326
Implementing the Process: Overcoming Resistance 327
Review Current Results 328
Developing Roles and Responsibilities 328
Identifying a Champion 329
Developing the ROI Leader 329
Establishing a Task Force 329
Assigning Responsibilities 330
Establishing Goals and Plans 331
Setting Evaluation Targets 331
Developing a Plan for Implementation 332
Revising or Developing Policies and Guidelines 332
Preparing the Project Team 334
Involving the Project Team 334
Using ROI as a Learning and Project Improvement Tool 334
Teaching the Team 334
Initiating ROI Studies 335
Selecting the Initial Project 335
Developing the Planning Documents 335
Reporting Progress 336
Establishing Discussion Groups 336
Preparing the Sponsors and Management Team 336
Removing Obstacles 337
Dispelling Myths 337
Delivering Bad News 338
Using the Data 338
Monitoring Progress 339
Final Thoughts 340
References 343
Index 351
Acknowledgements xxiii
About the Authors xxv
1 The Importance and Challenges of Innovation 1
Innovation Hype 2
Articles 2
Books 2
Jobs 3
Speeches 3
Experience 3
The Realities of Innovation 4
Innovation is Not New 4
Innovation is Necessary for Survival 5
Innovation is Equated with Success 5
Innovation is Truly Global 6
Consumers and Investors Expect Innovation 6
Innovation is Often Disruptive 6
Innovation is Not a Single Event 7
Little Ideas Often Make a Big Difference 7
Innovation Comes in Many Types and Forms 8
Innovation Spans Many Different Horizons 8
Trouble in Paradise: The Misconceptions 9
Misconception 1: Small Companies are More Innovative 10
Misconception 2: Uncontested Markets are Good for Innovation 10
Misconception 3: Spending More on R&D Increases Innovation 10
Misconception 4: Companies Need More Radical Innovation 10
Misconception 5: Open Innovation Turbocharges R&D 11
Misconception 6: R&D Needs to be More Relevant 11
Misconception 7: Wall Street Rewards Innovation 11
Innovation Challenges 12
Innovation is Expensive 12
Managing Innovation is Difficult 13
An Innovation Culture is Necessary for Success 13
Innovation Requires Many Personas 14
Innovation Success Rates Need to Improve 16
The Value of Innovation is Unclear 16
Final Thoughts 17
2 Status and Concerns about Innovation Measurement 19
Innovation: Definition, Models, and Measures 20
Sources of Innovation 21
Measurement Shifts 23
Measurement Shifts are Common 23
Value Perception. 24
The Search for Money 24
Hoping, Knowing, Proving, and Showing Value 25
Innovation is Systematic 25
Macro View of Measurement 27
Industry Level Measures 29
Company Level 30
Concerns about Company Level Measures 30
Micro View of Measurement 32
Final Thoughts 34
3 The Case for a New System 35
Innovation: A Cost or an Investment? 36
The Value of Innovation: A Summary 38
Intangibles and the Fear of not Investing 38
Relationship Between Variables 39
ROI Studies 41
Types of Data 41
Inputs 41
Reaction and Planned Action 42
Learning 43
Application and Implementation 43
Impact 44
Return on Investment 44
How Does Your Current System Stack Up? 45
Focus of Use 45
Standards 47
Contents vii
Types of Data 47
Dynamic Adjustments 47
Connectivity 48
Approach 48
Conservative Nature 48
Simplicity 48
Theoretical Foundation 49
Acceptance 49
Using Design Thinking to Deliver and Measure Results 49
Start with Why: Aligning Projects with the Business 50
Make it Feasible: Selecting the Right Solution 51
Expect Success: Designing for Results 52
Make it Matter: Designing for Input, Reaction, and Learning 52
Make it Stick: Designing for Application and Impact 53
Make it Credible: Measuring Results and Calculating ROI 53
Tell the Story. Communicating Results to Key Stakeholders 54
Optimize the Results: Using Black Box Thinking to Increase Funding 54
Requirements for the Value of Innovation: A Measurement Process 55
ROI Measurement Methodology 56
Terminology: Projects, Solutions, Participants . . . 57
Final Thoughts 57
4 Introducing the ROI Methodology 59
The ROI Methodology 60
Types of Data 60
The Initial Analysis 63
The ROI Process Model 65
Planning the Evaluation 66
Evaluation Purpose 66
Feasibility 67
Data Collection Plan 68
ROI Analysis Plan 68
Project Plan 71
Collecting Data 71
Isolating the Effects of the Project 72
Converting Data to Monetary Values 72
Identifying Intangible Benefits 73
Tabulating Project Costs 74
Calculating the Return on Investment 74
Reporting 75
Operating Standards and Philosophy 75
Implementing and Sustaining the Process 76
Benefits of This Approach 76
Aligning with Business 77
Validating the Value Proposition 77
Improving Processes 77
Enhancing the Image and Building Respect 78
Improving Support 78
Justifying or Enhancing Budgets 78
Building a Partnership with Key Executives 79
Earning a Seat at the Table 79
Final Thoughts 79
5 Aligning Innovation Projects to the Organization 81
Creating Business Alignment 83
The Purpose of Alignment 83
Disciplined Analysis 84
Determining the Potential Payoff 86
Obvious Versus not-so-obvious Payoff 87
The Cost of a Problem 89
The Value of an Opportunity 90
To Forecast or not to Forecast? 90
Determining Business Needs 90
The Opportunity 91
Hard Data Measures 91
Soft Data Measures 92
Tangible versus Intangible Benefits: A Better Approach 93
Impact Data Sources 94
Determining Performance Needs 95
Analysis Techniques 95
A Sensible Approach 95
Determining Learning Needs 96
Determining Preference Needs 97
Case Study: Southeast Corridor Bank 98
Payoff Needs 98
Business Needs 99
Performance Needs 99
The Solution 100
Learning Needs 101
Preference Needs 102
Developing Objectives for Innovation Projects 102
Reaction Objectives 102
Learning Objectives 103
Application and Implementation Objectives 103
Impact Objectives 104
ROI Objectives 105
Final Thoughts 106
6 Collecting Data Along Chain of Impact with a Toolbox of Methods 107
Questionnaires and Surveys 109
Types of Questions and Statements 109
Design Issues 110
A Detailed Example 111
Improving the Response Rate for Questionnaires and Surveys 113
Using Interviews 120
Types of Interviews 121
Interview Guidelines 121
Using Focus Groups 122
Applications for Evaluation 122
Guidelines 123
Measuring with Tests 124
Measuring with Simulation 124
Task Simulation 124
Role-Playing/Skill Practice 125
Using Observation 125
Guidelines for Effective Observation 125
Observation Methods 127
Using Action Plans 128
Using Action Plans Successfully 129
Advantages/Disadvantages of Action Plans 132
Using Performance Contracts 133
Monitoring Business Performance Data 134
Existing Measures 134
Developing New Measures 135
Selecting the Appropriate Method for Each Level 135
Type of Data 135
Participants' Time for Data Input 136
Manager Time for Data Input 136
Cost of Method 137
Disruption of Normal Work Activities 137
Accuracy of Method 137
Utility of an Additional Method 137
Cultural Bias for Data Collection Method 138
Final Thoughts 138
7 Measuring Reaction and Perceived Value 139
Why Measure Reaction and Perceived Value? 140
Customer Satisfaction 141
Immediate Adjustments 141
Predictive Capability 141
Important but not Exclusive 142
Sources of Data 143
Participants 143
Participant Managers 143
Other Team Members 143
Internal or External Customers 144
Project Leaders and Team Members 144
Sponsors and Senior Managers 144
Records and Previous Studies 144
Areas of Feedback 145
Data Collection Timing 146
Data Collection Methods 146
Questionnaires and Surveys 146
Interviews 147
Focus Groups 147
Using Reaction Data 147
Final Thoughts 148
8 Measuring Learning 149
Why Measure Learning and Confidence? 150
The Importance of Intellectual Capital 151
The Learning Organization 152
The Compliance Issue 152
The Use and Development of Competencies 152
The Role of Learning in Innovation Projects 153
The Challenges and Benefits of Measuring Learning 153
Challenges 154
The Benefits of Measuring Learning 154
Measurement Issues 155
Project Objectives 155
Typical Measures 155
Timing 156
Data Collection Methods 157
Questionnaires and Surveys 157
Performance Tests 157
Technology and Task Simulations 158
Case Studies 159
Role-Playing and Skill Practice 159
Informal Assessments 159
Administrative Issues 160
Reliability and Validity 160
Consistency 161
Pilot Testing 161
Scoring and Reporting 161
Using Learning Data 162
Final Thoughts 162
9 Measuring Application and Implementation 163
Why Measure Application and Implementation? 165
Information Value 165
Project Focus 166
Problems and Opportunities 166
Reward Effectiveness 167
Challenges 167
Linking with Learning 168
Building Data Collection into the Project 168
Ensuring a Sufficient Amount of Data 168
Addressing Application Needs at the Outset 169
Measurement Issues 169
Methods 169
Objectives 170
Areas of Coverage 170
Data Sources 170
Timing 170
Responsibilities 171
Data Collection Methods 171
Using Questionnaires to Measure Application and Implementation 172
Using Interviews, Focus Groups, and Observation 172
Using Action Plans 172
Barriers to Application 174
Application Data Use 174
Final Thoughts 175
10 Measuring Impact 177
Why Measure Business Impact? 178
Higher-Level Data 178
A Business Driver for Projects 179
"The Money" for Sponsors 179
Easy to Measure 180
Collecting Effective Impact Measures 180
Data Categories 180
Metric Fundamentals 181
Identifying Specific Measures Linked to Projects 182
Business Performance Data Monitoring 183
Identify Appropriate Measures 184
Convert Current Measures to Usable Ones 184
Develop New Measures 184
Data Collection Methods 185
Using Action Plans to Develop Business Impact Data 185
Using Performance Contracts to Measure Business Impact 187
Using Questionnaires to Collect Business Impact Measures 189
Measuring the Hard to Measure 190
Everything Can Be Measured 190
Perceptions are Important 191
Every Measure Can Be Converted to Money, but not Every Measure Should Be 191
Special Emphasis on Intangibles 192
Final Thoughts 192
11 Isolating the Effects of Innovation 193
Why the Concern over this Issue? 196
Reality 196
Myths 196
Preliminary Issues 198
Chain of Impact 198
Identify other Factors: A First Step 199
Isolation Methods 200
Control Groups 200
Trend Line Analysis 203
Mathematical Modeling 205
Estimates 206
Participants' Estimate of Impact 206
Manager's Estimate of Impact 209
Customer Estimates of Project Impact 209
Internal or External Expert Estimates 210
Estimate Credibility: The Wisdom of Crowds 210
Calculate the Impact of other Factors 212
Select the Technique 213
Final Thoughts 214
12 Converting Data to Money 215
Why Convert Data to Monetary Values? 217
Value Equals Money 217
Impact is More Understandable 217
Converting to Monetary Values is Similar to Budgeting 218
Monetary Value is Vital to Organizational Operations 218
Monetary Values are Necessary to Understand
Problems and Cost Data 219
Key Steps in Converting Data to Money 219
Standard Monetary Values 222
Converting output Data to Money 222
Calculating the Cost of Quality 223
Converting Employee Time Using Compensation 227
Finding Standard Values 228
When Standard Values are not Available 229
Using Historical Costs from Records 229
Time 229
Availability 230
Access 230
Accuracy 230
Using Input from Experts 230
Using Values from External Databases 231
Linking with other Measures 232
Using Estimates from Participants 233
Using Estimates from the Management Team 233
Using Project Staff Estimates 234
Technique Selection and Finalizing Value 234
Choose a Technique Appropriate for the Type of Data 235
Move from Most Accurate to Least Accurate 235
Consider Source Availability 235
Use the Source with the Broadest Perspective on the Issue 236
Use Multiple Techniques When Feasible 236
Apply the Credibility Test 236
Consider the Possibility of Management Adjustment 238
Consider the Short-Term/Long-Term Issue 238
Consider an Adjustment for the Time Value of Money 239
Final Thoughts 239
13 Addressing Intangibles 241
Why Intangibles are Important 244
Intangibles are the Invisible Advantage 244
We are Entering the Intangible Economy 245
More Intangibles are Converted to Tangibles 245
Intangibles Drive Innovation Projects 246
The Magnitude of the Investment 246
Measuring and Analyzing Intangibles 246
Measuring the Intangibles 247
Converting to Money 249
Identifying and Collecting Intangibles 251
Analyzing Intangibles 252
Final Thoughts 253
14 Measuring ROI 255
Why Monitor Costs and Measure ROI? 258
Fundamental Cost Issues 259
Fully Loaded Costs 259
Costs Reported without Benefits 260
Develop and Use Cost Guidelines 261
Sources of Costs 262
Prorated versus Direct Costs 262
Employee Benefits Factor 263
Specific Costs to Include 263
Initial Analysis and Assessment 264
Development of Project Solutions 264
Acquisition Costs 264
Implementation Costs 264
Maintenance and Monitoring 265
Support and Overhead 265
Evaluation and Reporting 265
The ROI Calculation 265
Benefits/Costs Ratio 266
ROI Formula 267
ROI Objective 269
Other ROI Measures 270
Payback Period (Breakeven Analysis) 270
Discounted Cash Flow 271
Internal Rate of Return 271
Final Thoughts 272
15 Forecasting Value, Including ROI 273
Why Forecast ROI? 278
Expensive Projects 279
High Risks and Uncertainty 279
Postproject Comparison 279
Compliance 280
The Trade-offs of Forecasting 280
Preproject ROI Forecasting 282
Basic Model 282
Basic Steps to Forecast ROI 283
Sources of Expert Input 287
Securing Input 287
Conversion to Money 288
Estimate Project Costs 288
Case Study 289
Forecasting with a Pilot Program 293
Forecasting with Reaction Data 293
Case Study: Forecasting ROI from Reaction Data 294
Use of the Data 295
Forecasting Guidelines 296
Final Thoughts 299
16 Reporting Results 301
The Importance of Communicating Results? 303
Communication is Necessary to Make Improvements 303
Communication is Necessary to Explain the Contribution 303
Communication is a Politically Sensitive Issue 304
Different Audiences Need Different Information 304
Principles of Communicating Results 304
Communication Must Be Timely 305
Communication Should Be Targeted to Specific Audiences 305
Media Should Be Carefully Selected 305
Communication Should Be Unbiased and Modest in Tone 305
Communication Must Be Consistent 306
Make the Message Clear 306
Testimonials Must Come from Respected Individuals 306
The Audience's Bias of the Project Will Influence the Communication Strategy 306
Storytelling is Essential 307
The Process for Communicating Results 307
The Need for Communication 308
The Communication Plan 309
The Audience for Communications 309
Basis for Selecting the Audience 311
Information Development: The Impact Study 312
Media Selection 312
Meetings 312
Interim and Progress Reports 314
Routine Communication Tools 315
E-mail and Electronic Media 316
Project Brochures and Pamphlets 316
Case Studies 316
Delivering the Message 316
Routine Feedback on Project Progress 317
Storytelling 319
Presentation of Results to Senior Management 320
Reactions to Communication 322
Final Thoughts 322
17 Implementing and Sustaining ROI 323
Why is this Important? 324
Resistance is Always Present 326
Implementation is the Key to Success 326
Consistency is Needed 326
Efficiency 326
Value is Maximized 326
Implementing the Process: Overcoming Resistance 327
Review Current Results 328
Developing Roles and Responsibilities 328
Identifying a Champion 329
Developing the ROI Leader 329
Establishing a Task Force 329
Assigning Responsibilities 330
Establishing Goals and Plans 331
Setting Evaluation Targets 331
Developing a Plan for Implementation 332
Revising or Developing Policies and Guidelines 332
Preparing the Project Team 334
Involving the Project Team 334
Using ROI as a Learning and Project Improvement Tool 334
Teaching the Team 334
Initiating ROI Studies 335
Selecting the Initial Project 335
Developing the Planning Documents 335
Reporting Progress 336
Establishing Discussion Groups 336
Preparing the Sponsors and Management Team 336
Removing Obstacles 337
Dispelling Myths 337
Delivering Bad News 338
Using the Data 338
Monitoring Progress 339
Final Thoughts 340
References 343
Index 351