Isolation of Results
Defining the Impact of the Program
Published on 1. February 2008
Book
Paperback/Softback
160 pages
978-0-7879-8719-0 (ISBN)
Description
Isolation of Results Isolation of Results is the third of six books in the Measurement and Evaluation Series from Pfeiffer. The proven ROI Methodology--developed by the ROI Institute--provides a practical system for evaluation planning, data collection, data analysis, and reporting. All six books in the series offer the latest tools, most current research, and practical advice for measuring ROI in a variety of settings. To bring credibility to a project it is vital to isolate the effects of a program on business data. Isolation of Results focuses on this critical topic, arguably the most valuable part of the ROI Methodology. The authors acknowledge that other factors can influence results, and this important resource shows a variety of ways in which the effects of the program can be isolated from other influences. The techniques presented include the most reliable and rigorous approaches and also contain the more subjective methods. Isolation of Results offers a comprehensive review of the topic and contains information about using control group arrangements, trend line analysis, forecasting, expert estimation, and adjustments.
The book also addresses issues of credibility of the method.
The book also addresses issues of credibility of the method.
More details
Series
Language
English
Place of publication
New York
United States
Publishing group
John Wiley & Sons Inc
Target group
Professional and scholarly
Dimensions
Height: 228 mm
Width: 154 mm
Thickness: 10 mm
Weight
232 gr
ISBN-13
978-0-7879-8719-0 (9780787987190)
Copyright in bibliographic data is held by Nielsen Book Services Limited or its licensors: all rights reserved.
Schweitzer Classification
Persons
Jack J. Phillips is chairman of the ROI Institute, Inc. A world-renowned expert on measurement and evaluation, Phillips provides consulting services to Fortune 500 companies and workshops for major conference providers throughout the world. Phillips is the author or editor of more than thirty books and more than one hundred articles. Bruce C. Aaron is responsible for research and evaluation of learning and knowledge management initiatives within Accenture, a global management consulting, technology services, and outsourcing company. During his tenure, Accenture Education has received several awards from professional associations for excellence in measurement and evaluation, including the 2004 ASTD ROI Impact Study of the Year award. The ROI Institute, Inc., is a benchmarking, research, and information sharing organization that provides consulting services, workshops, and certification in the ROI Methodology. Widely considered the leading authority on evaluation and measurement of learning and development in organizations, the ROI Institute conducts workshops and offers certification for thousands of practitioners through a variety of strategic partners.
Content
Acknowledgments. Principles of the ROI Methodology. 1. The Importance of Isolating the Effects of Programs. Challenges in Understanding a Program's Impact. Case Study: What Caused the Improvement? Preliminary Issues in Isolating Program Effects. Pager: Please do not italicize the Contents H1 items or the sublists to this level. The Need to Isolate Program Effects. Chain of Impact: Initial Evidence of Program Effects. Identification of Factors Other Than the Program: A First Step. Final Thoughts. 2. Use of Control Groups. Control Group Design. Threats to Validity. Basic Control Group Design. Ideal Experimental Design. Posttest-Only Control Group Design. Which Design to Choose. Issues When Considering Control Groups. Viability. Practicality. Ethicial Considerations. Problems with Control Groups: A Case Example. Feasibility. Control Group Example 1: Retail Merchandise Company. Setting. Audience. Solution. Measures That Matter. Selection Criteria. Size of Groups. Duration of Experiment. Control Group Example 2: Federal Information Agency. Setting. Audience. Solution. Measures That Matter. Selection Criteria. Size of Groups. Duration of Experiment. Control Group Example 3: Midwest Electric, Inc. Setting. Needs Assessment. Audience. Solution. Measures That Matter. Selection Criteria. Size of Groups. Duration of Experiment. Control Group Example 4: International Software Company. Setting. Audience. Solution. Measures That Matter. Selection Criteria. Size of Groups. Duration of Experiment. Final Thoughts. 3. Use of Trend Lines and Forecasts. Trend Lines. Forecasts. Trend Line Analysis Example 1: Micro Electronics. Setting. Audience. Solution. Measures That Matter. Conditions Test. Trend Line Analysis Example 2: Healthcare, Inc. Setting. Audience. Solution. Measures That Matter. Conditions Test. Trend Line Analysis Example 3: National Book Company. Setting. Audience. Solution. Measures That Matter. Conditions Test. Final Thoughts. 4. Use of Expert Estimates. Participants' Estimates of Program Impact. Using Focus Groups to Obtain Participant Estimates. Using Questionnaires to Obtain Participant Estimates. Using Interviews to Obtain Participant Estimates. Advantages and Disadvantages of Participant Estimates. Case Study. Pager: Please style the following items as a sublist to the previous list. Setting. Audience. Solution. Measures. Estimates Provided. Credibility Check. Methodology. Pager: End of sublist. Immediate Managers' Estimates of Program Impact. Senior Management's Estimates of Program Impact. Customers? Estimates of Program Impact. Experts' Estimates of Program Impact. Determining the Impact of Other Factors. Estimate Example 1: Global Financial Services. Setting. Audience and Solution. Measures. Estimates Provided. Estimate Example 2: Cracker Box. Setting. Audience and Solution. Measures. Estimates Provided. Estimate Example 3: Public Bank of Malaysia. Setting. Audience and Solution. Measure. Estimates Provided. Estimate Example 4: Multi-National, Inc. Setting. Audience, Solution, and Measures. Estimates Provided. The Power of Estimates. Research. A Demonstration. Participant Reaction. Management Reaction. The Wisdom of Crowds. Key Issues in Using Estimates. Final Thoughts. 5. Use of Isolation Techniques. Matching Exercise: Isolating the Effects of a Program. Case Study. National Computer Company, Part A. Pager: Please style the following items as a sublist. Questions for Discussion. Responses. Pager: End of sublist. National Computer Company, Part B. Pager: Please style the following items as a sublist. Questions for Discussion. Responses. Pager: End of sublist. National Computer Company, Part C. Pager: Please style the following items as a sublist. Questions for Discussion. Responses. Pager: End of sublist. National Computer Company, Part D. Pager: Please style the following items as a sublist. Questions for Discussion. Responses. Pager: End of sublist. Why Isolation Is a Key Issue. Other Factors Are Always There. Without It, There Is No Business Link: Evidence Versus Proof. Other Factors and Influences Have Protective Owners. To Do It Right Is Not Easy. Without It, the Study Is Not Valid. Isolation Myths. Build Credibility with the Isolation Process. Selecting the Technique. Using Multiple Methods. Building Credibility. Final Thoughts. Index. About the Authors.