Program Evaluation and Performance Measurement

CHAPTER 1: KEY CONCEPTS AND ISSUES IN PROGRAM EVALUATION AND PERFORMANCE MEASUREMENT Introduction A Typical Program Evaluation: Assessing the Neighbourhood Integrated Service Team Program in Vancouver What is a Program? Key Concepts In program evaluation Key Evaluation Questions Formative and Summative Program Evaluations Ex Ante and Ex Post Program Evaluations Analyzing Cause and Effect Linkages in Program Evaluations The Process of Conducting a Program Evaluation Summary Discussion Questions for Chapter 1 References for Chapter 1 CHAPTER 2: UNDERSTANDING AND APPLYING PROGRAM LOGIC MODELS Introduction A Basic Logic Modelling Approach Logic Models That Categorize and Specify Intended Causal Linkages Applying the Framework for Constructing Logic Models Flow Charts Constructing Program Logics in Program Evaluations Program Technologies Program Objectives, Program Environments, and Organizational Objectives Strengths and Limitations of Program Logics Summary Discussion Questions Appendix 1: Applying What You Have Learned in Chapter Two: Developing a Logic Model for the Compass Program Answer Key References for Chapter 2 CHAPTER 3: RESEARCH DESIGNS FOR PROGRAM EVALUATIONS Introduction What is Research Design? Why Pay Attention to Experimental Designs? Using Experimental Designs to Evaluate Programs: The ELmira Nurse Home Visitation Program Establishing Validity in Research Designs Defining and Working with the Four Kinds of Validity Testing the Causal Linkages in Program Logic Models Research Designs and Performance Measurement Summary Discussion Questions References for Chapter 3 CHAPTER 4: MEASUREMENT IN PROGRAM EVALUATION Introduction Measurement Procedures Illustrating Measurement Terminology Measurement Validity Levels of Measurement Units of Analysis Sources of Data in Program Evaluations and Performance Measurement Systems Using Surveys to Estimate the Incremental Effects of Programs Survey Designs and Research Designs Validity of Measures and the Validity of Causes and Effects Summary Discussion Questions References for Chapter 4 CHAPTER 5: APPLYING QUALITATIVE EVALUATION METHODS Introduction Comparing and Contrasting Different Approaches to Qualitative Evaluation Qualitative Evaluation Methods: Some Basics Structuring Qualitative Program Evaluations The Credibility and Generalizability of Qualitative Findings Connecting Qualitative Evaluation Methods to Performance Measurement The Power of Case Studies Summary Discussion Questions CHAPTER 6: ASSESSING THE NEED FOR PROGRAMS Introduction What Are Needs? Benchmarking Needs: Criteria for Establishing the Existence and Magnitude of Needs Steps in Conducting Needs Assessments The St. Columba Collaboration Project: Needs Assessment in a Newark Neighborhood Summary Discussion Questions Appendix: Designing a Needs Assessment for a Small Non-Profit Organization References for Chapter 6 CHAPTER 7: CONCEPTS AND ISSUES IN ECONOMIC EVALUATION Introduction Three Types of Economic Evaluation Economic Evaluation in the Performance Management Cycle Historical Developments in Economic Evaluation Cost-Effectiveness Analysis Cost-Utility Analysis Cost-Benefit Analysis Cost-Effectiveness Analysis Example: A Study of Falls Prevention Among the Elderly Strengths and Limitations of Economic Evaluation Summary Discussion Questions References for Chapter 7 CHAPTER 8: PERFORMANCE MEASUREMENT AS AN APPROACH TO EVALUATION Introduction Growth of Performance Measurement Metaphors That Support and Sustain Performance Measurement Comparing Program Evaluation and Performance Measurement Summary Discussion Questions References for Chapter 8 CHAPTER 9 DESIGN AND IMPLEMENTATION OF PERFORMANCE MEASUREMENT SYSTEMS Introduction Key Steps in Designing and Implementing a Performance Measurement System Summary Discussion Questions References for Chapter 9 CHAPTER 10: USING AND SUSTAINING PERFORMANCE MEASUREMENT SYSTEMS Introduction Comparing Ideal and Real Organizations: The Role of Organizational Politics in Designing and Implementing Performance Measurement Systems Intended Uses of Performance Measurement Systems Actual Uses of Performance Measurement Problems and Issues in Implementing and Sustaining Performance Measurement Systems Summary Discussion questions References for Chapter 10 CHAPTER 11: PROGRAM EVALUATION AND PROGRAM MANAGEMENT: JOINING THEORY AND PRACTICE Introduction Can Management and Evaluation Be Joined? Prospects for Building Cultures That Support Evaluation Learning Organizations as Self-Evaluating Organizations Can Program Managers Evaluate Their Own Programs? How Should Evaluators Relate to Managers: Striving for Objectivity in Program Evaluations Can Program Evaluators Be Objective? Criteria for Best Practices in Program Evaluation: Assuring Stakeholders that Evaluations Are High Quality Ethics and evaluation practice Summary Discussion Questions References for Chapter 11 CHAPTER 12: THE NATURE AND PRACTICE OF PROFESSIONAL JUDGMENT IN PROGRAM EVALUATION Introduction The Nature of the Evaluation Enterprise Professionalism and Evaluation Practice Understanding Professional Judgment Improving Professional Judgment in Evaluation Practice Evaluation as a Craft: Implications for Professionalizing Evaluation Teamwork and Professional Judgment Summary Discussion Questions References for Chapter 12