Evaluating Public and Community Health Programs

Preface xiii Acknowledgments xv 1 AN INTRODUCTION TO PUBLIC AND COMMUNITY HEALTH EVALUATION 1 Overview of Evaluation 3 Levels of Evaluation 5 Preassessment Evaluations 6 The Participatory Approach to Evaluation 8 The Participatory Model for Evaluation 10 The Precursors to Program Evaluation 12 Cultural Considerations in Evaluation 17 Summary 21 Discussion Questions and Activities 22 Key Terms 22 2 THE COMMUNITY ASSESSMENT: AN OVERVIEW 23 Theoretical Considerations 26 The Ecological Model 29 Data Collection 31 Data Sources 42 Reviewing the Scientific Literature 49 The Report 50 Stakeholders' Participation in Community Assessments 52 Summary 53 Discussion Questions and Activities 54 Key Terms 54 3 DEVELOPING INITIATIVES: AN OVERVIEW 55 The Organization's Mission 58 Planning the Initiative 59 Incorporate Theory 60 Goals and Objectives 64 The Initiative's Activities 71 Use Existing Evidence-Based Programs 75 The Program's Theory of Change 77 The Logic Model Depicting the Theory of Change 77 Criteria for Successful Initiatives 81 Stakeholders' Participation in Planning and Developing Initiatives 82 Summary 82 Discussion Questions and Activities 83 Key Terms 84 4 PLANNING FOR EVALUATION: PURPOSE AND PROCESSES 85 The Timing of the Evaluation 86 The Purpose of Evaluation 87 The Contract for Evaluation 90 The Evaluation Team 91 Evaluation Standards 93 Managing the Evaluation Process 94 Factors That Influence the Evaluation Process 100 Planning for Ethical Program Evaluation 103 Involving Stakeholders 104 Creating and Maintaining Effective Partnerships 105 Summary 109 Discussion Questions and Activities 110 Key Terms 110 5 DESIGNING THE EVALUATION: PART 1: DESCRIBING THE PROGRAM OR POLICY 111 The Context of the Initiative 114 The Social, Political, and Economic Environment 116 The Organizational Structure and Resources 117 The Initiative and Its Relationship to the Organization 118 The Stage of Development of the Initiative 118 Data Access and Availability 119 The Program Initiative 119 The Policy Initiatives 125 Summary 131 Discussion Questions and Activities 132 Key Terms 132 6 DESIGNING THE EVALUATION: PART 2A: PROCESS EVALUATION 133 Purposes of Process Evaluation 135 Key Issues in Process Evaluation 137 Selecting Questions for Process Evaluation 146 Resources for Evaluation 152 Measuring Resources, Processes, and Outputs 153 Tools for Process Evaluation 155 Ethical and Cultural Considerations 157 Summary 168 Discussion Questions and Activities 169 Key Terms 169 7 DESIGNING THE EVALUATION: PART 2B: OUTCOME EVALUATION 171 The Relationship Between Process and Outcome Evaluation 172 Sorting and Selecting Evaluation Questions 176 Summary 191 Discussion Questions and Activities 192 Key Terms 192 8 COLLECTING THE DATA: QUANTITATIVE 193 Quantitative Data 194 Factors Influencing Data Collection 195 Using Surveys 200 Designing Survey Instruments 206 Pilot Testing 209 Triangulation 210 Institutional Review and Ethics Boards 211 The Data Collection Team 212 Managing and Storing Data 213 Stakeholder Involvement 213 Summary 214 Discussion Questions and Activities 215 Key Terms 215 9 COLLECTING THE DATA: QUALITATIVE 217 Qualitative Data 218 Ensuring Validity and Reliability 219 Interview-Format Approaches 221 Document and Record Review 231 Observational Approaches 233 Case Reviews 235 Digital Approaches 236 Geographic Information Systems 237 Training Data Collectors 238 Managing and Storing Qualitative Data 239 Stakeholder Involvement 239 The Data Collection Team 240 Summary 241 Discussion Questions and Activities 242 Key Terms 242 10 ANALYZING AND INTERPRETING QUANTITATIVE AND QUALITATIVE DATA: QUANTITATIVE (PART 1) 243 Analyzing and Reporting Quantitative Data 244 Reaching Conclusions 249 Stakeholder Involvement 250 Summary 257 Discussion Questions and Activities 257 QUALITATIVE (PART 2) 258 Analyzing Qualitative Data 258 Interpreting the Data and Reaching Conclusions 264 The Role of Stakeholders 267 Summary 268 Discussion Questions and Activities 269 Key Terms 269 11 REPORTING EVALUATION FINDINGS 271 The Content of the Report 274 The Audience for the Report 280 The Timing of the Report 281 The Format of the Report 282 Summary 286 Discussion Questions and Activities 287 Key Terms 287 12 CASE STUDY: THE COMMUNITY ASSESSMENT 289 Background 289 Establish a Team 289 Determine the Availability of Data 291 Decide on the Data-Collection Approaches and Methods 292 Training 293 Resource Procurement 294 Analysis and Interpretation of the Data 294 Summary of Findings 294 The Intervention 295 Design the Evaluation 298 Collect the Data 306 Analyze and Interpret the Data 310 Report the Results 313 Discussion Questions and Activities 315 13 CASE STUDY: PROCESS EVALUATION 317 Background 317 Theoretical Framework 318 Community Assessment Findings 319 The Evaluation Plan 326 Answering the Evaluation Question 328 Reporting the Results 338 Discussion Questions and Activities 340 A MODEL AGREEMENT BETWEEN EVALUATION CONSULTANT AND TEAM MEMBERS 341 B MODEL PREAMBLE FOR ADULT INDIVIDUAL INTERVIEWS 345 C MODEL DEMOGRAPHIC SHEET 347 D MODEL FIELD NOTES REPORT 349 E MODEL INTERVIEW REFUSAL REPORT 351 F DATA COLLECTION TRAINING MANUAL TEMPLATE 353 Conducting the Interview 355 G GUIDELINES FOR COMPLETING AN EVALUATION REPORT 363 Methodology 364 REFERENCES 367 INDEX 373