Introduction

We have planned and developed this book to aid and enlighten those who evaluate, or intend to evaluate, programs, as well as those administrators and other evaluation stakeholders who use evaluation to meet program improvement and accountability needs. The book is intended particularly for use by practicing evaluators and students in graduate programs focused on evaluation theory and practice, but its handbook nature should prove useful to evaluation clients and others with an interest in learning about evaluation and obtaining sound, effective evaluation services.

Evaluation studies should be directed toward helping clients and other stakeholders use findings well and particularly toward improving and certifying the value of evaluation services. This is a heavy professional responsibility. In this book we have drawn together information from the evaluation literature and a wide range of practical experiences to guide, to advise, and to demonstrate that success in the worthwhile pursuit of systematic evaluation is both essential and clearly possible.

Evaluation is a vital component of the continuing health of organizations. If evaluations are conducted well, organizations and their people will have the satisfaction of knowing with confidence which elements are strong and where changes are needed. Evaluation is therefore a constructive pursuit.

This book is designed as a textbook for graduate courses concerned with the critical analysis and application of program evaluation theory, approaches and models, and methods, and more widely as a handbook for use in planning, conducting, and assessing program evaluations. The book builds and expands on the widely circulated Evaluation Models monograph in New Directions for Evaluation (Stufflebeam, 2001b).

Throughout this book, we typically refer to evaluation approaches (rather than models), using the more generic term to cover all generalized ways of designing and conducting evaluations. We selected this term because it encompasses illicit as well as commendable ways of doing evaluations and includes all good approaches, whether or not they are referred to as models.

We undertook this writing project at the urging of a number of colleagues and representatives of Jossey-Bass, initially seeking only to update Evaluation Models (Stufflebeam, 2001b). Leaders at the publishing company convinced us, however, of the need for an updated, extended treatment of Evaluation Models plus practical guidelines and procedures for applying the best evaluation approaches. In this book we address these needs and also discuss the foundational topic of evaluation theory. Readers will find checklists for guiding such core evaluation tasks as designing, budgeting, contracting, reporting on, and assessing evaluations, plus others focused on data collection and analysis. Although the heart of the book is an updated, expanded treatment of evaluation approaches (found in Parts Two and Three), this core content is now embedded in a broader discussion of theoretical and practical topics. We have focused the book on helping evaluators and others strengthen their theoretical understanding and working knowledge of evaluation.

Changes to the First Edition

This second edition of Evaluation Theory, Models, and Applications has undergone substantial revision since the first edition was published (Stufflebeam & Shinkfield, 2007). Major changes are the inclusion of several additional evaluation approaches and the elimination of others (and the addition of a new second author due to Anthony Shinkfield's busy schedule). In the first edition, twenty-six unique evaluation approaches were introduced and described. In this edition, the descriptions of evaluation approaches have been reduced to twenty-three, but with the addition of transformative evaluation, participatory evaluation, customer feedback evaluation, and meta-analysis. In addition, many of the evaluation approaches originally described in the first edition have been substantially revised and updated, and in many instances an approach's description has been extended to provide greater depth, detail, and insight into its specific characteristics.

Also, each chapter in this second edition begins with a list of chapter learning objectives. Key references have been added throughout chapters so that interested readers may locate additional information concerning the topic under discussion. Further, each chapter now includes a short section titled "Suggested Supplemental Readings," in which readers may locate additional source documents, books, articles, and reports intended to supplement and sometimes elaborate further on the chapter's core content. The book is also supported by relevant materials housed on the Web sites of Western Michigan University's Evaluation Center (www.wmich.edu/evalctr/) and Jossey-Bass (www.josseybass.com/go/evalmodels).

Intended Audience

Because program evaluation is such a pervasive concern in society, we have designed the book to serve the needs of a broad range of individuals and groups that must use evaluations to assess, ensure, or improve the quality of programs. The book can be useful to graduate students, evaluation and research instructors, evaluators, program administrators, business leaders, specialists in research and evaluation methodology, professionals, and other service providers who must meet requirements for public accountability, as well as those who commission program evaluations. The book treats program evaluation across disciplines, and thus is intended for use in such fields as nursing, community development, housing, education, medicine, psychotherapy, disease control, business administration, jurisprudence, national defense, engineering, social services, philanthropy, and international development, among others.

Overview of the Book's Contents

Evaluators and users of evaluations can use this book to acquire knowledge of approaches that are available for evaluating programs; the concepts and theories undergirding different evaluation approaches; and principles, standards, and procedures for guiding and judging the work of evaluators. The book provides evaluations of twenty-three evaluation approaches, detailed information about six evaluation approaches, techniques for carrying out the full range of steps in any program evaluation, and guidance for institutionalizing and mainstreaming evaluation.

Faced with a growing number of program evaluation approaches, evaluators need competence to assess and choose wisely among available options and then confidently and effectively apply the selected approach. Overall, in choosing topics for this book, we sought to provide a sense of the general nature of program evaluation, an overview and comparative analysis of alternative approaches to evaluation, in-depth instruction-with examples-in each of six ways to conduct credible program evaluations, standards for choosing among approaches, and practical guidelines for designing and carrying out an evaluation from beginning to end.

Two dominant factors-the theoretical and practical essentials of evaluation-intertwine throughout the book, and are underlined by nine themes. The first theme is:

The evaluation discipline should be grounded in sound theory-that is, a coherent set of conceptual, hypothetical, pragmatic, and ethical principles forming a general framework to guide the study and practice of evaluation.

The second theme is:

Society needs and is using evaluations to inform decisions and hold service providers accountable for the implementation and outcomes of the services they provide.

The evaluator must plan, develop, and deploy a distinctive evaluation methodology that is technically sound and responsive to the client's needs.

Part One of the book introduces program evaluation in three chapters that set out evaluation's fundamentals. Chapter 1 discusses the role of evaluation in society; defines evaluation and other key evaluation concepts; denotes the principal uses of program evaluations; identifies different, complementary methodological approaches; and describes the evaluation profession in its historical context. In general, this opening chapter offers a sweeping perspective on the evaluation field and background information for use in studying the ensuing chapters. Chapter 2 looks closely at the nature of evaluation theory, particularly program evaluation theory. It defines evaluation theory, distinguishes between evaluation models and evaluation theories, identifies criteria for judging theories, and lists illustrative hypotheses for research on program evaluation. Stressing that nothing is as useful as a sound theory, the chapter calls for increased and improved efforts to generate and validate program evaluation theories. Chapter 3 reviews and discusses principles and standards for use in guiding and assessing program evaluations. It begins with a discussion of a professionally generated set of standards for educational program evaluations (Joint Committee on Standards for Educational Evaluation, 2011) that require evaluations to meet conditions of utility, feasibility, propriety, accuracy, and evaluation accountability. The chapter subsequently summarizes and discusses the American Evaluation Association's guiding principles for evaluators (2004), which are focused on ensuring that competent evaluators will serve the general and public welfare by conducting evaluations that are methodologically sound, competently conducted, ethical, respectful of...