Introduction

What Is This Book About?

To put it simply, this book is about integrating qualitative inquiry—its approaches, methods, stances, and theories—into the practice of evaluation. What do we mean by “qualitative inquiry” and “evaluation”? We agree with Thomas Schwandt (2007) that qualitative inquiry “aims at understanding the meaning of human action” (italics in original) (p. 248). It includes many forms of social inquiry such as case study research, naturalistic inquiry, ethnography, and others. Likewise, many definitions of evaluation are in play these days; however, Peter Dahler-Larsen (2012) summarizes them quite succinctly when he defines evaluation as “basically a systematic, methodological, and thus ‘assisted’ way of investigating and assessing an activity of public interest in order to affect decisions or actions concerning this activity or similar activities” (p. 9).

This book is about the intersection of evaluation and qualitative inquiry; that space in a Venn diagram where qualitative inquiry and evaluation overlap. It is the space where the two colored circles representing qualitative inquiry and evaluation come together to create a new color. That part of the diagram is what we are calling qualitative evaluation. That is what this book is about.

Sometimes, to understand what something is, it is helpful to understand what it is not. This book is not a how-to textbook. This edited volume will not tell you how to develop and implement a qualitative evaluation. Instead, we hope this book will stimulate your thinking on how different evaluation approaches dovetail with qualitative inquiry. In addition, the second part of the book offers “tales of the field” (Van Maanen, 1988) that share the ins and outs of the practice of qualitative evaluation.

Who Is This Book For and What Do We Hope You Will Get from Reading It?

As we surveyed the landscape of available literature, we found a wealth of resources on qualitative research and also on evaluation. However, we found few titles dedicated to an exploration of the intersection of qualitative inquiry and evaluation. This book focuses on that intersection, delving deeply into what it means to carry out qualitative evaluation and examining the practice of qualitative evaluation from various theoretical and practical angles.

This book is geared toward students and evaluators who are relatively new to the practice of qualitative inquiry in evaluation, but who have some foundational knowledge of both evaluation and qualitative inquiry. Its purpose is twofold: as a practical introduction for students and new evaluators and as a reference and resource for practicing evaluators. As an edited volume, this compilation of many voices is meant to spark dialogue among evaluators. Our hope is that the array of perspectives herein provides a window on the world of qualitative evaluation—a travelogue of sorts that describes the terrain and offers examples and lessons learned along the way.

For evaluators who are already using qualitative evaluation approaches, we hope the featured theoretical perspectives and practice considerations will deepen their understanding of the knowledge, skills, and epistemological stances that seasoned qualitative evaluators bring to their work. In short, we hope readers of all backgrounds will find it helpful to “listen in” as the contributing evaluators reflect on the theoretical perspectives that inform their practice and the practice-based experiences that have surfaced new layers of learning about their craft.

Many graduate students receive methodological training in qualitative research but are not adequately prepared to employ those skills out in the field, particularly in the context of an evaluation. In addition, many experienced evaluators with quantitative research backgrounds are interested in learning how to incorporate qualitative approaches into the evaluations they conduct. As many readers are no doubt aware, incorporating qualitative inquiry in evaluation entails much more than simply “doing some interviews” to gather a smattering of quotes to “round out” the quantitative data and analysis. The practicing evaluator must address a substantial number of considerations, and these considerations are not necessarily covered in qualitative research or evaluation textbooks. The chapters in Part 2 of this book illustrate these concepts and tell the tales of practitioners faced with real-world challenges and opportunities as they implement qualitative evaluations.

Going “into the field” to carry out qualitative inquiry—whether facilitating a meeting with evaluation stakeholders or conducting interviews—can be an exciting, uncertain, and even daunting task. We hope this collection of chapters deepens readers' thinking about the endeavor by exploring territory beyond that covered in more methodologically oriented texts and teaching.

How to Read This Book

As we constructed this volume, we thought about our own experiences as we each entered the field of evaluation and what we have encountered in our practice. We specifically divided the volume into two parts: one focuses on the intersection of evaluation approaches and qualitative inquiry; the other focuses on how evaluators think about and navigate their evaluation practice.

Part 1 of this volume is of particular value to people who are not steeped in evaluation theory. The first set of chapters provides grounding for people who may understand the epistemology of qualitative inquiry, but not necessarily how it dovetails with evaluation approaches. On the flipside, Part 1 also provides grounding for evaluators who want to deepen their exploration of the role of qualitative inquiry in evaluation.

Chapter 1, written by Thomas A. Schwandt and Timothy J. Cash, provides an historical journey from the advent of qualitative inquiry through its increasing incorporation into the practice of evaluation. Following an overview of the contested definition of qualitative evaluation, they conclude their chapter by describing the three enduring contributions that qualitative inquiry has made in: (1) shaping the ways an evaluation is focused; (2) how values are understood and portrayed; and (3) how evaluations are communicated.

After this orientation, we turn the discussion over to several evaluators who have been leaders in shaping different evaluation approaches and effectively integrating qualitative inquiry. Michael Quinn Patton has long been a pioneer in designing evaluations that are actually used by program stakeholders. In Chapter 2, he shares how he thinks about the intersection of utilization-focused evaluation and qualitative inquiry and highlights both opportunities and challenges for practice.

For those entering the realm of using evaluation to affect public policy, George F. Grob offers lessons he has learned from many years of work with policy makers at the local, state, and national levels. In Chapter 3, he shares his thoughts and experiences regarding the value of qualitative data in informing policy makers about public programs. His pragmatic approach demonstrates how evaluation operates in the “simultaneously fast paced and glacially slow” arena of policy making.

In Chapter 4, Katrina L. Bledsoe argues that although theory-driven evaluation is often associated with randomized controlled trials and the generalizability of programs, to effectively implement theory-driven evaluation, an evaluator must embrace qualitative methods when developing logic models, working with participants to articulate a program's theory, and also when collecting and analyzing data.

Jill Anne Chouinard and J. Bradley Cousins are leaders in the development of participatory evaluation. Chapter 5 provides an exploration of the intersections among qualitative inquiry, practical participatory evaluation, and transformative participatory evaluation.

Part 2 of this volume is of particular value to professionals new to the field of evaluation and those who are looking for examples of how qualitative evaluation is done “on the ground.” How does one enter an evaluation setting in a manner that portrays professionalism and command of the situation, but also quickly establishes trust? How does one interact with the evaluation funders when their concept of an evaluation design is inconsistent with the questions they want answered? How does one handle an evaluation that gets off track? These questions, and more, are addressed through case studies of evaluation practice by evaluators who bring varying degrees of experience to the endeavor.

Part 2 begins with an introductory chapter written by Laurie Stevahn and Jean A. King, who have spent much of their academic careers guiding students in the development of important skills and competencies for conducting effective evaluations. In Chapter 6, they describe the competencies required to conduct effective evaluations generally, and they delve deeply into the specialized competencies required to conduct effective qualitative evaluations.

In Chapter 7, Vivianne E. Baur and Tineke A. Abma describe how they used a transformative paradigm with responsive evaluation and hermeneutic dialogue to address power differences in the management of residential units for senior citizens. In addition to discussing how they interacted with the Board that funded...