Evaluation Research Methods
1st Edition
Published on 21. December 2004
Book
Hardback
1736 pages
978-1-4129-0152-9 (ISBN)
Description
Evaluation is a set of research methods and associated methodologies with a distinctive purpose. They provide a means to judge actions and activities in terms of values, criteria and standards. At the same time evaluation is also a practice that seeks to enhance effectiveness in the public sphere and policy making. In order to improve as well as judge, there is a need to explain what happens and would have to be done differently for different outcomes to be achieved. It is in this explanatory mode that evaluation overlaps most directly with mainstream social science.
This collection offers a complete guide to evaluations research methods. It is organized in four volumes.
Volume 1 focuses on foundation issues and includes sections on the rationale for evaluation, central methodological debates, the role of theory and applying values, criteria and standards.
Volume 2 examines explaining through evaluation and covers sections on experimentation and causal inference, outcomes and inputs, socio-economic indicators, economics and cost benefit approaches and realist methods.
Volume 3 addresses qualitative methods and includes sections on case studies, responsive, developmental and accompanying evaluation, participation and empowerment, constructivism and postmodernism and multi-criteria and classificatory methods.
Volume 4 concentrates on evaluation to improve policy with sections on performance management, systematic reviews, institutionalization and utilization and policy learning and design.
The collection offers a unique and unparalleled guide to this rapidly expanding research method. It demonstrates how method and theory are applied in policy and strategy and will be an invaluable addition to any social science library.
Elliot Stern is the editor of Evaluation: the International Journal of Theory, Research and Practice, and works both as an independent consultant. He was previously Principal Advisor for evaluation studies at the Tavistock Institute, London.
This collection offers a complete guide to evaluations research methods. It is organized in four volumes.
Volume 1 focuses on foundation issues and includes sections on the rationale for evaluation, central methodological debates, the role of theory and applying values, criteria and standards.
Volume 2 examines explaining through evaluation and covers sections on experimentation and causal inference, outcomes and inputs, socio-economic indicators, economics and cost benefit approaches and realist methods.
Volume 3 addresses qualitative methods and includes sections on case studies, responsive, developmental and accompanying evaluation, participation and empowerment, constructivism and postmodernism and multi-criteria and classificatory methods.
Volume 4 concentrates on evaluation to improve policy with sections on performance management, systematic reviews, institutionalization and utilization and policy learning and design.
The collection offers a unique and unparalleled guide to this rapidly expanding research method. It demonstrates how method and theory are applied in policy and strategy and will be an invaluable addition to any social science library.
Elliot Stern is the editor of Evaluation: the International Journal of Theory, Research and Practice, and works both as an independent consultant. He was previously Principal Advisor for evaluation studies at the Tavistock Institute, London.
More details
Series
Edition
Four-Volume Set
Language
English
Place of publication
Thousand Oaks
United States
Target group
Professional and scholarly
Product notice
Cloth over boards
Dimensions
Height: 234 mm
Width: 156 mm
Weight
3115 gr
ISBN-13
978-1-4129-0152-9 (9781412901529)
Copyright in bibliographic data is held by Nielsen Book Services Limited or its licensors: all rights reserved.
Schweitzer Classification
Content
VOLUME ONE
PART ONE: FOUNDATIONS ISSUES IN EVALUATION
SECTION ONE: TYPOLOGIES AND PARADIGMS
The Logic of Evaluation and Evaluation Practice - Michael Scriven
Evaluation as Sensemaking - George Julnes and Melvin M Mark
Knowledge Construction in a Realist World
Establishing Evaluative Conclusions - Deborah M Fournier
A Distinction between General and Working Logic
Evaluation as Practical Hermeneutics - Thomas A Schwandt
In with the New - Ray Pawson and Nick Tilley
Introducing Scientific Realism
A Theoretical Framework and Grid for Analysis of Programme-Evaluation Practices - Michel Demarteau
Policy Evaluation and Learning - Peter van der Knaap
Feedback, Enlightenment or Argumentation?
Merit Criteria and Performance Standards - Evert Vedung
Public Managers amidst Ambiguity - Tineke A Abma and Mirko Noordegraaf
Towards a Typology of Evaluative Practices in Public Management
SECTION TWO: ETHICS AND VALUES
Evaluation as Advocacy - Jennifer C Greene
How We Inhabit our Evaluations - Saville Kushner
Research on Evaluation Ethics - Michael Morris
What Have we Learned and Why Is It Important?
Anything Goes - Clem Adelman
Evaluation and Relativism
The Limits of Constructivism in Evaluation - Saville Kushner
The Future of Ethics in Evaluation - Dianna L Newman
Developing the Dialogue
SECTION THREE: QUANTITATIVE AND QUALITATIVE DEBATES
Specifying Inquiry Modes - Melvin M Mark, Gary T Henry and George Julnes
Philosophy of Science and the Quantitative - Qualitative Debates - William R Shadish
Thirteen Common Errors
The Foundations of Qualitative Research - Dawn Snape and Liz Spencer
VOLUME TWO
PART TWO: EVALUATION PRACTICE FOR ENGAGEMENT
SECTION ONE: CASE STUDIES AND DIALOGICAL METHODS
The Abridged Version of Case Study Research - Robert K Yin
Design and Method
A Dialogical, Story-Based Evaluation Tool - Jessica Dart and Rick Davies
The Most Significant Change Technique
Stake's Responsive Evaluation - Tineke A Abma and Robert E Stake
Core Ideas and Evolution
SECTION TWO: PARTICIPATION AND EMPOWERMENT
Deliberative Democratic Evaluation in Practice - Ernest R House and Kenneth R Howe
Empowerment Evaluation - David M Fetterman
Participatory Program Planning - Laura Nichols
Including Program Participants and Evaluators
Democratic Evaluation - Beatrice Floc'hlay and Eric Plottu
From Empowerment Evaluation to Public Decision-Making
Participatory Evaluation in a Community Organization: Fostering Stakeholder Empowerment and Utilization - Danielle Papineau and Margaret C Kiely
SECTION THREE: COMBINING METHODS AND DATA
Toward a Conceptual Framework for Mixed-Method Evaluation Designs - Jennifer C Greene, Valerie J Caracelli and Wendy F Graham
An Example of Qualitative Research Methods in Realist Evaluation - Mansoor A F Kazi
From Puzzles to Problems - Daniel J McLinden and William MK Trochim
Assessing the Value of Education in a Business Context with Concept Mapping and Pattern Matching
Qualitative Evaluations, Combined Methods and Key Challenges - Jeff Hearn, John Lawler and George Dowswell
General Lessons from the Qualitative Evaluation of Community Intervention in Stroke Rehabilitation
SECTION FOUR: PLANNING AND MANAGING EVALUATION
Beginning an Evaluation with RUFDATA - Murray Saunders
Theorizing a Practical Approach to Evaluation Planning
Evaluability Assessment - M F Smith
Reflections on the Process Method
RENOMO - A Design Tool for Evaluations - Wolfgang Beywl and Philip Potter
Designing Evaluations Responsive to Stakeholders' Interests by Working with Nominal Groups using the Moderation
SECTION FIVE: MULTI-SITE AND MULTI-LEVEL EVALUATION
A Framework for Understanding and Improving Multisite Evaluations - Roger B Straw and James M Herrell
Conceptual Challenges Confronting Cluster Evaluation - Blaine R Worthen and Constance C Schmitz
Local Evaluation in a National Early Years Excellence Centres Pilot Programme: Integrating Performance Management and Participatory Evaluation - Colin Biott and Tina Cook
Evaluative Site Visits - Frances Lawrenz, Nanette Keiser and Bethann Lavoie
A Methodological Review
VOLUME THREE
PART THREE: EXPLANATORY AND ECONOMIC MODELS
SECTION ONE: PROGRAMME THEORY AND LOGIC MODELS
Program Theory - Patricia J Rogers
Not Whether Programs Work but How They Work
Theory-Driven Program Evaluation in the New Millennium - Stewart I Donaldson
Logic Models - John A McLaughlin and Gretchen B Jordan
A Tool for Telling Your Programs Performance Story
Implementing a Theory of Change Evaluation in the Cleveland Community-Building Initiative - Sharon Milligan et al
A Case Study
Integrating a Comparison Group Design into a Theory of Change Evaluation - Beth C Weitzman, Diana Silver and Keri-Nicole Dillman
The Case of the Urban Health Initiative
A Pattern-Matching Approach to Link Program Theory and Evaluation Data - Jules M Marquart
Reconstructing Program Theories: Methods Available and Problems to be Solved - Frans L Leeuw
SECTION TWO: EXPERIMENTS AND THEIR LIMITS
Reforms as Experiments - Donald T Campbell
The False Choice between Theory-Based Evaluation and Experimentation - Thomas D Cook
Methods for the Experimenting Society - Donald T Campbell
The Randomised Trial - Lone Scocozza
A Critique from the Philosophy of Science
Should Evaluation be Freed from Its Causal Links? An Answer Illustrated by European Economic Development Policies - Jacques Toulemonde
SECTION THREE: ECONOMICS AND COSTS
Economic Evaluation in the Social Welfare Field - Tom A J Sefton
Making Ends Meet
Cost-Analysis Approaches in Evaluation and Decision Making - Henry M Levin and Patrick J McEwan
Policy Evaluation and the Concepts of Deadweight and Additionality - James J McEldowney
A Commentary
The Limits of Cost Benefit Evaluation - Ernest R House
Ethical Issues in Cost-Effectiveness Analysis - Steven D Pinkerton et al
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES
Evaluation Research and the Performance Management Movement - Ann Bonar Blalock
From Estrangement to Useful Integration?
Outcomes and the Problem - Lawrence B Mohr
VOLUME FOUR
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES (continued)
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES (continued)
Bridging the Macro and Micro Levels in Outcome Evaluation - Ingrid M E Munck
Evaluation and the Social Construction of Impacts - Frans-Bauke van der Meer
Effectiveness in Health Promotion - Keith Tones
Indicators and Evidence of Success
Desk Screening of Development Projects - Nathalie Holvoet and Robrecht Renard
Is It Effective?
Effective Use and Misuse of Performance Measurement - Burt Perrin
Opportunities for Program Evaluators to Facilitate Performance-Based Management - M A Scheirer and K Newcomer
SECTION TWO: CAPACITY AND UTILISATION
Developing Systematic Internal Evaluation - Arnold J Love
Improving the Quality of Internal Evaluation - Rosalie T Torres
The Evaluator as Consultant-Mediator
Internal Evaluation - Jonathan A Morell
A Synthesis of Traditional Methods and Industrial Engineering
Have We Learned Anything New about the Use of Evaluation? - Carol H Weiss
The Evaluator's Responsibility for Utilization - Michael Quinn Patton
Building Evaluation Capacity within Organisations - Richard C Sonnichsen
Evaluation Utilization as Argumentation - Ville Valovirta
SECTION THREE: POLICY, EVIDENCE AND LEARNING
Evaluation for Decisions - Carol H Weiss
Is Anybody There? Does Anybody Care?
From Knowing to Doing - Sandra Nutley, Isabel Walter and Huw T O Davies
A Framework for Understanding the Evidence-Into-Practice Agenda
Evidence-Based Policy - Ray Pawson
In Search of a Method
Evidence-Based Policy - Ray Pawson
The Promise of `Realist Synthesis'
Top-Down and Bottom-Up Approaches to Implementation Research - Paul A Sabatier
Towards an Evaluation of the Effects of Laws - Sidsel Sverdrup
Utilizing Time-Series Data of Complaints
PART ONE: FOUNDATIONS ISSUES IN EVALUATION
SECTION ONE: TYPOLOGIES AND PARADIGMS
The Logic of Evaluation and Evaluation Practice - Michael Scriven
Evaluation as Sensemaking - George Julnes and Melvin M Mark
Knowledge Construction in a Realist World
Establishing Evaluative Conclusions - Deborah M Fournier
A Distinction between General and Working Logic
Evaluation as Practical Hermeneutics - Thomas A Schwandt
In with the New - Ray Pawson and Nick Tilley
Introducing Scientific Realism
A Theoretical Framework and Grid for Analysis of Programme-Evaluation Practices - Michel Demarteau
Policy Evaluation and Learning - Peter van der Knaap
Feedback, Enlightenment or Argumentation?
Merit Criteria and Performance Standards - Evert Vedung
Public Managers amidst Ambiguity - Tineke A Abma and Mirko Noordegraaf
Towards a Typology of Evaluative Practices in Public Management
SECTION TWO: ETHICS AND VALUES
Evaluation as Advocacy - Jennifer C Greene
How We Inhabit our Evaluations - Saville Kushner
Research on Evaluation Ethics - Michael Morris
What Have we Learned and Why Is It Important?
Anything Goes - Clem Adelman
Evaluation and Relativism
The Limits of Constructivism in Evaluation - Saville Kushner
The Future of Ethics in Evaluation - Dianna L Newman
Developing the Dialogue
SECTION THREE: QUANTITATIVE AND QUALITATIVE DEBATES
Specifying Inquiry Modes - Melvin M Mark, Gary T Henry and George Julnes
Philosophy of Science and the Quantitative - Qualitative Debates - William R Shadish
Thirteen Common Errors
The Foundations of Qualitative Research - Dawn Snape and Liz Spencer
VOLUME TWO
PART TWO: EVALUATION PRACTICE FOR ENGAGEMENT
SECTION ONE: CASE STUDIES AND DIALOGICAL METHODS
The Abridged Version of Case Study Research - Robert K Yin
Design and Method
A Dialogical, Story-Based Evaluation Tool - Jessica Dart and Rick Davies
The Most Significant Change Technique
Stake's Responsive Evaluation - Tineke A Abma and Robert E Stake
Core Ideas and Evolution
SECTION TWO: PARTICIPATION AND EMPOWERMENT
Deliberative Democratic Evaluation in Practice - Ernest R House and Kenneth R Howe
Empowerment Evaluation - David M Fetterman
Participatory Program Planning - Laura Nichols
Including Program Participants and Evaluators
Democratic Evaluation - Beatrice Floc'hlay and Eric Plottu
From Empowerment Evaluation to Public Decision-Making
Participatory Evaluation in a Community Organization: Fostering Stakeholder Empowerment and Utilization - Danielle Papineau and Margaret C Kiely
SECTION THREE: COMBINING METHODS AND DATA
Toward a Conceptual Framework for Mixed-Method Evaluation Designs - Jennifer C Greene, Valerie J Caracelli and Wendy F Graham
An Example of Qualitative Research Methods in Realist Evaluation - Mansoor A F Kazi
From Puzzles to Problems - Daniel J McLinden and William MK Trochim
Assessing the Value of Education in a Business Context with Concept Mapping and Pattern Matching
Qualitative Evaluations, Combined Methods and Key Challenges - Jeff Hearn, John Lawler and George Dowswell
General Lessons from the Qualitative Evaluation of Community Intervention in Stroke Rehabilitation
SECTION FOUR: PLANNING AND MANAGING EVALUATION
Beginning an Evaluation with RUFDATA - Murray Saunders
Theorizing a Practical Approach to Evaluation Planning
Evaluability Assessment - M F Smith
Reflections on the Process Method
RENOMO - A Design Tool for Evaluations - Wolfgang Beywl and Philip Potter
Designing Evaluations Responsive to Stakeholders' Interests by Working with Nominal Groups using the Moderation
SECTION FIVE: MULTI-SITE AND MULTI-LEVEL EVALUATION
A Framework for Understanding and Improving Multisite Evaluations - Roger B Straw and James M Herrell
Conceptual Challenges Confronting Cluster Evaluation - Blaine R Worthen and Constance C Schmitz
Local Evaluation in a National Early Years Excellence Centres Pilot Programme: Integrating Performance Management and Participatory Evaluation - Colin Biott and Tina Cook
Evaluative Site Visits - Frances Lawrenz, Nanette Keiser and Bethann Lavoie
A Methodological Review
VOLUME THREE
PART THREE: EXPLANATORY AND ECONOMIC MODELS
SECTION ONE: PROGRAMME THEORY AND LOGIC MODELS
Program Theory - Patricia J Rogers
Not Whether Programs Work but How They Work
Theory-Driven Program Evaluation in the New Millennium - Stewart I Donaldson
Logic Models - John A McLaughlin and Gretchen B Jordan
A Tool for Telling Your Programs Performance Story
Implementing a Theory of Change Evaluation in the Cleveland Community-Building Initiative - Sharon Milligan et al
A Case Study
Integrating a Comparison Group Design into a Theory of Change Evaluation - Beth C Weitzman, Diana Silver and Keri-Nicole Dillman
The Case of the Urban Health Initiative
A Pattern-Matching Approach to Link Program Theory and Evaluation Data - Jules M Marquart
Reconstructing Program Theories: Methods Available and Problems to be Solved - Frans L Leeuw
SECTION TWO: EXPERIMENTS AND THEIR LIMITS
Reforms as Experiments - Donald T Campbell
The False Choice between Theory-Based Evaluation and Experimentation - Thomas D Cook
Methods for the Experimenting Society - Donald T Campbell
The Randomised Trial - Lone Scocozza
A Critique from the Philosophy of Science
Should Evaluation be Freed from Its Causal Links? An Answer Illustrated by European Economic Development Policies - Jacques Toulemonde
SECTION THREE: ECONOMICS AND COSTS
Economic Evaluation in the Social Welfare Field - Tom A J Sefton
Making Ends Meet
Cost-Analysis Approaches in Evaluation and Decision Making - Henry M Levin and Patrick J McEwan
Policy Evaluation and the Concepts of Deadweight and Additionality - James J McEldowney
A Commentary
The Limits of Cost Benefit Evaluation - Ernest R House
Ethical Issues in Cost-Effectiveness Analysis - Steven D Pinkerton et al
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES
Evaluation Research and the Performance Management Movement - Ann Bonar Blalock
From Estrangement to Useful Integration?
Outcomes and the Problem - Lawrence B Mohr
VOLUME FOUR
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES (continued)
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES (continued)
Bridging the Macro and Micro Levels in Outcome Evaluation - Ingrid M E Munck
Evaluation and the Social Construction of Impacts - Frans-Bauke van der Meer
Effectiveness in Health Promotion - Keith Tones
Indicators and Evidence of Success
Desk Screening of Development Projects - Nathalie Holvoet and Robrecht Renard
Is It Effective?
Effective Use and Misuse of Performance Measurement - Burt Perrin
Opportunities for Program Evaluators to Facilitate Performance-Based Management - M A Scheirer and K Newcomer
SECTION TWO: CAPACITY AND UTILISATION
Developing Systematic Internal Evaluation - Arnold J Love
Improving the Quality of Internal Evaluation - Rosalie T Torres
The Evaluator as Consultant-Mediator
Internal Evaluation - Jonathan A Morell
A Synthesis of Traditional Methods and Industrial Engineering
Have We Learned Anything New about the Use of Evaluation? - Carol H Weiss
The Evaluator's Responsibility for Utilization - Michael Quinn Patton
Building Evaluation Capacity within Organisations - Richard C Sonnichsen
Evaluation Utilization as Argumentation - Ville Valovirta
SECTION THREE: POLICY, EVIDENCE AND LEARNING
Evaluation for Decisions - Carol H Weiss
Is Anybody There? Does Anybody Care?
From Knowing to Doing - Sandra Nutley, Isabel Walter and Huw T O Davies
A Framework for Understanding the Evidence-Into-Practice Agenda
Evidence-Based Policy - Ray Pawson
In Search of a Method
Evidence-Based Policy - Ray Pawson
The Promise of `Realist Synthesis'
Top-Down and Bottom-Up Approaches to Implementation Research - Paul A Sabatier
Towards an Evaluation of the Effects of Laws - Sidsel Sverdrup
Utilizing Time-Series Data of Complaints