Developing Monitoring and Evaluation Frameworks
1st Edition
Published on 3. September 2015
Book
Paperback/Softback
312 pages
978-1-4833-5833-8 (ISBN)
Description
This practical book provides clear, step-by-step guidance on how to develop a monitoring and evaluation framework in a participatory, logical, systematic, and integrated way. The authors outline the key stages and steps involved, including: scoping the framework; identifying planned results; using program theory and program logic; developing evaluation questions; identifying processes for ongoing data collection and analysis; determining means to promote learning; reporting; and dissemination of results. A final chapter focuses on planning for implementation of the framework, with reference to the broader program and organizational context. The authors draw on their extensive experience in developing monitoring and evaluation frameworks to provide examples of good practice that inform organizational learning and decision making, while offering tips and guidelines that can be used to address common pitfalls.
Reviews / Votes
"The text provides an excellent resource for developing monitoring and evaluation frameworks for a wide range of organizations and programs. Students and practitioners are provided with a great resource that can help organizations better society, and maximize evaluation efficiency by integrating monitoring with evaluation." -- Daniela C. Schroeter This book does a brilliant job of integrating program monitoring concepts into the broader field of evaluation. It fills a glaring gap in the evaluation literature, and I highly recommend it for practicing evaluators, evaluation scholars, and evaluation faculty and students. -- Stewart I. Donaldson At last--a step-by-step guide to realizing the potential of monitoring and evaluation by building them in from the beginning. This book provides a systematic and well-tested approach to planning and successfully implementing an integrated monitoring and evaluation framework. -- Patricia RogersMore details
Edition
First Edition
Language
English
Place of publication
Thousand Oaks
United States
Target group
College/higher education
Professional and scholarly
Dimensions
Height: 235 mm
Width: 191 mm
Thickness: 17 mm
Weight
586 gr
ISBN-13
978-1-4833-5833-8 (9781483358338)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Persons
Anne Markiewicz is an independent evaluation consultant and the Director of Anne Markiewicz and Associates. Commencing her professional career a social worker, Anne held academic positions in Social Work at two Australian Universities. In 1997, Anne pursued her interest in evaluation and commenced working as an independent consultant. In 2000, she completed a Master of Education in Program Evaluation at the University of Melbourne. Since that time, Anne has completed a significant number of evaluation projects for a range of Australian government departments, non-government organizations and international agencies. She has specialized in evaluating programs with social justice objectives.
Anne's practice as an evaluator has highlighted the need for early evaluation planning to provide a clear and agreed focus, ensure data availability and support the production of credible evaluations. She has increasingly focused on the development of Monitoring and Evaluation Frameworks for a range of clients. She has designed and delivered an array of related training programs, which also encompass areas such as stakeholder engagement, negotiation and evaluation reporting. This training has been delivered extensively in Australasia, particularly within Australia and Papua New Guinea. Anne has a commitment to evaluation capacity building and has provided mentoring support to a range of government and non-government organizations.
Anne is a past board member of the Australasian Evaluation Society, holding positions of Secretary and Vice President. She has received two awards for excellence in evaluation from the Australasian Evaluation Society. These were the 'Indigenous Evaluation Award' (2008) for an evaluation completed with the Australian Department of Finance and Deregulation, and the 'Outstanding Contribution to Evaluation Award' (2013). Anne has published articles on the political context to evaluation, balancing stakeholder interests in evaluation and the evaluation of programs for Indigenous Australians.
Dr. Ian Patrick is an independent evaluation consultant and Director of Ian Patrick and Associates. Ian has a number of threads in his career which have supported his move into evaluation. Building on a background in teaching and radio and television production, Ian shifted his focus to international program management. He became aware of the challenges involved in evaluation of complex programs operating in the Asia-Pacific region while working for several Australian universities and managing contractors. Ian completed a PhD in Asian Studies in 1997, and commenced working in academic roles in international development for both the RMIT University and Deakin University, Australia. Ian is currently an Honorary Senior Fellow with the Development Studies Program at the University of Melbourne.
Ian commenced working as an independent consultant in 1998 with roles encompassing evaluation, program design, strategy development and organizational review. The focus of this work has progressively shifted to concentrate on the evaluation area and has also achieved a balance between Australian and international contexts. He has undertaken evaluation related assignments for a range of international agencies, Australian government departments and non-government organisations. He has interests in evaluation practice in both public sector and civil society contexts with sectoral experience crossing areas such as law and justice, health, education, rural and urban development, environment, human rights, disaster management, media and the arts, and migration and Indigenous issues.
Ian is a member of the Australasian Evaluation Society and the American Evaluation Association. In 2012, he received the Best Evaluation Policy and Systems Award from the Australasian Evaluation Society for a Monitoring and Evaluation Framework developed for the Mongolia Australia Scholarship Program.
Anne's practice as an evaluator has highlighted the need for early evaluation planning to provide a clear and agreed focus, ensure data availability and support the production of credible evaluations. She has increasingly focused on the development of Monitoring and Evaluation Frameworks for a range of clients. She has designed and delivered an array of related training programs, which also encompass areas such as stakeholder engagement, negotiation and evaluation reporting. This training has been delivered extensively in Australasia, particularly within Australia and Papua New Guinea. Anne has a commitment to evaluation capacity building and has provided mentoring support to a range of government and non-government organizations.
Anne is a past board member of the Australasian Evaluation Society, holding positions of Secretary and Vice President. She has received two awards for excellence in evaluation from the Australasian Evaluation Society. These were the 'Indigenous Evaluation Award' (2008) for an evaluation completed with the Australian Department of Finance and Deregulation, and the 'Outstanding Contribution to Evaluation Award' (2013). Anne has published articles on the political context to evaluation, balancing stakeholder interests in evaluation and the evaluation of programs for Indigenous Australians.
Dr. Ian Patrick is an independent evaluation consultant and Director of Ian Patrick and Associates. Ian has a number of threads in his career which have supported his move into evaluation. Building on a background in teaching and radio and television production, Ian shifted his focus to international program management. He became aware of the challenges involved in evaluation of complex programs operating in the Asia-Pacific region while working for several Australian universities and managing contractors. Ian completed a PhD in Asian Studies in 1997, and commenced working in academic roles in international development for both the RMIT University and Deakin University, Australia. Ian is currently an Honorary Senior Fellow with the Development Studies Program at the University of Melbourne.
Ian commenced working as an independent consultant in 1998 with roles encompassing evaluation, program design, strategy development and organizational review. The focus of this work has progressively shifted to concentrate on the evaluation area and has also achieved a balance between Australian and international contexts. He has undertaken evaluation related assignments for a range of international agencies, Australian government departments and non-government organisations. He has interests in evaluation practice in both public sector and civil society contexts with sectoral experience crossing areas such as law and justice, health, education, rural and urban development, environment, human rights, disaster management, media and the arts, and migration and Indigenous issues.
Ian is a member of the Australasian Evaluation Society and the American Evaluation Association. In 2012, he received the Best Evaluation Policy and Systems Award from the Australasian Evaluation Society for a Monitoring and Evaluation Framework developed for the Mongolia Australia Scholarship Program.
Content
Preface
Acknowledgments
About the Authors
Chapter 1 Introduction to Developing Monitoring and Evaluation Frameworks
What Is a Monitoring and Evaluation Framework?
Why Invest?
What Functions Does a Monitoring and Evaluation Framework Serve?
What Gap Does This Text Intend to Fill?
Key Features of the Approach
Program-Level Focus
What Are the Differences Between Monitoring and Evaluation?
What Are the Complementaries Between Monitoring and Evaluation?
What is Included in a Monitoring and Evaluation Framework?
Format and Layout of the Text
Chapter 2 Foundation Concepts
1. Multiple Purposes for Monitoring and Evaluation
2. Results-Based Management Approach
3. Theory-Based Approach
4. Evaluation-Led Monitoring and Evaluation
5. Participatory Orientation
Chapter 3 First Steps: Scoping the Monitoring and Evaluation Framework, Stakeholder Mapping and Evaluation Capacity Building
1. Introduction to Scoping the Framework With Key Stakeholders
2. Key Steps in Scoping the Framework
Step 1: Identify Requirements
Step 2: Determine Participation Arrangements
Step 3: Identify Possible and Preferred Approaches
Step 4: Review Resource Parameters
Step 5: Confirm Purpose and Parameters of the Framework
3. From First Steps Onwards
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 4 Program Theory and Program Logic as a Foundation for the Monitoring and Evaluation Framework
1. Introduction to Developing Program Theory and Program Logic
2. Steps Involved in Developing Program Theory and Program Logic
Step 1: Plan Stakeholder Engagement Strategy
Step 2: Develop Program Theory
Step 3: Develop Program Logic
Step 4: Confirm Program Theory and Program Logic with Key Stakeholders
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 5 Evaluation Questions - Determining What We Want to Know
1. Introduction to Developing Evaluation Questions
2. Using Evaluation Domains to Guide Selection of Questions
3. Using Program Theory and Logic to Determine Evaluation Questions
4. Steps Involved in Developing Evaluation Questions
Step 1: Develop Draft Evaluation Questions
Step 2: Facilitate Stakeholder Participation
Step 3: Scope Number and Range of Questions Against Data and Resources Available
Step 4: Present Questions to Stakeholders for Final Endorsement
Step 5: Finalize Evaluation Questions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 6 The Monitoring Plan
1. Introduction
2. Introduction to Program Monitoring
3. The Monitoring Plan
4. Steps in Developing Monitoring Plan
Step 1: Identify Focus
Step 2: Develop Performance Indicators and Targets
Step 3: Identify Data Collection Processes and Tools
Step 4: Determine Responsibilities and Time Frames
5. The Future of Program Monitoring
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 7 The Evaluation Plan
1. Introduction
2. Focusing on Evaluation
3. The Evaluation Plan
4. Steps Involved in Developing the Evaluation Plan
Step 1: Determine Overall Evaluation Approach
Step 2: Identify Evaluation Questions Requiring Criteria and Standards
Step 3: Identify Focus of Evaluation and Methods for Each Questions
Step 4: Determine Responsibilities and Time Frames
Step 5: Review the Monitoring and Evaluation Plans
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 8 Collecting, Managing, Analyzing and Synthesizing Data to Reach Evaluative Conclusions
1. Introduction to Data Processes That Support Sound Conclusions
2. Organizational Context for Data Collection, Management, and Analysis
3. Data Quality
4. Steps Involved in Data Collection, Management and Analysis
Step 1: Develop Data Collection Plan
Step 2: Develop Data Management Plan
Step 3: Consider Approach to Data Synthesis
Step 4: Consider Approach to Making Evaluative Judgments and Reaching Evaluative Conclusions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 9 Learning, Reporting and Dissemination Strategies
1. Introduction
2. Steps Involved in Learning, Reporting and Dissemination
Step 1: Consider Developing or Refining a Learning Strategy That Maximizes Use of Conclusions, Recommendations and Lessons
Step 2: Consider the Identification of Recommendations and Lessons
Step 3: Provide Guidance on Developing a Reporting and Dissemination Strategy
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 10 Planning for Implementation of Monitoring and Evaluation Frameworks
1. Introduction to Planning and Implementation
2. Key Steps in Implementation of Monitoring and Evaluation Frameworks
Step 1: Confirm Program Management Arrangements
Step 2: Work Planning for Implementation
Step 3: Plan for Monitoring and Review of Framework
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 11 Conclusion
Appendix
References
Index
Acknowledgments
About the Authors
Chapter 1 Introduction to Developing Monitoring and Evaluation Frameworks
What Is a Monitoring and Evaluation Framework?
Why Invest?
What Functions Does a Monitoring and Evaluation Framework Serve?
What Gap Does This Text Intend to Fill?
Key Features of the Approach
Program-Level Focus
What Are the Differences Between Monitoring and Evaluation?
What Are the Complementaries Between Monitoring and Evaluation?
What is Included in a Monitoring and Evaluation Framework?
Format and Layout of the Text
Chapter 2 Foundation Concepts
1. Multiple Purposes for Monitoring and Evaluation
2. Results-Based Management Approach
3. Theory-Based Approach
4. Evaluation-Led Monitoring and Evaluation
5. Participatory Orientation
Chapter 3 First Steps: Scoping the Monitoring and Evaluation Framework, Stakeholder Mapping and Evaluation Capacity Building
1. Introduction to Scoping the Framework With Key Stakeholders
2. Key Steps in Scoping the Framework
Step 1: Identify Requirements
Step 2: Determine Participation Arrangements
Step 3: Identify Possible and Preferred Approaches
Step 4: Review Resource Parameters
Step 5: Confirm Purpose and Parameters of the Framework
3. From First Steps Onwards
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 4 Program Theory and Program Logic as a Foundation for the Monitoring and Evaluation Framework
1. Introduction to Developing Program Theory and Program Logic
2. Steps Involved in Developing Program Theory and Program Logic
Step 1: Plan Stakeholder Engagement Strategy
Step 2: Develop Program Theory
Step 3: Develop Program Logic
Step 4: Confirm Program Theory and Program Logic with Key Stakeholders
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 5 Evaluation Questions - Determining What We Want to Know
1. Introduction to Developing Evaluation Questions
2. Using Evaluation Domains to Guide Selection of Questions
3. Using Program Theory and Logic to Determine Evaluation Questions
4. Steps Involved in Developing Evaluation Questions
Step 1: Develop Draft Evaluation Questions
Step 2: Facilitate Stakeholder Participation
Step 3: Scope Number and Range of Questions Against Data and Resources Available
Step 4: Present Questions to Stakeholders for Final Endorsement
Step 5: Finalize Evaluation Questions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 6 The Monitoring Plan
1. Introduction
2. Introduction to Program Monitoring
3. The Monitoring Plan
4. Steps in Developing Monitoring Plan
Step 1: Identify Focus
Step 2: Develop Performance Indicators and Targets
Step 3: Identify Data Collection Processes and Tools
Step 4: Determine Responsibilities and Time Frames
5. The Future of Program Monitoring
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 7 The Evaluation Plan
1. Introduction
2. Focusing on Evaluation
3. The Evaluation Plan
4. Steps Involved in Developing the Evaluation Plan
Step 1: Determine Overall Evaluation Approach
Step 2: Identify Evaluation Questions Requiring Criteria and Standards
Step 3: Identify Focus of Evaluation and Methods for Each Questions
Step 4: Determine Responsibilities and Time Frames
Step 5: Review the Monitoring and Evaluation Plans
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 8 Collecting, Managing, Analyzing and Synthesizing Data to Reach Evaluative Conclusions
1. Introduction to Data Processes That Support Sound Conclusions
2. Organizational Context for Data Collection, Management, and Analysis
3. Data Quality
4. Steps Involved in Data Collection, Management and Analysis
Step 1: Develop Data Collection Plan
Step 2: Develop Data Management Plan
Step 3: Consider Approach to Data Synthesis
Step 4: Consider Approach to Making Evaluative Judgments and Reaching Evaluative Conclusions
SUMMARY CHECKLIST
CHAPTER REVIEW QUESTIONS
Chapter 9 Learning, Reporting and Dissemination Strategies
1. Introduction
2. Steps Involved in Learning, Reporting and Dissemination
Step 1: Consider Developing or Refining a Learning Strategy That Maximizes Use of Conclusions, Recommendations and Lessons
Step 2: Consider the Identification of Recommendations and Lessons
Step 3: Provide Guidance on Developing a Reporting and Dissemination Strategy
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 10 Planning for Implementation of Monitoring and Evaluation Frameworks
1. Introduction to Planning and Implementation
2. Key Steps in Implementation of Monitoring and Evaluation Frameworks
Step 1: Confirm Program Management Arrangements
Step 2: Work Planning for Implementation
Step 3: Plan for Monitoring and Review of Framework
SUMMARY CHECKLIST
CHAPTER REVIEW QUESITONS
Chapter 11 Conclusion
Appendix
References
Index