Randomized Control Trials in the Field of Development
A Critical Perspective
1st Edition
Published on 17. September 2020
448 pages
978-0-19-263552-5 (ISBN)
Description
This is an open access title available under the terms of a CC BY-NC-ND 4.0 International licence. It is free to read at Oxford Academic and offered as a free PDF download from OUP and selected open access locations.
In October 2019, Abhijit Banerjee, Esther Duflo, and Michael Kremer jointly won the 51st Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel "for their experimental approach to alleviating global poverty." But what is the exact scope of their experimental method, known as randomized control trials (RCTs)? Which sorts of questions are RCTs able to address and which do they fail to answer? The first of its kind, Randomized Control Trials in the Field of Development: A Critical Perspective provides answers to these questions, explaining how RCTs work, what they can achieve, why they sometimes fail, how they can be improved and why other methods are both useful and necessary. Bringing together leading specialists in the field from a range of backgrounds and disciplines (economics, econometrics, mathematics, statistics, political economy, socioeconomics, anthropology, philosophy, global health, epidemiology, and medicine), it presents a full and coherent picture of the main strengths and weaknesses of RCTs in the field of development. Looking beyond the epistemological, political, and ethical differences underlying many of the disagreements surrounding RCTs, it explores the implementation of RCTs on the ground, outside of their ideal theoretical conditions and reveals some unsuspected uses and effects, their disruptive potential, but also their political uses. The contributions uncover the implicit worldview that many RCTs draw on and disseminate, and probe the gap between the method's narrow scope and its success, while also proposing improvements and alternatives.
Without disputing the contribution of RCTs to scientific knowledge, Randomized Control Trials in the Field of Development warns against the potential dangers of their excessive use, arguing that the best use for RCTs is not necessarily that which immediately springs to mind. Written in plain language, this book offers experts and laypeople alike a unique opportunity to come to an informed and reasoned judgement on RCTs and what they can bring to development.
More details
Other editions
Additional editions
Florent Bedecarrats | Isabelle Guerin | Francois Roubaud
Randomized Control Trials in the Field of Development
A Critical Perspective
Book
10/2020
Oxford University Press
€141.00
Shipment within 15-20 days
Persons
Florent Bédécarrats is head of the data management unit at Nantes Métropole.
Isabelle Guérin is Senior Research Fellow at IRD (French Institute of Research for Sustainable Development), Cessma (Center for Social Science Studies on the African, American, and Asian Worlds), Affiliate Researcher at the French Institute of Pondicherry, and Member, School of Social Science, Institute for Advanced Study (2019-2020).
François Roubaud is Senior Research Fellow at IRD (French Institute of Research for Sustainable Development), DIAL (Development, Institutions, Globalisation).
Content
- Cover
- Randomized Control Trials in the Field of Development: A Critical Perspective
- Copyright
- Acknowledgments
- Contents
- List of Illustrations
- List of Tables
- Notes on Contributors
- Editors' Introduction: Controversies around RCT in Developmen: Epistemology, Ethics, and Politics
- 0.1 The Arguments behind the Debate: Epistemological, Political, and Ethical
- 0.1.1 The Epistemology of RCTs in the Field of Development
- 0.1.2 RCTs and "Development"
- 0.1.3 Ethics and RCTs
- 0.2 What Is the Scope of Application for RCTs?
- 0.2.1 Private, Market and Humanitarian Goods
- 0.2.2 Evaluate Impact or Test Behaviour?
- 0.2.3 Measuring versus Explaining
- 0.3 Why Is a Scientific Controversy Needed and Why Has It Not Taken Place?
- 0.3.1 Avoiding the Controversy, but Listening and Adapting
- 0.3.2 Can We Really Afford Not to Have a Controversy Considering the Crowding-out Effects?
- 0.4 What Are the Research Alternatives?
- 0.5 Outline of the Book
- Acknowledgement
- Introduction: Randomization in the Tropics Revisited, a Theme and Eleven Variations
- I.1 Are RCTs the Best Way of Learning, or of Accumulating Useful Knowledge?
- I.2 Statistical Inference Is Simpler in RCTs than with Other Methods
- I.3 RCTs Are Rigorous and Scientific
- I.4 External Validity
- I.5 Pre-registration of Trials
- I.6 Experimentation: Kick It and See
- I.7 RCTs and Other Methods
- I.8 Small versus Large
- I.9 Models
- I.10 Causality
- I.11 Ethics
- Acknowledgements
- 1: Should the Randomistas (Continue to) Rule?
- 1.1 Introduction
- 1.2 Foundations of Impact Evaluation
- 1.3 The Influence of the Randomistas on Development Research
- 1.4 Taking Ethical Objections Seriously
- 1.5 Relevance to Policy-making
- 1.6 Conclusions
- Acknowledgement
- 2: Randomizing Development Method or Madness?
- 2.1 Introduction
- 2.1 National Development and Human Well-Being
- 2.1.1 National Development as a Four-fold Transformation of Countries
- 2.1.2 Levels of Median Income/Consumption Completely Explain Poverty
- 2.1.3 National Development and Broader Measures of Social Progress
- 2.1.4 National Development Brings Elimination of Poverty and High Levels of Human Well-being
- 2.2 RCTs in Development as a Method for Improving Human Well-being
- 2.2.1 Widely Accepted Claim I: The Magnitudes of Gains from National Development Are Orders of Magnitude Larger than from Targeted Programs
- 2.2.2 Widely Accepted Claim II: RCT Studies Do Not Address National Development
- 2.2.3 Needed But False Claim I: The Impact of Any Research (RCT or Otherwise) on National Development (or Sector-Wide Reforms) Is Vanishingly Small
- 2.2.4 Needed but False Claim II: Valuation of Human Well-being Is "Kinky"
- 2.2.5 Needed but False Claim III: RCTs Can Reliably Generate Evidence that Improves Targeted Programs Aimed at Kinky (Aggregate or Specific) Development Goals
- 2.2.6 Needed but (Probably) False Claim IV: Knowledge of the Type RCTs Can Generate Is a Binding Constraint to the Adoption and Implementation of Better Targeted Programs
- 2.3 Conclusion
- 3: The Disruptive Power of RCTs
- 3.1 Introduction
- 3.2 Expanding Knowledge by Creating Variation
- 3.3 The Ubiquity of Suboptimality and the Potential for Innovation
- 3.4 Why RCTs?
- 3.5 Three Examples
- 3.5.1 Microcredit contracts
- 3.5.2 Microcredit Interest Rates
- 3.5.3 Poverty, Migration, and Mobile Money
- 3.6 Market Failure and Private Goods
- 3.7 Conclusion
- Acknowledgement
- 4: RCTs in Development Economics, Their Critics and Their Evolution
- 4.1 Introduction
- 4.2 The Critiques of RCTs
- 4.2.1 The Nothing Magic Critiques
- 4.2.2 The Black Box Critiques
- 4.2.3 The External Validity Critiques
- 4.2.4 The Trivial Significance Critiques
- 4.2.5 The Policy Sausage Critiques
- 4.2.6 The Ethical Critiques
- 4.2.7 The "Too Much" Critiques
- 4.3 The Challenge of Responding to The Critiques
- 4.3.1 What Is a Randomista?
- 4.3.2 The Argument behind the Arguments
- 4.4 The Evolution of the "Movement"
- 4.4.1 Problem Driven Iterative Adaption
- Principle One: Solving Locally Nominated and Defined Problems in Performance
- Principle Two: Create an 'Authorizing Environment' for Decision-making that Encourages 'Positive Deviance' and Experimentation
- Principle Three: It Embeds This Experimentation in Tight Feedback Loops that Facilitate Rapid Experiential Learning
- 4.4.2 PDIA-driven Evolution and Critiques of RCTs
- Nothing Magic
- Black Box
- External Validity
- Trivial Significance
- Policy Sausage
- The Ethical Critiques
- Too Much: The Final Critique
- 4.5 Conclusion
- Appendix to Chapter 4: Full Quotations
- Acknowledgement
- 5: Reducing the Knowledge Gap in Global Health Delivery: Contributions and Limitations of Randomized Controlled Trials
- 5.1 Background: RCTs in Medicine and Global Health
- 5.2 Contributions to Policy and Practice
- 5.3 Unintended Consequences: Growing Gap in Evidence and Funding for Key Health Areas
- 5.4 Challenges and Limitations
- 5.5 Beyond RCTs for the Sustainable Development Goals Era: Observational Evaluation Frameworks for Health System Strengthening and Universal Health Coverage
- 5.6 Conclusion
- 6: Trials and Tribulations: The Rise and Fall of the RCT in the WASH Sector
- 6.1 Introduction: The Need to Think
- 6.1.1 Background: Recent Evidence in Sanitation and Child Health
- 6.1.2 Lessons from a Range of Studies
- 6.1.3 Recent Evidence from High-quality RCTs
- 6.2 The Gold Standard? Challenges for Sanitation RCTs
- 6.2.1 Parameter Heterogeneity: Different WASH RCTs Should Give Different Answers
- 6.2.2 Type 3 Errors, Weak First Stages, and Treatments that Do Not Treat
- 6.2.3 Important Questions that Will Not Be Randomized
- 6.3.4 The Overlooked Issues that Could Be Advanced with RCTs (or Non-RCT Intervention Studies)
- 6.4 Conclusion: Good Use of Good Evidence Is the Only Standard
- 7: Microfinance RCTs in Development Miracle or Mirage?
- 7.1 Introduction
- 7.2 The RCT on Microcredit: A Sinking Flagship Product?
- 7.2.1 A Focus on the Design of the AEJ:AE Special Issue
- 7.2.2 A Focus on the AEJ:AE Special Issue: Main Results
- 7.3 Validity and Scope of the Special Issue: A Critical Assessment
- 7.3.1 Internal Validity
- The Emblematic Case of the Moroccan RCT
- Distortion of the Protocol: Product and Sampling Tweaking
- Poor Data Quality and Measurement Errors
- Beyond the Moroccan RCT: A General Assessment
- 7.3.2 External Validity
- 7.4 Results: From Statistical Biases to Interpretative Biases
- 7.4.1 Making a Clean Sweep of Previous Research
- 7.4.2 Take-up
- 7.4.3 Microcredit, Self-employment and Freedom of Choice
- 7.4.4 Microcredit, Social Expenses, Social Transfers and Self-reliance
- 7.4.5 Microcredit and Over-indebtedness
- 7.5 Conclusion and Discussion
- 7.5.1 Disengagement from a Data Culture
- 7.5.2 Ignoring the Critics
- 7.5.3 Circumventing Scientific Ethics
- 7.5.4 What Remains of the Special Issue?
- Acknowledgement
- 8: The Rhetorical Superiority of Poor Economics
- 8.1 Poor Economics: A Puzzling Success, a Persuasive Rhetoric
- 8.1.1 Theoretical Framework: Workaday Rhetoric, Epistemic Communities and Discourse Analysis
- 8.1.2 Methodology and Outline
- 8.2 Hard Numbers: The Rhetoric of Numbers, the Number as Rhetorical Figure
- 8.2.1 Quantify and Disqualify
- 8.2.2 Ninety-nine Cents, Synecdoche for the Life of the Poor
- 8.3 Graphic Representations: Embodied and Metaphorical Storytelling, Cognitive Framing
- 8.3.1 What Is Kennedy's World? Representing and Reducing the Realm of Possibilities to Two Diagrams
- 8.3.2 The Extended Metaphor of the S-curve: When Ibu Tina Fell into the Poverty Trap
- 8.4 A Staggering Wealth of Anecdotes
- 8.4.1 Doctrine: RCT Data, Antidotes for Misleading Anecdotes?
- 8.4.2 The Practice of Anecdotes: A Paradoxical Plethora
- 8.4.3 Anecdotes Following Social Marketing and Storytelling Principles
- 8.4.4 Anecdotes and Charitable Efficiency: An Effect Demonstrated by Experiments
- 8.4.5 Anecdotes Testifying to the Authors' Ethos: Proximity, Familiarity, Credibility
- 8.4.6 Anecdotes, Didactics, and Distinction
- 8.4.7 The Discreet but Inchoate Heuristic Function of Anecdotes
- 8.5 Two Rhetorical Schemes with Strong Epistemic and Persuasive Effects
- 8.5.1 The Middle Way between Two Extremes: Common Sense, Objectivity, and Manipulative Framing
- 8.5.2 Small Causes, Big Effects: Oxymorons in Defense of the "All Micro"
- 8.6 Concluding Remarks: Persuasive but Poor Narratives
- 8.6.1 An Original Combination of the Three Pillars of Rhetoric
- 8.6.2 The Capacity to Amalgamate Different Audiences around a Common Content
- 8.6.3 Poor Narratives
- Acknowledgement
- 9: Are the "Randomistas" Evaluators?
- 9.1 Introduction
- 9.2 Evading the Hard Lessons of Evaluation History
- 9.2.1 A Faith-based Commitment
- 9.2.2 Weak Philosophical Foundations
- 9.2.3 Resilient Loyalty
- 9.2.4 Evolving Conceptions of Evaluation
- 9.2.5 Back to the Future?
- 9.2.7 Micro-economists Enter the Aid Effectiveness Fray
- 9.2.8 Experimentalism Redux
- 9.2.9 From Social Research to Evaluation
- 9.3 The Potential and Limitations of Experimental Methods
- 9.3.1 The Limitations of RCTs
- 9.3.2 Ethical Concerns
- 9.3.3 Unintended Effects
- 9.3.4 There Are Alternatives
- 9.4 The Current Evaluation Market Favors RCTs
- 9.4.1 The Waves of Evaluation Diffusion
- 9.4.2 The Lure of Medical Research
- 9.4.3 Distorted Incentives
- 9.5 Modest Contributions to Development Knowledge
- 9.5.1 A Narrow Scope
- 9.5.2 A Paternalistic Stance
- 9.5.3 Limited Contributions to Knowledge
- 9.6 RCTs Are One Tool among Many
- 9.6.1 How Evaluative Are RCTs?
- 9.6.2 Wielding the Right Tools
- 9.7 Conclusions
- Acknowledgement
- 10: Ethics of RCTs: Should Economists Care about Equipoise?
- 10.1 Introduction
- 10.2 What Is Equipoise?
- 10.3 Equipoise vs. Blindness
- 10.4 Should Economists Care about Equipoise?
- 10.5 Conclusion
- Acknowledgements
- 11: Using Priors in Experimental Design: How Much Are We Leaving on the Table?
- 11.1 Introduction
- 11.2 The Elicitation and Use of Priors
- 11.3 The Use of Priors in Study Design
- 11.4 Back-of-the-Envelope Estimates of Benefits of Using Priors
- 11.5 Conclusion
- 12: Epilogue: Randomization and Social Policy Evaluation Revisited
- 12.0 Preamble
- 12.0.1 The First Awakening
- 12.0.2 The Second Awakening
- 12.1 Introduction
- 12.2 Questions of Interest in Evaluating a Prototypical Social Program
- 12.3 The Evaluation Problem
- 12.3.1 A Model of Program Evaluation
- 12.3.2 The Parameters of Interest in Program Evaluation
- 12.4 The Case For and Against Randomized Social Experiments
- 12.5 Evidence on Randomization Bias
- 12.6 At What Stage Should Randomization Be Implemented?
- 12.7 The Tension between the Case for Social Experiments as a Substitute for Behavioral Models and Social Experiments as Supplementary Source of Information
- 12.8 Summary of the 1992 Paper
- 12.9 Postscript, 2019
- Interviews
- Interview with Jean-Paul Moatti and Rémy Rioux
- Interview with Gulzar Natarajan
- Annex A. A List of Research Agenda on Indian Economy
- Annex B. Issues of concern to policymakers
- Interview with Ila Patnaik
- References
- Index
