Information--regular, systematic, reliable--is the life-blood of democracy and the fuel of effective management. Surely today there is no problem with information, for this is the age of information overload. It pours onto our computer screens and out of our printers. Indeed, many governments claim, often with some justification, to be more open and transparent than ever before. But what if the life-blood is contaminated, or the fuel polluted? Then the body politic sickens and the engine of public management runs rough. It is the vital issue of the quality of the information we receive that this book addresses. Quality Matters compares approaches across different jurisdictional settings and across three different types of information evaluation. The chapters describe and analyze quality assurance in a number of countries and within a variety of international organizations. These have been selected either because they are widely considered to be leaders in evaluating information or because they have experience with assuring quality information that can instruct others. Contributors are from Australia, Canada, the European Union, France, the Netherlands, New Zealand, Sweden, Switzerland, United Kingdom, United States, and the World Bank. This pioneering study analyzes practices for assuring the quality of evaluation, performance auditing, and reporting in the face of political, organizational, and technical obstacles. A final chapter addresses the extent to which quality assurance systems become bothersome rituals or remain meaningful mechanisms to ensure quality control. This well-structured volume will be of particular interest to policymakers and adds much to the literature on program evaluation and performance auditing.
Introduction 1. Assuring the Quality of Evaluative Information 1 John Moyne and Robert Schwartz
Part 1: Evaluation 2. Devising and Using Evaluation Standards: 21 The French Paradox Jean-Claude Barbier 3. Instruments and Procedures for Assuring 41 Evaluation Quality: A Swiss Perspective Thomas Widmer 4. Triple Check for Top Quality or Triple Burden? 69 Assessing EU Evaluations Jaques Toulemonde, Hilkka Summa-Pollitt, and Neil Usher 5. Quality of Evaluative Information at the World Bank 91 Patrick G. Gras so 6. The Netherlands Court of Audit and Meta-Research: 113 Principles and Practice Andrea Kraan and Helenne van Adrichem 1. Auditing the Evaluation Function in Canada 129 Bob Segsworth and Stellina Volpe 8. Guidelines and Standards: Assuring the Quality 145 of Evaluation and Audit Practice by Instruction M. L Bemelmans-Videc
Part 2: Performance Audits 9. "Neat and Tidy.. .and 100% Correct": Assuring the 173 Quality of Supreme Audit Institution Performance Audit Work Jeremy Lonsdale and John Mayne. Part 3: Performance Reports 10. Professionals, Self-Evaluation, and Information 199 in the UK: The Higher Education Research Assessment Exercise and Clinical Governance Andrew Gray and Bill Jenkins 11. Decentralization Does Not Mean Poor Data Quality: 215 A Case Study from the U.S. Department of Education Alan L Ginsburg and Natalia Pane 12. "Believe it or not?": The Emergence of 237 Performance Information Auditing John Mayne and Peter Wilkins 13. How Supreme Audit Institutions Help to Assure 261 the Quality of Performance Reporting to Legislatures Stan Divorski 14. Assessment of Performance Reports: A Comparative 279 Perspective Richard Boyle
Part 4: Conclusion 15. Does Quality Matter? Who Cares about the Quality 301 of Evaluative Information? Robert Schwartz and John Mayne.