Document details

Impact Evaluation in Practice

Contains figures, tables, glossary pp. 229-236, index

ISBN 978-0-8213-8541-8 (print); 978-0-8213-8593-7 (ebook)

"This book off ers an accessible introduction to the topic of impact evaluation and its practice in development. Although the book is geared principally toward development practitioners and policy makers, we trust that it will be a valuable resource for students and others interested in impact evaluation. Prospective impact evaluations assess whether or not a program has achieved its intended results or test alternative strategies for achieving those results. We consider that more and better impact evaluations will help strengthen the evidence base for development policies and programs around the world. Our hope is that if governments and development practitioners can make policy decisions based on evidence—including evidence generated through impact evaluation—development resources will be spent more eff ectively to reduce poverty and improve people’s lives. The three parts in this handbook provide a nontechnical introduction to impact evaluations, discussing what to evaluate and why in part 1; how to evaluate in part 2; and how to implement an evaluation in part 3. These elements are the basic tools needed to successfully carry out an impact evaluation. The approach to impact evaluation in this book is largely intuitive, and we attempt to minimize technical notation. We provide the reader with a core set of impact evaluation tools—the concepts and methods that underpin any impact evaluation—and discuss their application to real-world development operations. The methods are drawn directly from applied research in the social sciences and share many commonalities with research methods used in the natural sciences. In this sense, impact evaluation brings the empirical research tools widely used in economics and other social sciences together with the operational and political-economy realities of policy implementation and development practice.
From a methodological standpoint, our approach to impact evaluation is largely pragmatic: we think that the most appropriate methods should be identified to fit the operational context, and not the other way around. This is best achieved at the outset of a program, through the design of prospective impact evaluations that are built into the project’s implementation. We argue that gaining consensus among key stakeholders and identifying an evaluation design that fits the political and operational context are as important as the method itself. We also believe strongly that impact evaluations should be candid about their limitations and caveats. Finally, we strongly encourage policy makers and program managers to consider impact evaluations in a logical framework that clearly sets out the causal pathways by which a program works to produce outputs and influence final outcomes, and to combine impact evaluations with monitoring and complementary evaluation approaches to gain a full picture of performance." (Preface)
PART ONE. INTRODUCTION TO IMPACT EVALUATION, 1
1 Why Evaluate? 3
2 Determining Evaluation Questions, 21
PART TWO. HOW TO EVALUATE, 31
3 Causal Inference and Counterfactuals, 33
4 Randomized Selection Methods, 49
5 Regression Discontinuity Design, 81
6 Difference-in-Differences, 95
7 Matching, 107
8 Combining Methods, 117
9 Evaluating Multifaceted Programs, 129
PART THREE. HOW TO IMPLEMENT AN IMPACT EVALUATION, 139
10 Operationalizing the Impact Evaluation Design, 143
11 Choosing the Sample, 171
12 Collecting Data, 199
13 Producing and Disseminating Findings, 211
14 Conclusion, 223