"This first guidance note, Introduction to Impact Evaluation, provides an overview of impact evaluation, explaining how impact evaluation differs from – and complements – other types of evaluation, why impact evaluation should be done, when and by whom. It describes different methods, approaches
...
and designs that can be used for the different aspects of impact evaluation: clarifying values for the evaluation, developing a theory of how the intervention is understood to work, measuring or describing impacts and other important variables, explaining why impacts have occurred, synthesizing results, and reporting and supporting use. The note discusses what is considered good impact evaluation – evaluation that achieves a balance between the competing imperatives of being useful, rigorous, ethical and practical – and how to achieve this. Footnotes throughout the document contain references for further reading in specific areas." (Introduction, page 1)
more
"Since impact evaluations focus on results and what contributes to results, logically there should be a greater enthusiasm for making use of the findings. Impact evaluations also tend to cost more than other types of evaluation, implying that they should command significant follow up as well. It tur
...
ns out, however, that credible evidence and significant costs are not sufficient to ensure that impact evaluation findings are used. This guidance note looks to several additional factors that contribute to effective evaluation use. Using impact evaluations depends importantly on how one designs and conducts them. As was illustrated in Guidance Note 3 on mixed methods, the choice of evaluation method should follow from the questions one is trying to answer. But whatever questions are being tackled, whatever evaluation methods are employed, to make the best and proper use of evaluation findings we can apply a common set of practices and insights." (Introduction)
more
"Mixed methods (MM) evaluations seek to integrate social science disciplines with predominantly quantitative (QUANT) and predominantly qualitative (QUAL) approaches to theory, data collection, data analysis and interpretation. The purpose is to strengthen the reliability of data, validity of the fin
...
dings and recommendations, and to broaden and deepen our understanding of the processes through which program outcomes and impacts are achieved, and how these are affected by the context within which the program is implemented. While mixed methods are now widely used in program evaluation, and evaluation RFPs frequently require their use, many evaluators do not utilize the full potential of the MM approach."
more
"This tool is used to assess the SBCC knowledge and competence of individuals and how successful a specific training has been for them. Individuals about to take an SBCC training fill in the pre-assessment provided in this document, then fill in the post-assessment right after completing the trainin
...
g. The questions and items are the same for both assessments. Individuals completing a successful training and filling in the post-assessment will answer a higher number of questions correctly and rate their own SBCC competencies at increased levels. It is the comparison that matters. The difference between an individual’s scores and ratings in the two assessments allows the training to be monitored and measurably evaluated. The score indicates whether an individual increased his or her SBCC knowledge, skills, and competencies during the training, and the ratings can suggest whether further training or capacity strengthening is needed and in which specific SBCC competencies." (Introduction, page 4)
more
"This book offers an accessible introduction to the topic of impact evaluation and its practice in development. Although the book is geared principally toward development practitioners and policy makers, we trust that it will be a valuable resource for students and others interested in impact evalua
...
tion. Prospective impact evaluations assess whether or not a program has achieved its intended results or test alternative strategies for achieving those results. We consider that more and better impact evaluations will help strengthen the evidence base for development policies and programs around the world [...] The three parts in this handbook provide a nontechnical introduction to impact evaluations, discussing what to evaluate and why in part 1; how to evaluate in part 2; and how to implement an evaluation in part 3. These elements are the basic tools needed to successfully carry out an impact evaluation." (Preface)
more
"This book off ers an accessible introduction to the topic of impact evaluation and its practice in development. Although the book is geared principally toward development practitioners and policy makers, we trust that it will be a valuable resource for students and others interested in impact evalu
...
ation. Prospective impact evaluations assess whether or not a program has achieved its intended results or test alternative strategies for achieving those results. We consider that more and better impact evaluations will help strengthen the evidence base for development policies and programs around the world. Our hope is that if governments and development practitioners can make policy decisions based on evidence—including evidence generated through impact evaluation—development resources will be spent more eff ectively to reduce poverty and improve people’s lives. The three parts in this handbook provide a nontechnical introduction to impact evaluations, discussing what to evaluate and why in part 1; how to evaluate in part 2; and how to implement an evaluation in part 3. These elements are the basic tools needed to successfully carry out an impact evaluation. The approach to impact evaluation in this book is largely intuitive, and we attempt to minimize technical notation. We provide the reader with a core set of impact evaluation tools—the concepts and methods that underpin any impact evaluation—and discuss their application to real-world development operations. The methods are drawn directly from applied research in the social sciences and share many commonalities with research methods used in the natural sciences. In this sense, impact evaluation brings the empirical research tools widely used in economics and other social sciences together with the operational and political-economy realities of policy implementation and development practice.
From a methodological standpoint, our approach to impact evaluation is largely pragmatic: we think that the most appropriate methods should be identified to fit the operational context, and not the other way around. This is best achieved at the outset of a program, through the design of prospective impact evaluations that are built into the project’s implementation. We argue that gaining consensus among key stakeholders and identifying an evaluation design that fits the political and operational context are as important as the method itself. We also believe strongly that impact evaluations should be candid about their limitations and caveats. Finally, we strongly encourage policy makers and program managers to consider impact evaluations in a logical framework that clearly sets out the causal pathways by which a program works to produce outputs and influence final outcomes, and to combine impact evaluations with monitoring and complementary evaluation approaches to gain a full picture of performance." (Preface)
more
"This paper explores why measuring the impacts of information and communication technology (ICT) is important for development – and why it is statistically challenging. Measuring impacts in any field is difficult, but for ICT there are added complications because of its diversity and rapidly chang
...
ing nature. A number of impact areas are identified in section 1, and their relationships explored, in the context of their place in the social, economic and environmental realms. The result is a complex web of relationships between individual impact areas, such as economic growth and poverty alleviation, and background factors, such as a country’s level of education and government regulation.
Existing measurement frameworks are described in section 1, and relevant statistical standards examined. The latter includes internationally agreed standards for the ICT sector, ICT products and ICT demand. The contribution of the Partnership on Measuring ICT for Development and its member organizations to ICT measurement, and its goals for measuring ICT impacts are outlined. Methodologies used in the measurement of ICT are discussed and compared in section 2 of the paper, and empirical evidence reviewed, in section 3. Most research conducted has found positive effects of ICT in the impact areas investigated. However, research has tended to focus on positive, rather than negative impacts; therefore, the latter tend to be indicated by anecdotal evidence. There is relatively little evidence from developing countries and there are indications that findings in respect of developed countries may not apply to developing countries. In respect of both developed and developing countries, there are few studies that provide internationally comparable evidence. The difficulties of ICT impact measurement, major data gaps and the lack of clear statistical standards suggest several issues for consideration. These are presented in the final section of the paper." (Abstract)
more
"The Nonprofit Outcomes Toolbox identifies stages in the use of outcomes and shows you how to use specific facets of existing outcome models to improve performance and achieve meaningful results. Going beyond the familiar limits of the sector, this volume also illustrates how tools and approaches lo
...
ng in use in the corporate sector can be of great analytical and practical use to nonprofit, philanthropic, and governmental organizations." (Publisher description)
more
"This book reviews quantitative methods and models of impact evaluation, presenting an analysis of the quantitative research underlying recent programme evaluations and case studies prepared for a series of impact evaluation workshops in different countries, sponsored by the World Bank Institute (WB
...
I). The handbook also details challenges and goals in other realms of evaluation, including monitoring and evaluation, operational evaluation, and mixed-methods approaches combining quantitative and qualitative analyses. It is designed to put theory on evaluation methods and practices into practice in a hands-on fashion for practitioners, especially researchers new to the evaluation field and policymakers involved in implementing development programmes worldwide." (Publisher description)
more
"This publication allows community radio stations to assess their performance regarding: community participation and ownership; radio governance structures and procedures; radio programme structure; radio station management; financial management and resource structure; as well as networking. Using a
...
detailed scoring system, the manual provides a comprehensive list of indicators that categorises assessed stations into four groups: evolving, progressing, performing and model community radios. It considers the issues that are at the heart of community media: public accountability, community representation, locally relevant programming, diverse funding, and due acknowledgement of staff, including volunteers. The manual is clear and concise providing a sound basis for the task it describes. Tailored to the needs of community radios in Nepal, not every single indicator may apply to stations in other countries. Nevertheless, the scoring methodology can easily be adapted to other contexts." (CAMECO Update 4-2009)
more
"This 2009 version of the ‘Handbook on Planning, Monitoring and Evaluating for Development Results’ aims to support UNDP in becoming more results-oriented and to improve its focus on development changes and real improvements in people’s lives. It replaces and updates the previous Handbook from
...
2002. It was a breakthrough in driving the effective application of the results-based management approach in programming and performance management. The Handbook recognizes that planning, monitoring and evaluation require a focus on nationally owned development priorities and results, and should reflect the guiding principles of national ownership, capacity development and human development. Globally, there has been a significant shift away from the project approach in favour of programme and national approaches." (Preface, page III)
more
"Die anvisierte Leserschaft dieser Landkarte ist die entwicklungspolitische Fachwelt, die mit der Analyse von tatsächlichen, das heißt empirisch nachprüfbaren (de facto) Wirkungen befasst ist und die einen schnellen, informativen Zugriff auf das Vorhandene benötigt. Dies sind in erster Linie Pro
...
jekt- und Programmverantwortliche in den Entwicklungsorganisationen in Geber- und Empfängerländern, die gehalten sind, die Wirkungen von Entwicklungsmaßnahmen nachzuweisen und in ihren Berichten zu dokumentieren; Mitarbeiter/-innen von Evaluierungs- und Qualitätsmanagementeinheiten, die Wirkungsanalysen zumeist als Teil von (Querschnitts-)Evaluationen in Auftrag geben; Wissenschaftler/-innen, die bei dem Design von empirischen Wirkungsstudien beraten oder diese selbst durchführen; sowie Gutachter/-innen und Berater/-innen, die vor der Aufgabe stehen, Wirkungsanalyseverfahren anzuwenden oder in einer Organisation einzuführen. In der vorliegenden Landkarte wurde versucht, alle, insbesondere im deutschsprachigen Kontext bekannten und in der Praxis erprobten, gut beschriebenen Wirkungsanalyseverfahren aufzunehmen, wobei ein Anspruch auf Vollständigkeit nicht erhoben werden kann. Dabei liegt der Schwerpunkt nicht nur auf qualitativen und partizipativen Verfahren, die selbst oft zusätzliche quantitative Elemente enthalten, sondern es werden auch quantitative experimentelle und nicht-experimentelle Designs beschrieben, die mitunter als „rigorose“ Verfahren bezeichnet werden. Dabei werden hier keine Bewertungen im Sinne von gut oder schlecht ausgesprochen. Ziel ist es stattdessen, den Nutzern dieser Landkarte die Möglichkeit zu geben, Verfahren nach rein sachlichen und pragmatischen Erwägungen für einen bestimmen Untersuchungszweck oder -auftrag auszuwählen." (Vorwort)
more
"This guide outlines a step-by-step process for developing a performance story report in the context of natural resource management. The guide can be used in a range of contexts by evaluators and others who want to use participatory evaluation approaches to report on outcomes. Users may include cons
...
ultants, government and non-government organisations, community groups, students, natural resource management organisations, industry bodies and academics. The guide will enable them to evaluate and report on program outcomes and continually improve their programs based on lessons learned through participatory evaluation approaches. Performance story evaluation is useful for both internal and external evaluations and learning. Performance story reports are effective for reporting annually on achievements from one year to the next." (Introduction, page 5)
more
"This report investigates the impact media and ICTs can have on the lives of the poor, based on the experiences of nine donors and NGOs forming part of the "Building Communication Opportunities (BCO)" alliance. It suggests that radio will have the most influence on social and political change where
...
it is widely accessible, trusted by listeners, and open to inclusive participation. ICTs can help make markets work for the poor, but the surrounding circumstances are highly influential in determining in how far they make a difference. Communication networks appear to be particularly effective in building communities of activists where they enable the pooling of resources and expertise and leverage wider influence on decision-makers. However, the report concludes that evidence of the impact of ICTs is still weak. More debate is needed about how ICTs are best deployed. This requires learning how people really use the tools, as well as a more effective assessment of past and current experiences." (CAMECO Update 1-2009)
more