"The aim of this paper is to provide guidance on the evaluation of complex interventions in international development. Our contribution to the literature is threefold. First, unlike other reviews on the same subject, our focus is exclusively directed to evaluations of development interventions. Seco
...
nd, we identify methods that are new or little used in the evaluation literature. Third, we map methods to project types with the aim of identifying methods that are useful to the evaluation of particular projects.
We start by making a distinction between complex interventions and complex systems. While much recent research has been devoted to understanding how interventions can be implemented and understood in complex systems, our interest rests primarily with the goal of understanding and evaluating complex interventions. We therefore abstract from complexities arising from the interaction between interventions and complex systems. In our framework, complex interventions can be implemented in simple as well as in complex systems, but the latter are not the primary focus of our paper. Removing the consideration of interactions between interventions and complex systems reduces the scope of our review, but helps focus attention on a manageable number of issues and methods.
We define complex interventions as interventions characterised by multiple components, multiple stakeholders, or multiple target populations, or interventions incorporating multiple processes of behavioural change. These interventions cannot be represented by single-intervention or single-outcome models, and present several challenges to evaluation. Occasionally, or purposely, these interventions give rise to emergent outcomes such as non-linear effects, tipping points, and multiple equilibria. The presence of emergent outcomes of this type adds additional difficulties to evaluation because existing methods are ill-equipped to detect and estimate non-linear impacts of interventions." (Inroduction)
more
"This timely and comprehensive textbook covers theoretical and practical issues related to evaluation of programs, particularly social programs and projects, with an emphasis on viewing evaluation topics through a social justice, diversity, and inclusive perspective. It provides students with an equ
...
al appreciation of both the art (for example engaging stakeholders, communicating across diverse settings, recruiting participants) and science (writing questions that matter, using credible designs, data analysis) of evaluation planning, implementation, and dissemination. This textbook will help students recognize that all populations are diverse, and that high quality evaluations need to pay attention to the diversity of all populations being served." (Publisher description)
more
"These are simple recommendations on the inclusion of data on disability for researchers conducting evidence syntheses and primary studies." (Page 1)
"The resource provides a guide for the expected costs of various evaluative activities in an Australian context. The main purpose for commissioners of evaluative activity to have a sense of the budget they should set aside for these activities. However, the guide also alerts potential users to the r
...
ange of possible evaluative activities. 'Scale' in the guide refers to a combination of the following interrelated factors: how many people or other units of analysis are the 'target' of the program; whether the project is made up of a variety of sub-programs/projects that are distinct; the geographic spread of the program; the range of key evaluation questions that need to be answered and the range of data collection methods that this implies; whether the evaluative activities will include any 'hard-to-reach' populations; whether the evaluation will involve particular skill-sets, e.g. culturally sensitive evaluation team members." (https://www.betterevaluation.org)
more
"Addresses the challenges of conducting program evaluations in real-world contexts where evaluators and their clients face budget and time constraints. The book is organized around the authors’ seven-step model that has been tested in workshops and practice environments to help the evaluation impl
...
ementers and managers make the best choices when faced with real world constraints. The Third Edition includes a new chapter on gender equality and women’s empowerment and discussion of digital technology and data science." (Publisher description)
more
"This text provides a solid foundation in program evaluation, covering the main components of evaluating agencies and their programs, how best to address those components, and the procedures to follow when conducting evaluations. Different models and approaches are paired with practical techniques,
...
such as how to plan an interview to collect qualitative data and how to use statistical analyses to report results. In every chapter, case studies provide real world examples of evaluations broken down into the main elements of program evaluation: the needs that led to the program, the implementation of program plans, the people connected to the program, unexpected side effects, the role of evaluators in improving programs, the results, and the factors behind the results. In addition, the story of one of the evaluators involved in each case study is presented to show the human side of evaluation." (Publisher description)
more
"Evaluation: A Systematic Approach is the best-selling comprehensive introduction to the field of program evaluation, covering the range of evaluation research activities used in appraising the design, implementation, effectiveness, and efficiency of social programs. Evaluation domains are presented
...
in a coherent framework that not only explores each, but recognizes their interrelationships, their role in improving social programs and the outcomes they are designed to affect, and their embeddedness in social and political context. Relied on as the “gold standard” by professors, students, and practitioners for 40 years, the new Eighth Edition includes a new practical chapter on planning an evaluation, entirely new examples throughout, and a major re-organization of the book’s content to better serve the needs of program evaluation courses." (Publisher description)
more
"Demystifying the evaluation journey, this is the first evaluation mentoring book that addresses the choices, roles, and challenges that evaluators must navigate in the real world. Experienced evaluator and trainer Donna R. Podems covers both conceptual and technical aspects of practice in a friendl
...
y, conversational style. She focuses not just on how to do evaluations but how to think like an evaluator, fostering reflective, ethical, and culturally sensitive practice. Extensive case examples illustrate the process of conceptualizing and implementing an evaluation--clarifying interventions, identifying beneficiaries, gathering data, discussing results, valuing, and developing recommendations. The differences (and connections) between research, evaluation, and monitoring are explored. Handy icons identify instructive features including self-study exercises, group activities, clarifying questions, facilitation and negotiation techniques, insider tips, advice, and resources." (Publisher description)
more
"Systemic evaluation design continuously addresses six questions: 1. What are the intervention purposes? 2. What is the scope and focus of the intervention and evaluation? 3. What ought to be the consequences of the evaluation and what evaluation purposes promote those consequences? 4. What are the
...
criteria (or values) that should underpin the judgment of merit, worth and significance? 5. What questions inform the collection of data so that judgments can be made using those criteria? 6. How can the evaluation be feasible? A systemic approach can be summarised as: understanding inter-relationships, engaging with multiple perspectives and reflecting on boundary choices. Systemic evaluation design flows from three key principles: 1. Systemic evaluation design is a process and a product. Evaluation design is a process that occurs throughout the evaluation, not just a product of the first stage of an evaluation. 2. Systemic evaluation design focuses on consequences. And by that I mean the consequences of the evaluation. Some might call it outcomes. 3. Systemic evaluation design emphasises what to leave out rather than what to put in. It is not possible to include everything that happened in the intervention, nor possible to include every single perspective or viewpoint or framing of the intervention." (Page ii)
more
"Evaluation Practice for Collaborative Growth highlights the approaches, tools, and techniques that are most useful for evaluating educational and social service programs. This book walks the reader through a process of creating answerable evaluations questions, designing evaluation studies to answe
...
r those questions, and analyzing, interpreting, and reporting the evaluation's findings so they are useful and meaningful for key stakeholders. The text concludes with a devoted to the shifting landscape of evaluation practice as it faces complex systems and issues that are shaped by society. Additionally, the author provides a list of knowledge and skills needed to adapt to a changing landscape and encourages organizations to use evaluation as a mechanism for learning and adapting to change. Her orientation toward community-based approaches and social justice prevail throughout the book's content and align well with a reader's desire to be inclusive and accountable in programing efforts. Nonprofit leaders, social science professionals, and students will find this book helpful for understanding basic program evaluation concepts, methods, and strategies." (Publisher description)
more
"The goal of this handbook is to provide guidance to practitioners and organizations interested in evaluating mental health prevention and early intervention (PEI) programs. In the course of developing an evaluation plan for a series of PEI programs implemented statewide by the California Mental Hea
...
lth Services Authority (CalMHSA), we found many resources that discussed general approaches to evaluating mental health programs and policies but few that discussed how to tailor an evaluation approach to mental health PEI activities in particular. PEI activities may require different evaluation strategies than treatment interventions do. First, the activities being administered are different in that they are not clinical interventions. Indeed, PEI activities are highly diverse and provided in many different types of nonclinical settings. Thus, the process outcomes that are used as indicators of successful implementation tend to look different for PEI interventions." (Introduction)
more
"El poder atribuir los cambios a nuestra intervención es el principal desafío de los evaluadores. Conceptos como eficacia, eficiencia e impacto ocupan la atención de los evaluadores. Pero ¿este es el único enfoque para evaluar los cambios y las trasformaciones vinculadas a nuestras intervencion
...
es? ¿Qué sucedería si consideramos a la intervención como el "cliente" y al donante como una parte interesada, junto a las comunidades y a otras instituciones y actores que puedan beneciarse o perjudicarse con el proyecto? ¿Qué pasaría si en lugar de medir impactos reexionamos sobre las consecuencias éticas y políticas de nuestra intervención?. ¿Y si ponemos en valor la sostenibilidad de los cambios en lugar de evaluar el cumplimiento (o justicar el incumplimiento) de las actividades previstas?" (Contratapa)
more
"The Monitoring and Evaluation Capacity Assessment Toolkit (MECAT) is a set of tools that guide organizations1 through a process to assess their current monitoring and evaluation (M&E) capacity, identify gaps, and plan ways to strengthen their M&E systems. MECAT uses four methods and supporting tool
...
s in the assessment process: (1) a group assessment, (2) an individual assessment, (3) key informant interviews, and (4) a desk review. With this approach, organizations, national health programs, and subnational health teams can accurately assess program strengths and weaknesses and plan the steps needed to strengthen the M&E functions." (Introduction)
more
"Robust monitoring, evaluation, reporting, and learning (MERL) are critical components of successful programming. The MERL components enable program stakeholders to monitor progress and evaluate the achievement of expected results. Reporting processes and timelines should be clearly defined and tail
...
ored to meet the needs of key audiences and stakeholders, and provision should be made for the program to continually reflect and learn from experiences gained during implementation. Measuring the success of peacebuilding programs poses specific challenges that are unique to this program area. This module was developed to guide PEACE III local program partners—peacebuilding practitioners—through the development and implementation of effective and practical MERL systems for their projects. This five-year cross-border peacebuilding program is implemented by Pact in partnership with Mercy Corps and a range of local partners with activities in Ethiopia, Kenya, Somalia, South Sudan, and Uganda. PEACE III aims to strengthen cross-border conflict management in the Horn of Africa and is pursuing two related objectives: 1) to strengthen local cross-border conflict management and 2) to improve the responsiveness of regional and national institutions to cross-border conflict. This manual was created to support and provide examples to peacebuilding practitioners and is an addition to Pact’s existing MERL Modules. Other useful examples of MERL training manuals for peacebuilding programs exist. This module does not seek to replicate those manuals, but rather draws on them and integrates their expertise here. Parts of this manual were also drawn from Pact’s MERL Modules but customized with practices and examples relevant for peacebuilding programs. It is the authors’ hope that this module can serve as an introduction to MERL for peacebuilding practitioners and can point them to other relevant resources in the field. Each begins with an outline of the learning objectives, includes learning activities throughout the chapters, and ends with a summary of key points and learning." (Introduction)
more