Toolbox Evaluation & Impact Assessment

Evaluation & Impact Assessment

TOOLBOX

Evaluation & Impact Assessment

This toolbox offers you training materials and good practice experiences on evaluation and impact assessment. We focus in particular on

We strive to include a wide variety of sources. Are your favourite materials not yet included? We would be delighted if you would share them with us.

Filter

569

Sort by:

Direction:

Download (20)

Multi‐Method Analysis of Locally Built Artificial Intelligence Models Designed for Journalists in an African Country

Akintunde Babatunde Abdullateef Mohammed Adeola Abdulateef Elega

Media and Communication, volume 14 (2026), 19 pp.

"Artificial intelligence (AI) models for journalism have been built within the Global North context, hence deepening the concerns of the “AI divide” between the North and South. This has compelled organizations in various areas of the Global South to develop journalism‐focused AI models to add ... more

Download (123)

Considerations and Practical Applications for Using Artificial Intelligence (AI) in Evaluations

D. Cekova, L. Corsetti, S Ferretti, S. Vaca

Rome: CGIAR Independent Advisory and Evaluation Service (IAES) (2025), 42 pp.

" Guided by the CGIAR-wide Evaluation Framework, integrating AI tools requires a governance approach to balance innovation with ethical responsibility, ensuring transparency, fairness, accountability, and inclusivity. This Technical Note encourages and guides CGIAR evaluators to ethically explore, n ... egotiate, and experiment with AI tools: Explore: Evaluators are invited to discover how AI, especially GenAI, can enhance evaluation efficiency, from scoping and data analysis to reporting. The Note provides practical guidance on AI applications and examples to support creative yet responsible exploration. Negotiate: Integrating AI should be openly discussed with commissioners, stakeholders, and teams. The Note prioritizes jointly defining boundaries, expectations, and ethical parameters (transparency, accountability, and data sensitivity) at each phase of evaluation. Use AI Responsibly: While AI tools are evolving, evaluators are encouraged to pilot and iterate their use. The document supports experimentation through practical tips, prompt examples, and tool selection criteria, all while emphasizing documentation and learning from each use case. Effective AI governance is grounded in core principles: Transparency requires clear documentation of AI tool usage, data sources, model limitations, and decision-making processes. Accountability involves assigning responsibility for AI decisions and outputs and establishing oversight and redress mechanisms. Fairness and inclusion must proactively mitigate bias and discrimination, with particular attention to underrepresented groups and data gaps. Data privacy and security must align with applicable data protection regulations and ensure secure handling practices. Human oversight ensures that evaluators retain control over processes and can intervene as needed. In operationalizing ethical AI governance in CGIAR evaluations, due diligence is required in assessing AI tools for ethical alignment before deployment: reviewing the transparency of vendors, the documentation of models, and their intended use cases. Where relevant, components involving AI—especially those engaging human subjects or sensitive data—should undergo ethics review. AI applications must be adapted to the local and cultural contexts in which evaluations are conducted, as what is suitable in one setting may be inappropriate in another. Additionally, participants should be informed about the use of AI systems and the implications of data collection or processing to ensure informed consent. Ethical AI governance should be embedded in the entire evaluation lifecycle. During the design phase, evaluators should define AI tools to use, why they are selected, and assess risks. In data collection, AI tools should be used in ways that uphold data privacy and protection standards and avoid reinforcing harmful stereotypes or excluding groups. During the analysis phase, the role of AI in supporting interpretation should be documented, with an acknowledgment of limitations or biases. In dissemination, documentation, and reporting, AI’s contribution, limitations, and human validation should be disclosed. By rapid adaptation of content across formats, languages, and complexity levels, AI opens possibilities for broader, more inclusive communication of findings. Finally, the follow-up phases should include a reflection on the ethical implications observed and how these lessons can improve future evaluations." (Executive summary) more

Download (53)

Made in Africa: Artificial Intelligence for Monitoring, Evaluation, Research and Learning (MERL). A Practitioner Perspective and Landscape Study

Varaidzo Magodo-Matimba

MERL Tech Initiative (2025), 49 pp.

"This report examines whether and how AI might serve African monitoring, evaluation, research and learning (MERL) systems—or whether these technologies should be adopted at all. We argue that Africa's relationship with AI must be shaped through locally-defined priorities, community participation, ... and epistemic sovereignty rather than externally imposed technological agendas. While AI promises enhanced data analysis and real-time insights, our research reveals fundamental misalignments between current AI tools and African contexts. The systematic absence of African languages from AI training data creates erasure, rendering entire communities invisible in evidence gathering. AI tools consistently fail to surface African research, instead reproducing Western evaluation frameworks inappropriate for African contexts. These are not merely technical limitations but political choices about whose knowledge matters. Using a qualitative design, we interviewed 23 AI experts, technologists, evaluators, civil-society leaders, and policymakers from across Africa and the global MERL ecosystem, complemented by literature review and practitioner consultations with our AI in AfricaMERL Working Group. Our findings: Technical and structural barriers: African MERL practitioners face interconnected challenges including limited digital infrastructure, fragmented data systems, and educational institutions unprepared to integrate AI with evaluation practice. Current regulatory frameworks focus on protecting against AI risks while risking failing to enable African-led innovation; Reframing capacity deficits: The fundamental gap is not African practitioners' ability to use Global North AI systems, but rather these systems' lack of indigenous data, nuance, and contextual sophistication to capture African realities. Dominant evaluation frameworks themselves require capacity development to understand African epistemologies; Made in Africa as resistance and creation: Practitioners define African AI approaches through community ownership, participatory design, African language inclusion, locally-controlled infrastructure, and Ubuntu values prioritizing collective benefit. This represents both resistance to epistemic hegemony and creation of alternative systems; Environmental and human rights concerns: AI's development material and global supply costs, from conflict mineral extraction in the DRC to massive energy consumption, reveal how African resources fuel technological development controlled by and benefiting actors outside the continent." (Executive summary, pages 4-5) more

External Link (348)

Google Scholar (324)

WorldCat (306)

Understanding journalism impact: A multi-dimensional taxonomy for professional, organizational and societal change

Anya Schiffrin, André Corrêa d’Almeida, Lindsay Green-Barber, Adelina Yankova, Dylan W. Groves

Journal of Applied Journalism & Media Studies, volume 14, issue 2 (2025), pp. 255 - 280

"How should we measure the impact of investigative journalism? Media scholars and practitioners have turned their attention towards understanding the causal effect of media reports on a range of social, political and economic outcomes. Their interest has been spurred by the increased availability of ... more

Google Scholar (309)

WorldCat (257)

Program Evaluation Theory and Practice: A Comprehensive Guide

Donna M. Mertens, Jori N. Hall, Amy T. Wilson

New York; London: Guilford Press, 3rd ed. (2025), xxii, 631 pp.

"Covering both the theory and practice of evaluation in one engaging volume, this leading text is now in a revised third edition. It features additional evaluation approaches, such as the Indigenous paradigm; new examples, ranging from small to large and multi-site evaluations; and more. Instructors ... more

Google Scholar (305)

WorldCat (272)

The Evaluation Handbook: An Evaluator's Companion
Deep Insights

Debra J. Rog, Leonard Bickman (eds.)

New York: Guilford Press (2025), xvii, 606 pp.

"This pragmatic evaluator's companion from leaders in the field provides essential guidance for designing, implementing, and managing exemplary evaluations using established and emerging methods, designs, and analyses. With contributions from skilled experts, the Handbook has a unique focus on the c ... more

Download (99)

Evolution of Program Evaluation: A Historical Analysis of Leading Theorists’ Views and Influences

Metin Kus

Education Quarterly Reviews, volume 8, issue 1 (2025), pp. 142-155

"Program evaluation has undergone significant evolution, shaped by diverse theoretical perspectives and influential scholars. This study provides a historical analysis of leading theorists’ views and their impact on the field, tracing key developments from early accountability-focused models to co ... more

Google Scholar (301)

WorldCat (321)

Evaluation Essentials: Methods for Conducting Sound Research

Beth Osborne DaPonte

San Francisco: Jossey-Bass, 2nd ed. (2025), x, 198 pp.

"This nuts-and-bolts introduction to program evaluation is also applicable to evaluation of policies and evaluation of organizational performance. It is rich with over 40 examples from public policy, public health, non-profit management, social work, arts management, education, international assista ... more

Google Scholar (319)

WorldCat (306)

Evaluation Essentials from A to Z

Marvin C. Alkin, Anne Vo, Christina A. Christie

New York; London: Guilford Press, 3rd ed. (2025), xiii, 330 pp.

"loved for its conversational style and reliable advice, this text is now in a revised and updated third edition, reflecting key developments in evaluation. It includes expanded coverage of equity and social justice issues, values and cost analysis, visualizing qualitative data with software, and mo ... more

Download (94)

Artificial Intelligence and Evaluation: Emerging Technologies and Their Implications for Evaluation

Steffen Bohni Nielsen, Francesco Mazzeo Rinaldi, Gustav Jakob Petersson (eds.)

Oxford: Routledge (2025), 330 pp.

"Artificial Intelligence and Evaluation: Emerging Technologies and Their Implications for Evaluation is a groundbreaking exploration of how the landscape of program evaluation will be redefined by artificial intelligence and other emerging digital technologies. In an era where digital technologies a ... more

Download (110)

Evaluating anti-corruption interventions: The state of practice

Tom Shipley

Bergen: Chr. Michelsen Institute (CMI), U4 Anti-Corruption Resource Centre (2024), 73 pp.

Download (112)

Evaluation Terms of Reference

Washington, DC: Adaptation Fund, Technical Evaluation Reference Group (AF-TERG) (2024), 22 pp.

"This guidance note covers Terms of Reference (ToR) guidance for all levels and types of evaluation outlined in the Adaptation Fund’s Evaluation Policy, including baseline studies and midterm reviews. While the emphasis is on developing ToRs for evaluation purposes, the ToR guidance can apply to o ... more

External Link (351)

Google Scholar (330)

WorldCat (321)

From Statistics to Stories: Indices and Indicators as Communication Tools for Social Change

Lauren Kogen

International Journal of Press/Politics, volume 29, issue 4 (2024), pp. 1090-1108

"The terms ‘indices’ and ‘indicators’ may immediately cause eyelids to droop. How, then, might they serve to impassion publics and, ultimately, promote social change? This paper examines the extent to which indices and indicators can be considered communication tools for social movements and ... more

External Link (283)

Expanding Our Understanding of Evidence for Meaningful Participation

Trimita Chakma, Georgia Booth, Ruby Johnson

Amsterdam: Porticus; Closer Than You Think Collective (2024), 60 pp.

"We developed the Evidence Quality Rubric, a tool that balances conventional research standards with inclusion factors, such as representation, power analysis, and bias interrogation, which can be applied to assess the quality of existing evidence. The Evidence Quality Rubric, applied to 20 case stu ... dies, reveals that solely using conventional standards of triangulation, methodology, and validation provides an incomplete assessment of evidence quality in participatory practices. These conventional standards exhibit a wide range of quality in the case studies, from limited to excellent. However, relying only on conventional standards neglects elements captured by the inclusion standards, such as representation, power dynamics, and bias. These are essential for evaluating participatory approaches involving excluded communities. For instance, a blog post by a refugee leader (case study #6) scored low on conventional standards due to a lack of formal methodology, but excelled on inclusion by directly amplifying refugee voices and examining power dynamics. Conversely, even research considered strong by conventional standards, such as a monitoring report on a migration project (case study #13), had limitations in representing stakeholder perspectives and analysing power when assessed through an inclusion lens. Notably, some studies demonstrated high quality across both sets of standards. A meta-synthesis of evaluations of refugee-led organisations (case study #10) and an evaluation of a youth participatory grantmaking pilot (case study #19) employed rigorous methodologies while also centring marginalised voices and critically examining power and bias. Both examples showcase that the inclusion standards contribute to a more comprehensive, nuanced, and context-sensitive evaluation of the evidence alongside conventional standards. This expanded assessment framework is essential for recognising the unique value and challenges of participatory approaches and ensuring that the experiences and perspectives of excluded communities are adequately considered in the evaluation of evidence quality. While the Evidence Quality Rubric has highlighted the strengths and weaknesses of various case studies in terms of conventional and inclusion standards, it is clear that a more structured approach is necessary to generate inclusive evidence that also meets the basic conventional standards. In response, we propose the Inclusive Evidence Guidelines to provide practical guidance for participatory evidence gathering that reflects the priorities of affected communities. These guidelines emphasise principles such as centring participant perspectives, co-designing locally relevant methods, conducting systematic analysis, applying intersectional lenses, and ensuring accountability to communities. Overall, we emphasise the importance of documenting participation not just to meet donor demands but to facilitate learning and promote a more holistic understanding of the effectiveness of participation in contributing to our missions. The aim is to enhance transparency, fairness, and effectiveness in philanthropic practices through documented evidence." (Executive summary, page 8) more

Download (112)

Inception Report

Washington, DC: Adaptation Fund, Technical Evaluation Reference Group (AF-TERG) (2024), 17 pp.

"The remainder of this guidance note is structured by five intuitive questions answering what inception reports are, what their benefits are, and when and how should they be written. Annex 1 provides an outline and checklist of the recommended sections for Fund Evaluation Inception Reports, and Anne ... more

Google Scholar (324)

WorldCat (373)

Are We Making a Difference? Global and Local Efforts to Assess Peacebuilding Effectiveness
Deep Insights

Stacey L. Connaughton, Jasmine R. Linabary (eds.)

Lanham et al.: Rowman & Littlefield (2024), xxvii, 320 pp.

"Sections of the book engage in critical reflection on what peacebuilding effectiveness is and who gets to decide, provide practical examples and case studies of the successes and failures of assessing peacebuilding work, and support innovative strategies and tools to move the field forward. Chapter ... more

Download (99)

Evaluation Reporting

Washington, DC: Adaptation Fund, Technical Evaluation Reference Group (AF-TERG) (2024), 21 pp.

"The purpose of this guidance note is to support the preparation of evaluation reports, which is key in conducting reliable, useful and ethical evaluations, in accordance with the Adaptation Fund’s Evaluation Policy. It provides a foundation for reporting for all levels and types of evaluations ou ... more

Download (95)

Real-Time Evaluations

Washington, DC: Adaptation Fund, Technical Evaluation Reference Group (AF-TERG) (2024), 21 pp.

"The purpose of this guidance note is to support the planning and implementation of fit-for purpose project real-time evaluations (RTEs) in accordance with the Adaptation Fund’s Evaluation Policy. The intended audience for this guidance note is people who plan and manage Fund evaluation activities ... more

Download (98)

Commissioning and Managing an Evaluation

Washington, DC: Adaptation Fund, Technical Evaluation Reference Group (AF-TERG) (2024), 37 pp.

"Commissioning an evaluation and managing an evaluation are different but interrelated processes. Commissioning an evaluation encompasses the activities to initiate a Fund evaluation, largely focused on developing the evaluation's Terms of Reference (ToR) and recruiting and contracting the evaluator ... more

External Link (259)

A guide to evaluation of value for money in UK public services: Why cost-benefit analysis alone may be insufficient to evaluate VFM, and how to navigate a solution

Julian King, Alex Hurrell

London: Verian Group (2024), 30 pp.

The Evaluation Handbook: An Evaluator's Companion Deep Insights

Are We Making a Difference? Global and Local Efforts to Assess Peacebuilding Effectiveness Deep Insights

The Evaluation Handbook: An Evaluator's Companion
Deep Insights

Are We Making a Difference? Global and Local Efforts to Assess Peacebuilding Effectiveness
Deep Insights