Toolbox Evaluation & Impact Assessment

Evaluation & Impact Assessment

TOOLBOX

Filter

139

Sort by:

Direction:

External Link (19)

Impact Framework for Media Projects. Toolkit

International Media Support (IMS);Local Media for Democracy Project;European Federation of Journalists (EFJ);Journalismfund Europe;Centre of Media Pluralism (2024), 13 pp.

"The impact framework is a set of tools and guidelines that helps to define targets for, track, and assess the impact of activities under the supported media projects in a clear and organised way; it can also be used for other media projects. Its methodology embraces a user-centric approach, links p ... more

External Link (17)

Google Scholar (13)

WorldCat (16)

Understanding journalism impact: A multi-dimensional taxonomy for professional, organizational and societal change

Anya Schiffrin, André Corrêa d’Almeida, Lindsay Green-Barber, Adelina Yankova, Dylan W. Groves

Journal of Applied Journalism & Media Studies (2023), [no pag.]

"How should we measure the impact of investigative journalism? Media scholars and practitioners have turned their attention towards understanding the causal effect of media reports on a range of social, political and economic outcomes. Their interest has been spurred by the increased availability of ... more

External Link (18)

IndiKit: Guidance on Smart Indicators for Relief and Development Projects
Key Guidance

People in Need (2023)

"IndiKit was developed by the Czech organization People in Need (PIN). It aims to make monitoring and evaluation (M&E) of relief and development interventions easier and better - by helping humanitarian and development workers to use well-formulated project indicators and to correctly collect and an ... more

Download (1)

A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Practices
Key Guidance

USAID (2021), v, 79 pp.

"The purpose of this review is to support education practitioners, host country government representatives, donors, implementers, non-governmental organizations (NGOs), civil society organizations, and other stakeholders in applying best practices to monitor and evaluate distance learning initiative ... more

Download (1)

Independent Media and Free Flow of Information: An Evidence Gap Map Protocol

Miriam Berretta, Charlotte Lane, Kerstin Tomiak, Katherine Garcia, Ingunn Storghaug, Jane Hammaker, Douglas Glandon, John Eyers

New Delhi: International Initiative for Impact Evaluation (3ie) (2021), 58 pp.

"This report presents the protocol for a systematic search to identify and map the evidence base of impact evaluations and systematic reviews of interventions that aim to promote an independent media as a democratic institution in low- and middle-income countries. The EGM was developed by 3ie, made ... more

Download (2)

Media for Democracy Assessment Tool: Learning, Evaluation, and Research (LER) Activity
Key Guidance

Susan Abbott, Maureen Taylor, Michelle Foster, Joan Barata Mir

Washington, DC; Landover, MD: USAID;Cloudburst Group (2021), 93 pp.

"The purpose of the Media for Democracy Assessment Tool (MAT) is to assist United States Agency for International Development (USAID) in producing media assessments to inform strategy on media and democracy programming, help inform potential media development programming goals, and help provide an i ... more

Download (1)

Communication for Development: An Evaluation Framework in Action
Top Insights

Jessica Noske-Turner (ed.)

Rugby: Practical Action Publishing (2020), xiii, 124 pp.

"This book presents five cases that reflect on the experiences of using practices consistent with the 'Evaluating C4D framework' [published by June Lennie and Jo Tacchi in 2013]. Case studies are important to help move from a set of ideal principles to an understanding of how the framework may be op ... more

Download (1)

Decoding Media Impact: Insights, Advice and Recommendations

Katie Donnelly

Philadelphia, Penns.: Media Impact Funders (2020), 31 pp.

"This guide is meant to serve as a practical resource for funders who want to understand where to start. Informed by feedback from our network, it represents a synthesis of the past seven years of work we’ve done in the impact space, and includes examples of successful media impact evaluation, too ... more

Download (2)

A Valuing Impact Toolkit for ICT Investment

NEF Consulting;Digital Impact Alliance (dial) (2020), 124 pp.

"ICT4D projects are known for having social, economic, and environmental impacts that are difficult to capture. Investments in digital infrastructure and innovation often have high up-front costs with largely medium-to-long-term benefits, not quick, short-term outcomes desired by policy makers. The ... more

Download (2)

How to Perform Impact Assessments: Key Steps for Assessing Communication Interventions

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 29 pp.

"This report brings together disparate terminology, findings and recommendations from the private and public sectors and academia to synthesise a set of five general steps for practitioners when performing impact assessments. Impact assessment (IA) compares a variable of interest after an interventi ... on (e.g. a communication campaign) to what it would it be if that intervention had never happened. Accompanying each of the five steps are examples of how migration communicators can assess the impact of campaigns to, first, change attitudes and, second, change behaviour. Step 1: Set objectives for the intervention that define what the desired effect is. Ideally, this should (1) meet the SMART criteria of specific, measurable, attainable, relevant and timebound; (2) focus on ultimate outcomes, rather than only outputs; (3) avoid a number of common mistakes in setting objectives and (4) carefully consider what the—as specific as possible— target audience is. Step 2: Identify a measure of the impact(s) that the intervention has as its objective. These should be (1) clearly defined and unambiguous; (2) externally valid (they act reliably if used for other interventions) and (3) internally valid (they measure what they claim to measure). There already exist long-lists of measures (also known as indicators) to choose from. Step 3: Estimate what would have happened had there been no intervention, known as a ‘counterfactual’. Ideally, but not necessarily, this requires performing a pre-intervention measurement, also known as baseline assessment. However, this is not always possible. This should not dissuade practitioners but instead assumptions of any IA, as discussed below, should be made openly. A long list of methods for measurement, including sampling considerations, are provided. Step 4: Perform the intervention. This may involve separating a randomised sample into a treatment group(s)—that receives the intervention(s)—and a control group—that does not. If this is not feasible, other forms of counterfactual are possible that ‘construct’ a control group. This section also overviews: (1) types of communication campaigns; (2) the MINDSPACE checklist of behavioural influences for interventions; (3) key recommendations on persuasive interventions from ICMPD’s previous work; (4) recommendations on campaigns deterring irregular migration. Step 5: Post-intervention measurement and analysis: (1) those receiving the intervention should be measured post-intervention (ideally, as well as, either a genuine or constructed control group); (2) the impact must be calculated: usually the difference in the variable of interest pre- and post-intervention and/or between the treatment and control group; (3) theoretical consideration of why and what aspect of the campaign caused an impact; are the findings likely to be universalizable?; (4) creation (and, in some cases, incorporation) of recommendations based on combination of impact and theoretical considerations." (Executive summary) more

Download (2)

How to perform impact assessments: Key steps for assessing communication interventions [Arabic]

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 26 pp.

Download (1)

Comment réaliser des évaluations d’impact : étapes clés pour évaluer les actions de communication

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 30 pp.

"Ce rapport réunit une terminologie hétérogène, des conclusions et des recommandations émanant des secteurs privé et public et du monde universitaire pour définir un ensemble de cinq étapes générales à suivre par les professionnels lors de la réalisation d’une EI. L’EI compare une va ... riable étudiée après une action (par ex. une campagne de communication) à ce qu’elle serait si cette action n’avait jamais eu lieu. Accompagnant chacune des cinq étapes, nous présenterons des exemples de la façon dont les communicants en migration peuvent évaluer l’impact des campagnes pour, d’abord, modifier les attitudes et, ensuite, changer les comportements. Étape 1 : Fixer des objectifs pour l’action qui définissent l’effet souhaité. Dans l’idéal, ils devraient (1) correspondre aux critères SMART : spécifique, mesurable, atteignable, pertinent et temporellement défini; (2) se concentrer sur les résultats finaux, plutôt qu’uniquement sur les produits; (3) éviter certaines erreurs courantes dans la définition des objectifs et (4) déterminer de façon aussi spécifique que possible l’identité du public cible. Étape 2 : Identifier une mesure de l’impact ou des impacts de l’objectif de l’action. Ces impacts devraient être (1) clairement définis et non ambigus; (2) valides d’un point de vue externe (ils restent fiables s’ils sont utilisés pour d’autres actions) et (3) valides d’un point de vue interne (ils mesurent ce qu’ils prétendent mesurer). Il existe déjà de longues listes de mesures (également connues sous le nom d’indicateurs) parmi lesquelles il est possible de choisir. Étape 3 : Estimer ce qui se serait passé si l’action n’avait pas eu lieu, ou situation « contrefactuelle ». Idéalement, mais pas nécessairement, cela implique la réalisation de mesures en amont de l’action, également connues sous le nom d’évaluation de base. Néanmoins, ce n’est pas toujours possible. Cela ne devrait pas dissuader les professionnels, en revanche, les hypothèses relatives aux EI devraient faire l’objet d’une discussion ouverte, comme évoqué ci-dessous. Une longue liste de méthodes de mesure, y compris des considérations liées à l’échantillonnage, sont fournies. • Étape 4 : Réaliser l’action. Cela peut impliquer de séparer un échantillon aléatoire en un ou plusieurs groupes traités, qui font l’objet de l’action/des actions, et un groupe témoin, qui n’en fait pas l’objet. Si cela n’est pas faisable, d’autres formes de situations contrefactuelles sont possibles pour « construire » un groupe témoin. Cette section aborde également : (1) les types de campagnes de communication; (2) la check-list MINDSPACE d’influences comportementales pour les actions; (3) les recommandations clés portant sur les actions persuasives issues des travaux précédents de l’ICMPD; (4) les recommandations relatives aux campagnes de dissuasion de la migration irrégulière. • Étape 5 : Mesures et analyses en aval de l’action : (1) ceux qui ont bénéficié de l’action doivent faire l’objet de mesures en aval de l’action (idéalement dans le cadre un groupe témoin authentique ou construit); (2) l’impact doit être calculé : généralement la différence entre la variable étudiée en amont et en aval de l’action et/ou entre le groupe traité et témoin; (3) réflexion théorique sur les raisons et les aspects précis de la campagne qui ont causé un impact; les conclusions sont-elles transposables ?; (4) création (et, dans certains cas, incorporation) de recommandations basées sur la combinaison de l’impact et des réflexions théoriques." (Résumé) more

External Link (13)

"Big Changes Start Small": Stories of People Making a Difference. An Impact Report

Bonn: Deutsche Welle DW Akademie (2020), 97 pp.

"DW Akademie collects and documents the impacts of its projects through a range of methods. There are three approaches that are currently standard in development cooperation: Applied studies, which enable new insights on impacts, impact-oriented monitoring performed with local partners in the course ... more

External Link (15)

Countering Violent Extremism in Nigeria: Using a Text-Message Survey to Assess Radio Programs

James V. Marrone, Todd C. Helmus, Elizabeth Bodine-Baron, Christopher Santucci

RAND Corporation (2020), 32 pp.

"Using a randomized encouragement design, we assigned 2,064 people to listen to either 'Ina Mafita' or to a control program (professional soccer matches) each week over the course of two months. Recruitment and engagement were conducted remotely via short message service (short message system [SMS] ... more

Download (1)

El buen-convivir y bien-transformar en la comunicación: Sistematización de prácticas innovadoras para el Buen Vivir

Adalid Contreras

Quito: Signis América Latina y Caribe (2019), 204 pp.

"Hemos estructurado el documento en tres grandes partes. La primera es una lectura del Vivir Bien / Buen Vivir desde las prácticas de las experiencias sistematizadas. Es un esfuerzo por conceptualizar la Comunicación para el Vivir Bien / Buen Vivir desde las prácticas vividas. Siguiendo las narra ... more

Download (0)

The Impact Field Guide and Toolkit: From Art to Impact
Key Guidance

Sahar Driver

London et al.: Doc Society, 2nd ed. (2019), 381 pp.

"This guide is about more than just tools and resources, it’s about building community. As with the process of evaluation, we saw an opportunity to learn about how impact happens and to track how films are catalysing social change. That’s the impetus behind the case studies that make up our Impa ... more

Download (1)

Guía práctica y herramientas para el impacto: Del arte al impacto
Key Guidance

Sahar Driver

London et al.: Doc Society (2019), 237 pp.

"El objetivo de esta guía va más allá de presentar herramientas y recursos, pretende crear comunidad. Igual que sucede con el proceso de evaluación, vimos una oportunidad para aprender cómo se produce el impacto y hacer un seguimiento de cómo las películas están catalizando el cambio social. ... more

Download (2)

O guia de campo: Kit der ferramentas para o impacto. Da arte ao impacto
Key Guidance

Sahar Driver

London et al.: Doc Society (2019), 238 pp.

"Nós oferecemos ideias e abordagens para que você não tenha que lidar com um obstáculo que alguém já ultrapassou e, com isso, concentrar-se nos desafios próprios de seu projeto. Disponibilizamos todas as nossas colaborações, conversas, acordos e desentendimentos com equipes de filmagem incr ... more

Download (0)

The impact field guide and toolkit: From art to impact [in Arabic]
Key Guidance

Sahar Driver

London et al.: Doc Society, 2nd ed. (2019), 238 pp.

External Link (14)

Google Scholar (13)

WorldCat (11)

Evaluations and Impact Assessments in Communication for Development

Lauren Kogen

In: Handbook of Communication for Development and Social Change

Jan Servaes (ed.)

Singapore: Springer (2019), 17 pp.

"Within the development field, project evaluations and impact assessments are essential. Donors are increasingly requiring rigorous evaluations in order to (1) ensure that aid dollars are spent on projects that are having positive impacts and not being wasted on projects that are ineffective and (2) ... more

IndiKit: Guidance on Smart Indicators for Relief and Development Projects Key Guidance

A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Practices Key Guidance

Media for Democracy Assessment Tool: Learning, Evaluation, and Research (LER) Activity Key Guidance

Communication for Development: An Evaluation Framework in Action Top Insights

The Impact Field Guide and Toolkit: From Art to Impact Key Guidance

Guía práctica y herramientas para el impacto: Del arte al impacto Key Guidance

O guia de campo: Kit der ferramentas para o impacto. Da arte ao impacto Key Guidance

The impact field guide and toolkit: From art to impact [in Arabic] Key Guidance

IndiKit: Guidance on Smart Indicators for Relief and Development Projects
Key Guidance

A Roadmap for Measuring Distance Learning: A Review of Evidence and Emerging Practices
Key Guidance

Media for Democracy Assessment Tool: Learning, Evaluation, and Research (LER) Activity
Key Guidance

Communication for Development: An Evaluation Framework in Action
Top Insights

The Impact Field Guide and Toolkit: From Art to Impact
Key Guidance

Guía práctica y herramientas para el impacto: Del arte al impacto
Key Guidance

O guia de campo: Kit der ferramentas para o impacto. Da arte ao impacto
Key Guidance

The impact field guide and toolkit: From art to impact [in Arabic]
Key Guidance