Toolbox Campaigning, Advocacy, PR

Sort by:

Direction:

Empowerment Evaluation and Social Justice: Confronting the Culture of Silence

David M. Fetterman

New York; London: Guilford Press (2023), xiv, 171 pp.

"From the founder of empowerment evaluation (EE), a framework uniquely suited to advancing social justice causes, this book explains the theories, principles, and steps of conducting EE from scratch or within a preexisting evaluation or work plan. David M. Fetterman describes how EE enables program ... more

External Link (11)

Meta-Synthesis of UN Women Evaluations on Advocacy and Communications

Ekaterina Dorodnykh, Soo Yeon Kim, Aileen Allen

New York; Independent Evaluation Service (IES): UN Women (2023), ix, 32 pp.

"This meta-synthesis collates evaluation insights and evidence on the advocacy and communications efforts related to the key results areas of UN Women’s Strategic Plan: Governance and Participation in Public Life; Women’s Economic Empowerment; Ending Violence Against Women; and Women, Peace and ... more

Download (1)

Using Markers for Digital Engagement and Social Change: Tracking Meaningful Narrative Exchange in Transmedia Edutainment with Text Analytics Techniques

Roel O. Lutkenhaus, Hua Wang, Arvind Singhal, Jeroen Jansz, Martine P. A. Bouman

Digital Health, volume 8 (2022), pp. 1-18

"While social media offer an unprecedented opportunity for orchestrating large-scale communication campaigns, it is often difficult to track audience responses on various digital platforms over time and to ascertain if their engagement is aligned with the original intention. In this article, we shar ... more

External Link (16)

Assessing the Effectiveness of Online Facebook Campaigns Targeting Potential Irregular Migrants: A Pilot Study in Three West African Countries

Esther Haarman, Jasper Tjaden, Gustavo López

Geneva: International Organization for Migration (IOM) (2020), 41 pp.

"Studies have shown that many irregular migrants embark on journeys without accurate or complete information about migration. In response, IOM and other organizations run information campaigns intended to inform potential migrants of the dangers associated with irregular migration and to facilitate ... informed decisions. The use of online and social media platforms for these campaigns has become increasingly popular in recent years, due to their potential for quickly reaching millions of people at low cost. While the evidence base for offline information campaigns has been gradually improving, it remains unclear how online communication campaigns affect potential migrants, and methodologies to do this remain underdeveloped. This report therefore presents results from a pilot impact assessment of an online campaign based on Facebook posts in Guinea, Nigeria and Senegal during September 2019 and February 2020 [.] Conclusion: Facebook ad campaigns can reach many people quickly. However, large audience sizes and relatively cheap costs come with a trade-off. The impact of an information campaign on potential migrants’ perceptions, attitudes, knowledge and behaviour remains largely unknown because many Facebook users that engage with the campaign do not fit the intended target group. The advantage of offline over online campaigns is that implementers have more control and information about who attends activities or who is exposed to campaign content. The engagement of users with campaign posts varies largely by audience and post characteristics. Campaign and social media teams are well advised to conduct pilot tests before scaling up activities. Pilot tests can inform a customized approach for each online activity on Facebook, and thereby maximize impact. In addition to A/B tests, lift tests may offer a useful opportunity to test the impact of Facebook posts. Compared to impact assessments of offline information campaigns, measuring the impact of Facebook ads on potential migrants presents several unique challenges, including: Many potential migrants are not on Facebook or do not use ad content to seek information about migration. Internet connectivity is also limited in many West African countries. While Facebook allows users to narrow down target groups, it is not clear whether those Facebook users that are exposed to the content fit the profile of potential migrants. Whether the intended target group is reached can be affirmed only through surveys, which have a low response rate; It is not possible to interview the same Facebook users several times to track changes in their perceptions. Engagement metrics like post reactions, link clicks or video plays can be useful indicators. However, these low-engagement metrics do not offer clear evidence of the short- or long-term effects on Facebook users." (Executive summary) more

External Link (11)

Monitoring Advocacy Activities

Nigel Simister, Jenny Ross

INTRAC (2020), 5 pp.

"Most advocacy work is carried out in complex environments where there are many different influences on change. This has important consequences for how objectives and indicators are developed. CSOs need to understand the immediate and short-term changes resulting from different advocacy activities i ... more

External Link (13)

M&E of Advocacy

Jenny Ross, Nigel Simister

INTRAC (2020), 6 pp.

"Monitoring and evaluating advocacy work involves unique challenges. These can usually be overcome provided expectations are realistic. Ongoing monitoring and evaluation carried out during advocacy interventions should enhance and support real-time decision-making. Advocacy evaluations should includ ... more

Download (2)

How to Perform Impact Assessments: Key Steps for Assessing Communication Interventions

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 29 pp.

"This report brings together disparate terminology, findings and recommendations from the private and public sectors and academia to synthesise a set of five general steps for practitioners when performing impact assessments. Impact assessment (IA) compares a variable of interest after an interventi ... on (e.g. a communication campaign) to what it would it be if that intervention had never happened. Accompanying each of the five steps are examples of how migration communicators can assess the impact of campaigns to, first, change attitudes and, second, change behaviour. Step 1: Set objectives for the intervention that define what the desired effect is. Ideally, this should (1) meet the SMART criteria of specific, measurable, attainable, relevant and timebound; (2) focus on ultimate outcomes, rather than only outputs; (3) avoid a number of common mistakes in setting objectives and (4) carefully consider what the—as specific as possible— target audience is. Step 2: Identify a measure of the impact(s) that the intervention has as its objective. These should be (1) clearly defined and unambiguous; (2) externally valid (they act reliably if used for other interventions) and (3) internally valid (they measure what they claim to measure). There already exist long-lists of measures (also known as indicators) to choose from. Step 3: Estimate what would have happened had there been no intervention, known as a ‘counterfactual’. Ideally, but not necessarily, this requires performing a pre-intervention measurement, also known as baseline assessment. However, this is not always possible. This should not dissuade practitioners but instead assumptions of any IA, as discussed below, should be made openly. A long list of methods for measurement, including sampling considerations, are provided. Step 4: Perform the intervention. This may involve separating a randomised sample into a treatment group(s)—that receives the intervention(s)—and a control group—that does not. If this is not feasible, other forms of counterfactual are possible that ‘construct’ a control group. This section also overviews: (1) types of communication campaigns; (2) the MINDSPACE checklist of behavioural influences for interventions; (3) key recommendations on persuasive interventions from ICMPD’s previous work; (4) recommendations on campaigns deterring irregular migration. Step 5: Post-intervention measurement and analysis: (1) those receiving the intervention should be measured post-intervention (ideally, as well as, either a genuine or constructed control group); (2) the impact must be calculated: usually the difference in the variable of interest pre- and post-intervention and/or between the treatment and control group; (3) theoretical consideration of why and what aspect of the campaign caused an impact; are the findings likely to be universalizable?; (4) creation (and, in some cases, incorporation) of recommendations based on combination of impact and theoretical considerations." (Executive summary) more

Download (2)

How to perform impact assessments: Key steps for assessing communication interventions [Arabic]

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 26 pp.

Download (1)

Comment réaliser des évaluations d’impact : étapes clés pour évaluer les actions de communication

James Dennison

International Centre for Migration Policy Development (ICMPD);EUROMED Migration (2020), 30 pp.

"Ce rapport réunit une terminologie hétérogène, des conclusions et des recommandations émanant des secteurs privé et public et du monde universitaire pour définir un ensemble de cinq étapes générales à suivre par les professionnels lors de la réalisation d’une EI. L’EI compare une va ... riable étudiée après une action (par ex. une campagne de communication) à ce qu’elle serait si cette action n’avait jamais eu lieu. Accompagnant chacune des cinq étapes, nous présenterons des exemples de la façon dont les communicants en migration peuvent évaluer l’impact des campagnes pour, d’abord, modifier les attitudes et, ensuite, changer les comportements. Étape 1 : Fixer des objectifs pour l’action qui définissent l’effet souhaité. Dans l’idéal, ils devraient (1) correspondre aux critères SMART : spécifique, mesurable, atteignable, pertinent et temporellement défini; (2) se concentrer sur les résultats finaux, plutôt qu’uniquement sur les produits; (3) éviter certaines erreurs courantes dans la définition des objectifs et (4) déterminer de façon aussi spécifique que possible l’identité du public cible. Étape 2 : Identifier une mesure de l’impact ou des impacts de l’objectif de l’action. Ces impacts devraient être (1) clairement définis et non ambigus; (2) valides d’un point de vue externe (ils restent fiables s’ils sont utilisés pour d’autres actions) et (3) valides d’un point de vue interne (ils mesurent ce qu’ils prétendent mesurer). Il existe déjà de longues listes de mesures (également connues sous le nom d’indicateurs) parmi lesquelles il est possible de choisir. Étape 3 : Estimer ce qui se serait passé si l’action n’avait pas eu lieu, ou situation « contrefactuelle ». Idéalement, mais pas nécessairement, cela implique la réalisation de mesures en amont de l’action, également connues sous le nom d’évaluation de base. Néanmoins, ce n’est pas toujours possible. Cela ne devrait pas dissuader les professionnels, en revanche, les hypothèses relatives aux EI devraient faire l’objet d’une discussion ouverte, comme évoqué ci-dessous. Une longue liste de méthodes de mesure, y compris des considérations liées à l’échantillonnage, sont fournies. • Étape 4 : Réaliser l’action. Cela peut impliquer de séparer un échantillon aléatoire en un ou plusieurs groupes traités, qui font l’objet de l’action/des actions, et un groupe témoin, qui n’en fait pas l’objet. Si cela n’est pas faisable, d’autres formes de situations contrefactuelles sont possibles pour « construire » un groupe témoin. Cette section aborde également : (1) les types de campagnes de communication; (2) la check-list MINDSPACE d’influences comportementales pour les actions; (3) les recommandations clés portant sur les actions persuasives issues des travaux précédents de l’ICMPD; (4) les recommandations relatives aux campagnes de dissuasion de la migration irrégulière. • Étape 5 : Mesures et analyses en aval de l’action : (1) ceux qui ont bénéficié de l’action doivent faire l’objet de mesures en aval de l’action (idéalement dans le cadre un groupe témoin authentique ou construit); (2) l’impact doit être calculé : généralement la différence entre la variable étudiée en amont et en aval de l’action et/ou entre le groupe traité et témoin; (3) réflexion théorique sur les raisons et les aspects précis de la campagne qui ont causé un impact; les conclusions sont-elles transposables ?; (4) création (et, dans certains cas, incorporation) de recommandations basées sur la combinaison de l’impact et des réflexions théoriques." (Résumé) more

External Link (13)

M&E Tools for Advocacy Work

Nigel Simister, Jenny Ross

INTRAC (2020), 7 pp.

"Many different tools and methodologies can be used to support advocacy work. Some are designed specifically for monitoring and evaluation (M&E) purposes. Others are designed to support the planning and management of advocacy interventions, but can be used or adapted for M&E purposes. CSOs often emp ... more

External Link (11)

M&E Systems for Advocacy Work

Nigel Simister, Jenny Ross

INTRAC (2020), 6 pp.

"Monitoring and evaluation (M&E) systems for advocacy interventions usually include a number of common elements. The nature of advocacy work means M&E systems need to be designed and implemented differently than in conventional projects. M&E systems for advocacy work should be simple and time-effici ... more

External Link (13)

Evaluating Behaviour Change Communication Campaigns in Health and Safety: A Literature Review

T. J. Bailey, L. N. Wundersitz

Adelaide: University of Adelaide, Centre for Automotive Safety Research (2019), 30 pp.

"This literature review examined selected international research, published from 2010 to the present, to investigate how six fields other than road safety (i.e. healthy eating/physical activity, smoking, alcohol and drug use, workplace health and safety, suicide and juvenile offending) evaluated the ... more

Download (1)

Stop the Traffik's Organisational Evaluation: Executive Summary

Robin Brady, Ruth van Dyke

Crackington Haven, Bude: Stop the Traffik;Robin’s Trading Company (2019), 12 pp.

"Since its inception STOP THE TRAFFIK (STT) has been focused on creating a world where people are not bought and sold. To contribute towards this goal, STT has developed an intelligence-led prevention approach: informing its focus at a local, national and global level. STT believes that it delivers ... more

Download (2)

Unique Methods in Advocacy Evaluation

Julia Coffmann, Ehren Reed

California Endowment (2019), 9 pp.

"Like all evaluations, advocacy evaluations can draw on a familiar list of traditional data collection methods, such as surveys, interviews, focus groups, or polling. But because the advocacy process can be complex, fast-paced, and dynamic, which makes data collection challenging, and because advoca ... more

External Link (17)

Google Scholar (14)

WorldCat (14)

Evaluating Public Communication: Exploring New Models, Standards, and Best Practice

Jim Macnamara

London; New York: Routledge (2018), xix, 392 pp.

"Chapters feature: A review of 30 frameworks and models that inform processes for evaluation in advertising, public relations, health communication and promotion, government communication and other specialist fields including the latest recommendations of industry bodies, evaluation councils, and re ... more

Download (2)

Evaluating the impact of information campaigns in the field of migration: A systematic review of the evidence, and practical guidance

Jasper Tjaden, Sandra Morgenstern, Frank Laczko

Geneva: International Organization for Migration (IOM);Global Migration Data Analysis Centre (GMDAC) (2018), 50 pp.

"Information campaigns designed to raise awareness of the potential risks of (irregular) migration have attracted much attention and investment across the world in recent years. Studies have repeatedly shown that many migrants start their journeys with limited or biased information and end up in vul ... nerable situations. In response, information campaigns have increased in number and the type, format, messages and strategy of such campaigns have diversified. This report presents the results from a systematic literature review of evaluations of such information campaigns in the field of migration. The study reveals that the evidence base available for programming and policymaking in this area is strikingly limited. We find that the uptake in the use of information campaigns has far outpaced any rigorous assessment of the effects that different campaigns may have on their respective target groups. In the absence of reliable evidence, the debate on the potential of this policy tool often relies on largely anecdotal evidence. Better evidence can show how information campaigns can be designed to best achieve their intended effects given the particular circumstances. The current lack of evidence limits the impact of future campaigns. While rigorous assessment of campaign impact can be difficult and costly, better evidence is clearly needed – wherever feasible and appropriate. Based on an extensive, systematic literature review, 60 relevant evaluations of information campaigns that targeted potential migrants and traffickers, as well as communities at large, were identified from a pool of 3,600 records. Only 30 of the selected campaign evaluations had publicly available results; the rest were collected through expert referrals. Two studies were published in peer-reviewed journals. Among the campaigns featured in the 60 studies, the most popular communication tools were workshop-type activities and cable TV programmes/advertisements. Most campaigns focused on trafficking, followed by irregular migration and, more generally, smuggling. A common issue is the lack of a clearly defined campaign objective and/or target group. This hampers any rigorous evaluation of programme effects. Whenever an objective is defined, it is most often aimed at “awareness-raising” and “knowledge generation.” The majority of the campaign evaluations claimed that the campaign under study was “successful” in inducing a change in knowledge, perceptions, attitudes and – to a lesser degree – (intended) behaviour. However, most of the evaluations reviewed provided relatively little evidence of the full impact of information campaigns. While many of the evaluations reported the number and profiles of campaign recipients or beneficiaries, impact was not directly measured. (In the evaluation literature, impact is defined as a change in outcome that is directly attributable to the programme and not any other factor.) Most of the evaluations identified did not meet minimum standards for robust evidence on programme effects. The large majority of evaluations were based on based on cross-sectional surveys of small numbers of participants (N) sampled at convenience, limiting the generalizability of the results. Only a few large-N studies employed a control-group design or involved pre- and post-measurements. None employed a (quasi-)experimental method for causal inference (e.g. randomized controlled trial, which is considered the “gold standard” for measuring impact)." (Executive summary) more

External Link (15)

The Counter-Narrative Monitoring & Evaluation Handbook

Loius Reynolds, Henry Tuck

London: Institute for Strategic Dialogue (ISD) (2016), 55 pp.

"The aim of this handbook is to provide civil society campaigners with the knowledge and skills they need to undertake high quality counter-narrative campaign evaluations. It presents model frameworks and case studies of successful counter-narrative M&E, as well as a range of useful tools and resour ... more

Download (1)

Advocacy for Development: Effectiveness, Monitoring and Evaluation

Jennifer B. Barrett, Margit van Wessel, Dorothea Hilhorst

Wageningen University (2016), 94 pp.

"Monitoring and evaluation of advocacy for development is an emerging field. Many civil society organisations, donors and evaluators are now involved with advocacy. Questions of how to understand and assess programmes are urgent. This e-book seeks to contribute to practical capacity on this front on ... more

Download (2)

Monitoring and Evaluation of Policy Influence and Advocacy

London: Overseas Development Institute (ODI) (2014), iii, 70 pp.

"Policy influence and advocacy are increasingly regarded as a means of creating sustainable policy change in international development. It is often also seen as a difficult area to monitor and evaluate. Yet there is an increasingly rich strand of innovation in options to monitor, evaluate and learn ... more

Google Scholar (11)

WorldCat (12)

Evaluation Theory, Models, and Applications
Top Insights

Daniel L. Stufflebeam, Chris L. S. Coryn

San Francisco: Jossey-Bass, 2nd ed. (2014), xxxiii, 766 pp.

"The book begins with an overview of the evaluation field and program evaluation standards, and proceeds to cover the most widely used evaluation approaches. With new evaluation designs and the inclusion of the latest literature from the field, this second edition is an essential update for professi ... more

Evaluation Theory, Models, and Applications Top Insights

Evaluation Theory, Models, and Applications
Top Insights