mediadev library

Assessing the Effectiveness of Online Facebook Campaigns Targeting Potential Irregular Migrants: A Pilot Study in Three West African Countries

Esther Haarman, Jasper Tjaden, Gustavo López

Geneva: International Organization for Migration (IOM) (2020), 41 pp.

"Studies have shown that many irregular migrants embark on journeys without accurate or complete information about migration. In response, IOM and other organizations run information campaigns intended to inform potential migrants of the dangers associated with irregular migration and to facilitate ... informed decisions. The use of online and social media platforms for these campaigns has become increasingly popular in recent years, due to their potential for quickly reaching millions of people at low cost. While the evidence base for offline information campaigns has been gradually improving, it remains unclear how online communication campaigns affect potential migrants, and methodologies to do this remain underdeveloped. This report therefore presents results from a pilot impact assessment of an online campaign based on Facebook posts in Guinea, Nigeria and Senegal during September 2019 and February 2020 [.] Conclusion: Facebook ad campaigns can reach many people quickly. However, large audience sizes and relatively cheap costs come with a trade-off. The impact of an information campaign on potential migrants’ perceptions, attitudes, knowledge and behaviour remains largely unknown because many Facebook users that engage with the campaign do not fit the intended target group. The advantage of offline over online campaigns is that implementers have more control and information about who attends activities or who is exposed to campaign content. The engagement of users with campaign posts varies largely by audience and post characteristics. Campaign and social media teams are well advised to conduct pilot tests before scaling up activities. Pilot tests can inform a customized approach for each online activity on Facebook, and thereby maximize impact. In addition to A/B tests, lift tests may offer a useful opportunity to test the impact of Facebook posts. Compared to impact assessments of offline information campaigns, measuring the impact of Facebook ads on potential migrants presents several unique challenges, including: Many potential migrants are not on Facebook or do not use ad content to seek information about migration. Internet connectivity is also limited in many West African countries. While Facebook allows users to narrow down target groups, it is not clear whether those Facebook users that are exposed to the content fit the profile of potential migrants. Whether the intended target group is reached can be affirmed only through surveys, which have a low response rate; It is not possible to interview the same Facebook users several times to track changes in their perceptions. Engagement metrics like post reactions, link clicks or video plays can be useful indicators. However, these low-engagement metrics do not offer clear evidence of the short- or long-term effects on Facebook users." (Executive summary) more

External Link (8)

The Impact of Mobile Cinema Events on Potential Migrants in Guinea: Impact Evaluation Report

Marie-Luce Bia-Zafinikamia, Jasper Tjaden, Horace Gninafon

Geneva: International Organization for Migration (IOM) (2020), 55 pp.

External Link (12)

Migrants as Messengers: The Impact of Peer-To-Peer Communication on Potential Migrants in Senegal. Impact Evaluation Report

Felipe Alexander Dunsch, Jasper Tjaden, William Quiviger

Geneva: International Organization for Migration (IOM) (2019), vi, 70 pp.

"The International Organization for Migration (IOM) has conducted a scientifically rigorous impact evaluation to assess the impact of the Migrants as Messengers (MaM) campaign in Dakar, Senegal. MaM was a peer-to-peer awareness-raising campaign made by migrants for migrants and implemented in Senega ... more

Download (2)

Evaluating the impact of information campaigns in the field of migration: A systematic review of the evidence, and practical guidance

Jasper Tjaden, Sandra Morgenstern, Frank Laczko

Geneva: International Organization for Migration (IOM);Global Migration Data Analysis Centre (GMDAC) (2018), 50 pp.

"Information campaigns designed to raise awareness of the potential risks of (irregular) migration have attracted much attention and investment across the world in recent years. Studies have repeatedly shown that many migrants start their journeys with limited or biased information and end up in vul ... nerable situations. In response, information campaigns have increased in number and the type, format, messages and strategy of such campaigns have diversified. This report presents the results from a systematic literature review of evaluations of such information campaigns in the field of migration. The study reveals that the evidence base available for programming and policymaking in this area is strikingly limited. We find that the uptake in the use of information campaigns has far outpaced any rigorous assessment of the effects that different campaigns may have on their respective target groups. In the absence of reliable evidence, the debate on the potential of this policy tool often relies on largely anecdotal evidence. Better evidence can show how information campaigns can be designed to best achieve their intended effects given the particular circumstances. The current lack of evidence limits the impact of future campaigns. While rigorous assessment of campaign impact can be difficult and costly, better evidence is clearly needed – wherever feasible and appropriate. Based on an extensive, systematic literature review, 60 relevant evaluations of information campaigns that targeted potential migrants and traffickers, as well as communities at large, were identified from a pool of 3,600 records. Only 30 of the selected campaign evaluations had publicly available results; the rest were collected through expert referrals. Two studies were published in peer-reviewed journals. Among the campaigns featured in the 60 studies, the most popular communication tools were workshop-type activities and cable TV programmes/advertisements. Most campaigns focused on trafficking, followed by irregular migration and, more generally, smuggling. A common issue is the lack of a clearly defined campaign objective and/or target group. This hampers any rigorous evaluation of programme effects. Whenever an objective is defined, it is most often aimed at “awareness-raising” and “knowledge generation.” The majority of the campaign evaluations claimed that the campaign under study was “successful” in inducing a change in knowledge, perceptions, attitudes and – to a lesser degree – (intended) behaviour. However, most of the evaluations reviewed provided relatively little evidence of the full impact of information campaigns. While many of the evaluations reported the number and profiles of campaign recipients or beneficiaries, impact was not directly measured. (In the evaluation literature, impact is defined as a change in outcome that is directly attributable to the programme and not any other factor.) Most of the evaluations identified did not meet minimum standards for robust evidence on programme effects. The large majority of evaluations were based on based on cross-sectional surveys of small numbers of participants (N) sampled at convenience, limiting the generalizability of the results. Only a few large-N studies employed a control-group design or involved pre- and post-measurements. None employed a (quasi-)experimental method for causal inference (e.g. randomized controlled trial, which is considered the “gold standard” for measuring impact)." (Executive summary) more