Evaluation of the BMAP-Forward Activity
Washington, DC: USAID; Integra Government Services International (2024), 165 pp.
"Activity Description: The Balkan Media Assistance Program to Foster Organizational Readiness While Advancing Resilient Development (BMAP-FORWARD) is a $13 million regional Activity (2022-2028) that aims to strengthen the independence and resilience of media outlets in Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, and Serbia.
Evaluation Purpose: USAID/Europe and Eurasia Bureau requested a mid-term performance evaluation of BMAP-FORWARD to understand which approaches, tools, and interventions are successfully contributing to planned outcomes according to the Activity’s three key objectives: 1) Increase strategic business practices of key media partners to operate as viable enterprises and better understand their markets and audiences; 2) Reinforce the ability of content producers to produce data-driven content, engage wider audiences, improve digital security efforts, and expand reach across the region; and 3) Foster stronger partnerships between media outlets and the technology, commercial, and entrepreneurial sectors to advance solutions on common challenges in the information space.
Evaluation Design: Mixed methods and non-experimental evaluation design approaches, including Expert Consultations and Outcome Harvesting, were employed to collect and analyze primary and secondary data, including reviewing 140 documents, conducting 51 key informant interviews with experts, 47 in-depth interviews with recipients of assistance, an online Expert Survey (n=63), and conducting 13 interviews and an online validation survey (n=48) to substantiate outcome stories. All data were triangulated to inform the evidence base to develop each finding and conclusion.
Key Findings: BMAP-FORWARD’s tailored and localized approach is a key strength of its ability to provide effective, useful, and timely assistance, which has strengthened Key Media Partners’ (KMPs’) shift towards strategic planning, ability to pursue diversified revenue sources, improve content production and technology use to engage wider audiences, produce data-driven content using new software, and engage in cross-border collaborations. BMAP-FORWARD assistance has strengthened KMPs’ digital security practices and protocols, although with room for improvement to sustain this support. While BMAPFORWARD has made inroads to foster partnerships between media and non-traditional actors, key factors inhibit progress towards this Activity Objective (Objective 3).
Key Recommendations: BMAP-FORWARD should: 1) continue to employ the highly tailored approach to learn KMPs’ needs, co-create solutions and action plans, identify resources to implement solutions, and present new ideas; 2) consider providing more opportunities for intentional engagement for peer-to-peer learning for newsroom staff from different outlets, including non-KMPs and different media across the region; 3) consider investing in strengthening in-house expertise capacity or identify long-term solutions for each KMP to navigate the evolving digital security challenges; and 4) consider designing and marketing sessions at forums geared towards facilitating potential partnerships between private sector actors and KMPs, which can be followed by ongoing assistance to support KMPs in navigating how to structure partnerships." (Abstract)
Evaluation Purpose: USAID/Europe and Eurasia Bureau requested a mid-term performance evaluation of BMAP-FORWARD to understand which approaches, tools, and interventions are successfully contributing to planned outcomes according to the Activity’s three key objectives: 1) Increase strategic business practices of key media partners to operate as viable enterprises and better understand their markets and audiences; 2) Reinforce the ability of content producers to produce data-driven content, engage wider audiences, improve digital security efforts, and expand reach across the region; and 3) Foster stronger partnerships between media outlets and the technology, commercial, and entrepreneurial sectors to advance solutions on common challenges in the information space.
Evaluation Design: Mixed methods and non-experimental evaluation design approaches, including Expert Consultations and Outcome Harvesting, were employed to collect and analyze primary and secondary data, including reviewing 140 documents, conducting 51 key informant interviews with experts, 47 in-depth interviews with recipients of assistance, an online Expert Survey (n=63), and conducting 13 interviews and an online validation survey (n=48) to substantiate outcome stories. All data were triangulated to inform the evidence base to develop each finding and conclusion.
Key Findings: BMAP-FORWARD’s tailored and localized approach is a key strength of its ability to provide effective, useful, and timely assistance, which has strengthened Key Media Partners’ (KMPs’) shift towards strategic planning, ability to pursue diversified revenue sources, improve content production and technology use to engage wider audiences, produce data-driven content using new software, and engage in cross-border collaborations. BMAP-FORWARD assistance has strengthened KMPs’ digital security practices and protocols, although with room for improvement to sustain this support. While BMAPFORWARD has made inroads to foster partnerships between media and non-traditional actors, key factors inhibit progress towards this Activity Objective (Objective 3).
Key Recommendations: BMAP-FORWARD should: 1) continue to employ the highly tailored approach to learn KMPs’ needs, co-create solutions and action plans, identify resources to implement solutions, and present new ideas; 2) consider providing more opportunities for intentional engagement for peer-to-peer learning for newsroom staff from different outlets, including non-KMPs and different media across the region; 3) consider investing in strengthening in-house expertise capacity or identify long-term solutions for each KMP to navigate the evolving digital security challenges; and 4) consider designing and marketing sessions at forums geared towards facilitating potential partnerships between private sector actors and KMPs, which can be followed by ongoing assistance to support KMPs in navigating how to structure partnerships." (Abstract)
1. INTRODUCTION, 13
Regional Context -- Program Description
2. EVALUATION PURPOSE AND QUESTIONS, 14
Evaluation Purpose -- Evaluation Questions
3. EVALUATION METHODOLOGY, 15
Evaluation Design Data Collection Methods -- Data Analysis
4. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS, 17
Cross-Cutting Findings -- Evaluation Question 1 -- Evaluation Question 2 -- Evaluation Question 3 -- Evaluation Question 4
Annex A: Evaluation Scope of Work, 47
Annex B: Activity Authorization Request, 49
Annex C: Evaluation Methods and Limitations, 59
Annex D: Evaluation Timeline, 67
Annex E: Evaluation Design Matrix, 78
Annex F: Data Collection Tools, 80
Annex G: Sources of Information, 99
Annex H: Data Collection Itinerary, 104
Annex I: Survey Results, 110
Annex J: Evaluation Team Members, 154
Annex K: Finding, Conclusions, and Recommendation Table, 158
Regional Context -- Program Description
2. EVALUATION PURPOSE AND QUESTIONS, 14
Evaluation Purpose -- Evaluation Questions
3. EVALUATION METHODOLOGY, 15
Evaluation Design Data Collection Methods -- Data Analysis
4. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS, 17
Cross-Cutting Findings -- Evaluation Question 1 -- Evaluation Question 2 -- Evaluation Question 3 -- Evaluation Question 4
Annex A: Evaluation Scope of Work, 47
Annex B: Activity Authorization Request, 49
Annex C: Evaluation Methods and Limitations, 59
Annex D: Evaluation Timeline, 67
Annex E: Evaluation Design Matrix, 78
Annex F: Data Collection Tools, 80
Annex G: Sources of Information, 99
Annex H: Data Collection Itinerary, 104
Annex I: Survey Results, 110
Annex J: Evaluation Team Members, 154
Annex K: Finding, Conclusions, and Recommendation Table, 158