Document details

Using Big Data for Evaluating Development Outcomes: A Systematic Map

Centre of Excellence for Development Impact and Learning (CEDIL) (2020), vi, 104 pp.

Contains 15 figures, 13 tables, bibliogr. pp. 73-104

Series: CEDIL Methods Working Paper, 2

CC BY

"Spatially and temporally relevant ‘big data’ that does not require data collection in the field has the potential to provide insights into people’s economic, social, behavioural and political lives, and hence could be used in measuring key development outcomes. Big data consists of humangenerated data including online searches, social media, citizen reporting or crowdsourced data, process-mediated data such as mobile phone call record details (CRD), commercial transactions data and machine-generated data from satellites, sensors or drones. The primary value of big data is that it is possible to measure outcomes that could not previously be measured using household surveys at the required temporal and spatial scale. The potential of big data to answer causal attribution, however, is still not widely understood, especially in low- and middle-income countries (L&MICs). The report is based on a map of the studies using big data and its objective is to discuss methodological, ethical and practical constraints relating to the use of big data. The systematic map includes impact evaluations (IEs) that use big data to evaluate development outcomes, systematic reviews (SRs) of big data IEs and other measurement studies that innovatively use big data to measure and validate any development outcomes. This study also explores the sectoral and geographical spread of big data's use in international development. This map includes studies written in English and published between 2005 and 2019, regardless of the target country's income level or population's status. We provide detailed breakdowns on the map for different country income classifications, fragile contexts and population characteristics. From the initial list of 17,393 studies we arrived at a final list of 437 studies, which included 48 IEs, 381 measurement studies and 8 SRs." (Executive summary)
1 Introduction, 3
2 Methodology, 7
3 Results, 15
4 Key findings, discussions and lessons, 31
5 Using big data in IEs: potentials and challenges, 36
6 Limitations of this study and potential next steps, 41
7 Conclusions, 43
8 References, 44
Appendix 1 The OECD definition of fragile states, 48
Appendix 2 Details of sub-maps, 51
Appendix 3 Search strategy and the databases searched, 53
Appendix 4 Data extraction tool, 58
Appendix 5 ML and manual screening, 64
Appendix 6 SR appraisal tool and summary, 65
Appendix 7 Additional tables and figures, 68
Appendix 8 Systematic map of big data sources and outcomes, 72
Appendix 9 List of included studies, 73