Ten Common Flaws in Evaluations
Global Development Network (2023)
"Evaluations have two key functions: lesson learning and accountability. How well they can fill these tasks depends on the suitability of the evaluation design to address the evaluation questions of interest, and the quality of those evaluations. Unfortunately, many evaluations suffer from flaws which reduce the confidence we can have in their findings, and their usefulness for both lesson learning and accountability. This blog lists 10 common flaws that I commonly come across. Not all evaluations have these flaws. There are many excellent evaluations. But these flaws are sufficiently common to deserve drawing attention to." (Introduction)
1 Inadequate description of the intervention
2 ‘Evaluation reports’ which are monitoring not evaluation
3 Data collection is not a method
4 Unsubstantiated evidence claims
5 Insufficient evidence
6 Positive bias in process evaluation
7 Limited perspectives: Who do evaluators speak to?
8 Ignoring the role of others
9 Causal claims based on monitoring data
10 Global claims based on single studies
2 ‘Evaluation reports’ which are monitoring not evaluation
3 Data collection is not a method
4 Unsubstantiated evidence claims
5 Insufficient evidence
6 Positive bias in process evaluation
7 Limited perspectives: Who do evaluators speak to?
8 Ignoring the role of others
9 Causal claims based on monitoring data
10 Global claims based on single studies
