"Evaluations have two key functions: lesson learning and accountability. How well they can fill these tasks depends on the suitability of the evaluation design to address the evaluation questions of interest, and the quality of those evaluations. Unfortunately, many evaluations suffer from flaws whi
...
ch reduce the confidence we can have in their findings, and their usefulness for both lesson learning and accountability. This blog lists 10 common flaws that I commonly come across. Not all evaluations have these flaws. There are many excellent evaluations. But these flaws are sufficiently common to deserve drawing attention to." (Introduction)
more
"[...] Equitable evaluation contends that conducting evaluation practices with an equity approach is more powerful, as evaluation is used as a tool for advancing equity. It emphasises that context, culture, history and beliefs shape the nature of evaluations, specifically in the diverse and often co
...
mplex African reality. Equitable evaluation can render power to the powerless, offer a voice to the silenced and give presence to those treated as invisible. Evidence from various sources shows that inequality is prevalent in the African continent, hence the need to focus on evaluative solutions that address the structural issues that contribute to the different forms of inequality, such as economic, political and social inequality. Despite a plethora of development interventions in the African continent, a large proportion of the population on the continent still lacks access to basic goods and services for survival. The effectiveness of developmental programmes in sub-Saharan Africa has been uncertain, to the extent that minimal inroads have been made in addressing key challenges such as poverty, inequality and, currently, the effects of climate change. The Centre for Learning on Evaluation and Results, Anglophone Africa, supported by the Ford Foundation, commissioned two volumes on equitable evaluation in the Global South. The book chapters explore the following:
1. It takes stock of what we know about inequality: What is inequality in the African context, and how does it affect the lives of the citizens of African countries?
2. What is equitable evaluation? How can the concept of equitable evaluation be adopted in evaluation practice?
3. What lessons can be learnt from evaluations of interventions that address inequality at various levels (sectoral, programmatic, project)?
4. What epistemological transformation in evaluation practice is needed to achieve an equitable society?
5. How have issues of inequality manifested within evaluation practice through organisations, institutions and international development?
This book is the first of two volumes of voices from the Global South on equitable evaluation. The predominant methodology utilised is qualitative in nature. This scholarly book aims to invigorate academic discussions surrounding development programmes, with the goal of generating insights that can be utilised by evaluation commissioners and decision-makers in development programmes. These insights will help in addressing inequality and promoting a more equitable society in Africa through improved evaluation processes. The target audience for this book is primarily academics engaged in the field of developmental programmes in sub-Saharan Africa. No part of this work was plagiarised or published elsewhere." (Research justification, page v)
more
"This glossary provides conceptual clarity on common terms used in results based management, monitoring and evaluation. This 2nd Edition includes new terms, as well as updates and improvements to the original Glossary published in 2002. It is intended to support thoughtful approaches to results-base
...
d management, monitoring and evaluation, which in turn support better progress on sustainable development through learning and accountability." (Page 1)
more
"Ziel dieses Buches ist es, kompakt und nutzerfreundlich Grundkenntnisse für die Planung, Durchführung und Anwendung von Evaluationen zu vermitteln. Hierfür greift es auf die inhaltlichen Konzepte und Unterlagen zahlreicher Kurse zur Evaluation im In- und Ausland zurück. Das Buch richtet sich an
...
Interessierte, die erst über wenige Kenntnisse im Bereich der Evaluation verfügen und sich weiterbilden möchten. Es dient sowohl Personen, die Evaluationen selbst durchführen als auch Entscheidungsträgern, die Evaluationen in Auftrag geben, aber auch denjenigen, die evaluiert werden. Alle Beteiligten an einer Evaluation sollten wissen, welche Charakteristika und Standards eine professionell und kompetent durchgeführte Evaluation auszeichnen. Die zweite Auflage wurde vollständig überarbeitet und um ein Kapitel zum Monitoring und einen Einblick in den politischen Kontext erweitert." (Verlagsbeschreibung)
more
"The Footprint Evaluation Initiative aims to ensure that all evaluations consider environmental sustainability, regardless of whether this is an explicit objective of the project, policy or program being evaluated. This report describes four ‘thought experiments’ undertaken as part of this proje
...
ct. The thought experiments explored whether it is relevant, feasible and useful to consider environmental sustainability in evaluation, how this might be done, what challenges and issues it raises, and what is needed to address these. This report aims to document and share what we learned during this process, provide concrete examples of how environmental sustainability might be considered in an evaluation, and share details of our thought experiment process that others might find helpful." (Summary)
more
"This timely and comprehensive textbook covers theoretical and practical issues related to evaluation of programs, particularly social programs and projects, with an emphasis on viewing evaluation topics through a social justice, diversity, and inclusive perspective. It provides students with an equ
...
al appreciation of both the art (for example engaging stakeholders, communicating across diverse settings, recruiting participants) and science (writing questions that matter, using credible designs, data analysis) of evaluation planning, implementation, and dissemination. This textbook will help students recognize that all populations are diverse, and that high quality evaluations need to pay attention to the diversity of all populations being served." (Publisher description)
more
"This guide is designed for anyone who wants to learn how communication can more effectively support evaluation: evaluation consultants, communication consultants, evaluation commissioners and programme/project staff participating in evaluations. The guide is structured around three broad phases of
...
the evaluation process: before, during and after. The guide aims to provide advice and practical hints on how to use communication for each of these phases. A communication plan of action template adapted for evaluation is provided at annex 1. Examples of innovative communication tools for evaluation are found at annex 2. A case study is included at annex 3 that can be used for learning purposes. Links to extra resources are found at annex 4. The guide explains how to use communication to support evaluation and readers can adapt and use these approaches depending upon the stage of the evaluation process, their own context and available resources." (Page 4)
more
"A review of eight projects with end-of-project and post project evaluation reports suggest that post project evaluations can contribute to better understanding of the sustainability of project impacts and identify unexpected and emerging outcomes years after project close. The review highlights som
...
e of the methodological issues that are unique to post project (as opposed to end of project) evaluation, and point to the advantages of planning for sustainability measurement from the outset of the project. In the majority of cases, the sampling approach adopted at the endline proved challenging to replicate post project because it was not designed to measure sustainability, or because of access to or demographic changes in the beneficiary population. Finding suitable projects for this review was difficult because so few post project evaluations are done, fewer are publically available, fewer still had comparable final evaluations and included local voices. Agencies that fund post project evaluations offer a range of reasons for doing so: to learn, to promote a success, to inform replication or scale, to provide justification for future funding, to promote accountabilities. What is less explicit is how findings are used within implementing and donor agencies, shared with partners, and influence future programming. As part of this report, Valuing Voices offers an evaluability checklist for doing a post project evaluation from the onset of the project, and a checklist for measuring sustainability through the entire project cycle." (Abstract)
more
"The IDEAS Guide is for organisations and people who are doing small media and communication projects. It is designed to cater for a range of different skill levels in evaluation. It is ideal for project managers and team members responsible for reporting and evaluation, as well as any other partici
...
pants who are interested in evaluation. The IDEAS Guide leads you through the process of designing a project, and thinking about the project’s evaluation. The guide is focused on how to do useful evaluation for learning and continual project improvement. To achieve this, it is important to think about the evaluation of the project as you design it, and to learn from evaluation during the implementation of the project. Focusing on what you can learn by carrying out a project is especially important when the project idea is new and innovative." (Introduction, page 5)
more
"The IDEAS Facilitators’ Guide provides briefing notes on how you, as a facilitator, can support small-grant recipients to use the IDEAS Guide to clarify their project design, and to plan and manage their evaluation. The Facilitators’ Guide is best used in conjunction with the IDEAS Guide. These
...
guides are designed to be used together in the planning and implementation of a workshop. Facilitators can also adapt the steps to suit the mode and length of delivery. The Facilitators’ Guide was developed to accompany the IDEAS Guide to help facilitators to effectively support capacity development for individuals working in grassroots and community organisations who might have little or no experience of evaluation. It may also be helpful for more experienced practitioners. The approach integrates evaluation into the design of projects." (Page 4)
more
"This one-of-a-kind book fills a gap in the literature by providing readers with a systematic approach to monitoring and evaluation (M&E) training for programs and projects. Bridging theoretical concepts with practical, how-to knowledge, the authors draw upon the scholarly literature, applied resour
...
ces, and over 50 years of combined experience to provide expert guidance that can be tailored to different M&E training needs and contexts, including those for novices, professionals, organizations and their staff, community members, and other groups with a desire to learn and sustain sound M&E practices." (Publisher description)
more
"Community radio in Australia is now well established and considered an important part of the radio sector, however, in today’s economically driven world it is at the bottom of the media money pile. In order to argue for its continuing existence, funding and development in an ever-changing media l
...
andscape, some means of capturing its value is essential. This article describes the development of a theoretical framework of value for community radio. The content of the framework was achieved by, first, examining, community media/radio literature through five relevant lenses of analysis. Secondly, a subsequent meta-analysis was applied to consolidate the framework. In order to test the utility of the draft theoretical framework of value, three case studies were conducted with different types of community radio stations in Perth, Western Australia. Two primary research methods were used: interviews with staff and audience focus groups. The testing exercise provided a multimodal insight into the values of community radio as reflected in real life practice. The analysis revealed how value was perceived by participants across three stations as personal motivations, and second, that value at individual stations was contingent upon the characteristics of the individual community radio stations." (Abstract)
more
"The goal of this document is to improve the effectiveness of DFID programmes and the measurement of their impacts by providing DFID Advisers with the practical skills to develop high quality theories of change, to understand the role they play in programme design and assessment. It is intended for
...
DFID advisors to more clearly and explicitly articulate their theories of change as a means of improving the effectiveness of interventions. Part I first explores the fundamentals of theories of change: what they are, why they are important, and how to create a theory of change. It explores theories of change at different levels, and concludes with advice on how theories of change can enhance the effectiveness and relevance of programming. Part II continues to build upon Part I by focusing on how theories of change can be used in the monitoring and evaluation stages of the project cycle. It provides practical guidance on how and why to use theories of change-focused monitoring and evaluation strategies, particularly exploring the ways in which theories of change can be included in any evaluation approach." (Document summary, page 3)
more
"[...] Amongst those agencies the following OECD DAC definition of evaluability is widely accepted and has been applied within this report: “The extent to which an activity or project can be evaluated in a reliable and credible fashion”.
Eighteen recommendations about the use of Evaluability Ass
...
essments are presented here, based on the synthesis of the literature in the main body of the report. The report is supported by annexes, which include an outline structure for Terms of Reference for an Evaluability Assessment.
An Evaluability Assessment should examine evaluability: (a) in principle, given the nature of the project design, and (b) in practice, given data availability to carry out an evaluation and the systems able to provide it. In addition it should examine the likely usefulness of an evaluation. Results of an Evaluability Assessment should have consequences: for the design of an evaluation, the design of an M&E Framework, or the design of the project itself. An Evaluability Assessment should not be confused with an evaluation (which should deliver the evaluative judgements about project achievements).
Many problems of evaluability have their origins in weak project design. Some of these can be addressed by engagement of evaluators at the design stage, through evaluability checks or otherwise. However project design problems are also likely to emerge during implementation, for multiple reasons. An Evaluability Assessment during implementation should include attention to project design and it should be recognised that this may lead to a necessary re-working of the intervention logic." (Executive summary, page 1)
more
"Aid work in relation to conflict and peace is often based on approaches, strategies and tactics that are rooted in theories of change (understandings about why particular inputs or activities are expected to achieve intended results [outputs, outcomes and impacts]) that are unstated or ill-defined.
...
They are embedded in the skills and approaches of individual practitioners and peacebuilding organisations, their capacities and technologies, attachments to favourite methodologies, and the perspectives of different stakeholders about conflict and peace. In the imaginary example of an anti-bias peace programme for journalists in Annex C, one question would be how the planned workshops, consciousness raising, and skills development might actually change conflict reporting. The programme could track the language used in reporting before and after the effort and also survey public attitudes. At the same time, it could see whether the activities were achieving the expected results – or if unexpected obstacles appeared. For instance, it might turn out that individual journalists have very little influence over the use of inflammatory language and that editors determine the use of such language to boost sales. That outcome would suggest that the “theory”, about inducing changes in reporting by training journalists, was flawed. One related task is to identify the sources of theories. Are they a) based on experience (the programme designers’ personal and professional experience or that of the stakeholders and beneficiaries consulted during programme design); or b) research-based? Evaluation can contribute to improving the design and implementation of ongoing programmes." (Page 80)
more
"Focusing on theories of change can improve the effectiveness of peacebuilding interventions. A review of 19 peacebuilding projects in three conflict-affected countries found that the process of articulating and reviewing theories of change adds rigour and transparency, clarifies project logic, high
...
lights assumptions that need to be tested, and helps identify appropriate participants and partners. However, the approach has limitations, including the difficulty of gathering theory-validating evidence." (Executive summary)
more
"This research report considers the application of ToC to governance programmes in post-conflict and fragile states. It should be noted that, because of what information is available, the examples reviewed in this report are from post-conflict and least developed (though not necessarily fragile) sta
...
tes. They are mainly (but not all explicitly) governance-focused, and while some have elements of accountability, public sector management and public service delivery, none are about public financial management. The case studies discussed show that articulating and reviewing Theories of Change in peacebuilding programmes adds rigour and transparency, clarifies project logic, highlights assumptions that need to be tested and helps identify appropriate participants and partners. Yet, for ToC application to be effective, particularly in governance settings, a number of factors need to be taken into consideration, including: conflict analysis is critical for determining the relevance of activities in post-conflict contexts but is rarely done; gathering evidence to validate a Theory of Change is challenging, particularly in conditions of conflict and fragility; theories of Change can encourage an overly linear approach, when the reality of change in conflict contexts is complex and more organic; people involved, both staff and programme participants, may require a lot of support to understand the dynamics and effective use of ToC." (Overview, page 1-2)
more
"Developmental evaluation offers a powerful approach to monitoring and supporting social innovations by working in partnership with program decision makers. In this book, eminent authority Michael Quinn Patton shows how to conduct evaluations within a developmental evaluation framework. Patton draws
...
on insights about complex dynamic systems, uncertainty, nonlinearity, and emergence. He illustrates how developmental evaluation can be used for a range of purposes: ongoing program development, adapting effective principles of practice to local contexts, generating innovations and taking them to scale, and facilitating rapid response in crisis situations. Students and practicing evaluators will appreciate the book's extensive case examples and stories, cartoons, clear writing style, "closer look" sidebars, and summary tables." (Publisher description)
more