"This text provides a solid foundation in program evaluation, covering the main components of evaluating agencies and their programs, how best to address those components, and the procedures to follow when conducting evaluations. Different models and approaches are paired with practical techniques,
...
such as how to plan an interview to collect qualitative data and how to use statistical analyses to report results. In every chapter, case studies provide real world examples of evaluations broken down into the main elements of program evaluation: the needs that led to the program, the implementation of program plans, the people connected to the program, unexpected side effects, the role of evaluators in improving programs, the results, and the factors behind the results. In addition, the story of one of the evaluators involved in each case study is presented to show the human side of evaluation." (Publisher description)
more
"CSOs sometimes need to summarise or aggregate information across multiple interventions. This can be a difficult and challenging task, especially for large non-governmental organisations (NGOs) working in many different countries and/or sectors. Summarisation and aggregation can be achieved through
...
a variety of methods. However, all come with associated costs." (Introduction)
more
"Beginning in 2002, working closely with co-evaluators and commissioners of evaluations, the author developed Outcome Harvesting to enable evaluators, grant makers, and managers to identify, formulate, verify, and make sense of changes that interventions have influenced in a broad range of cuttingâ
...
€“edge innovation and development projects and programs around the world. Over these years, he led Outcome Harvesting evaluative exercises involving almost 500 non-governmental organizations, networks, government agencies, funding agencies, community-based organizations, research institutes and university programs. In over fifty evaluations, with forty co-evaluators he has harvested thousands of outcomes on six continents. Outcome Harvesting has proven useful in evaluations of a great diversity of initiatives: human rights advocacy, political, economic and environmental advocacy, arts and culture, health systems, information and communication technology, conflict and peace, water and sanitation, taxonomy for development, violence against women, rural development, organic agriculture, participatory democracy, waste management, public sector reform, good governance, eLearning, social accountability, and business competition, amongst others. In this book, the author explains the steps of Outcome Harvesting and how to customize them according to the nine underlying principles. He shares his experience and gives practical advice on how to work with Outcome Harvesting and remain true to its essential features." (Back cover)
more
"Chapters feature: A review of 30 frameworks and models that inform processes for evaluation in advertising, public relations, health communication and promotion, government communication and other specialist fields including the latest recommendations of industry bodies, evaluation councils, and re
...
search institutes in several countries; Recommendations for standards based on contemporary social science research and industry initiatives such as the Task Force for Standardization of Communication Planning and Evaluation Models and the Coalition for Public Relations Research Standards; A comprehensive review of metrics that can inform evaluation including digital and social media metrics, 10 informal research methods, and more than 30 formal research methods for evaluating public communication; evaluation of public communication campaigns and projects in 12 contemporary case studies." (Back cover)
more
"The idea of using mixed methods has a long tradition in social research. But it is also recognized that mixed methods are often poorly applied. In quantitative analysis, the qualitative component, if any, is often poorly designed, integrated or reported. These guidelines are to support the design,
...
conduct and reporting of mixed-methods in quantitative impact evaluations. That is, impact studies using a large statistical design with a qualitative component. The guidelines are based largely on the CEDIL inception paper by Jimenez et al. (2018), ‘Mixing and matching: using qualitative methods to improve quantitative impact evaluations (IEs) and systematic reviews (SRs) of development outcomes’, supplemented by a review of other relevant guidelines e.g. the CONSORT extension for development effectiveness (Bose, 2010)." (Page 1)
more
"The Evaluating C4D Resource Hub houses a growing collection of the best available guides, toolkits, tools and methods to use for research monitoring and evaluation (R,M&E) of Communication for Development (C4D) initiatives. The Hub is structured around two combined frameworks: C4D Evaluation Framew
...
ork is an approach. It describes the values and principles that guide our decisions in C4D. The BetterEvaluation Rainbow Framework is a structure. It organises the practical tasks into seven categories or 'clusters' and provides options. The goal of the Evaluating C4D Resource Hub is to find the right kinds of options, tools and resources that suit your practical needs and match the approach you want to take." (www.betterevaluation.org/en/C4D-Hub)
more
"Evaluation Practice for Collaborative Growth highlights the approaches, tools, and techniques that are most useful for evaluating educational and social service programs. This book walks the reader through a process of creating answerable evaluations questions, designing evaluation studies to answe
...
r those questions, and analyzing, interpreting, and reporting the evaluation's findings so they are useful and meaningful for key stakeholders. The text concludes with a devoted to the shifting landscape of evaluation practice as it faces complex systems and issues that are shaped by society. Additionally, the author provides a list of knowledge and skills needed to adapt to a changing landscape and encourages organizations to use evaluation as a mechanism for learning and adapting to change. Her orientation toward community-based approaches and social justice prevail throughout the book's content and align well with a reader's desire to be inclusive and accountable in programing efforts. Nonprofit leaders, social science professionals, and students will find this book helpful for understanding basic program evaluation concepts, methods, and strategies." (Publisher description)
more
"This guide is designed for anyone who wants to learn how communication can more effectively support evaluation: evaluation consultants, communication consultants, evaluation commissioners and programme/project staff participating in evaluations. The guide is structured around three broad phases of
...
the evaluation process: before, during and after. The guide aims to provide advice and practical hints on how to use communication for each of these phases. A communication plan of action template adapted for evaluation is provided at annex 1. Examples of innovative communication tools for evaluation are found at annex 2. A case study is included at annex 3 that can be used for learning purposes. Links to extra resources are found at annex 4. The guide explains how to use communication to support evaluation and readers can adapt and use these approaches depending upon the stage of the evaluation process, their own context and available resources." (Page 4)
more
"Since The Seed Alliance started in 2012, the three program partners (FRIDA, FIRE and ISIF) have supported 151 projects throughout Africa, Asia Pacific, Latin America and the Caribbean, helping to strengthen and promote the Information Society within these regions. From the beginning of the program,
...
a set of criteria was defined to evaluate the projects supported by Seed Alliance. The criteria work as a system of categories disaggregated into sub-categories which are evaluated both when the projects are initially submitted and when the final reports are presented by the supported organizations. The current report results from the consulting work by the FOCUS team, which was aimed at redefining Seed Alliance´s evaluation system (score card system) looking to improve the comparability among the different projects." (Introduction, page 3)
more
"In a tracer study, data is collected and analysed repeatedly over time. For monitoring and evaluation purposes, tracer studies are often designed to track changes at individual level, following a development intervention. However, they might also focus on other units of analysis such as communities
...
, organisations or policies. Tracer studies are most useful when change is intended to be long-term and significant." (Introduction)
more
"Contribution analysis is a methodology used to identify the contribution a development intervention has made to a change or set of changes. The aim is to produce a credible, evidence-based narrative of contribution that a reasonable person would be likely to agree with, rather than to produce concl
...
usive proof. Contribution analysis can be used during a development intervention, at the end, or afterwards." (Introduction)
more
"At the W.K. Kellogg Foundation, we believe that evaluation is an effective management tool to both inform strategy development and track the progress and impact of strategy implementation. We have long been committed to supporting our grantees’ ability to derive and share lessons learned from the
...
ir work. To that end, the foundation published the first Evaluation Handbook (The Kellogg Foundation, 1998) almost two decades ago to guide evaluation for our grantees. Since that time, as the discipline of evaluation grew and expanded, the demand for evaluation has risen. More and more nonprofit leaders and practitioners strive to design evidence-based programs, and more and more funders require their grantees to provide evidence to demonstrate the success of their funded work. The democratization of evaluation makes it necessary that evaluation is both rigorous and practical. How to achieve the balance motivated us to update the handbook. Over the years, the foundation has learned a lot from our grantees about the challenges of evaluation. This handbook is our continuous effort to demystify evaluation and facilitate its use, for the foundation’s grantees and for all organizations committed to learning and strengthening their work. It is designed for people who have little to no exposure to formal evaluation training and provides a starting point for them as they consider evaluating their work. It is intended to help them become more informed consumers of evaluation. Evaluations can be simple or extensive depending on the scope and complexity of the work being evaluated. The scope of the evaluation could potentially include a single program, a multi-site initiative, or a multifaceted strategy aimed at systems and community change. Regardless of the complexity of the effort, the basics for evaluating it remain the same and this handbook was written to impart information about these basics." (Foreword)
more
"Process tracing is a qualitative analysis methodology. The main purpose of process tracing is to establish whether, and how, a potential cause or causes influenced a specified change or set of changes. This is done by applying formal tests to examine the strength of evidence linking potential cause
...
s to the changes. Process tracing also involves testing alternative ideas about how change might have come about." (Introduction)
more
"This report described a pilot study using the VOTO Interactive Voice Response (IVR) platform to 1) distribute content and 2) collect quantitative and qualitative monitoring and evaluation data regarding a media program in Rwanda. In addition to showing that the VOTO system worked well as a distribu
...
tion tool (though expensive) – our findings suggest great potential for the use of IVR as an M&E tool. Unfortunately, we cannot recommend a particular system as yet having the capacity to offer an efficient and cost-effective system for NGOs looking to collect qualitative and quantitative data cheaply and accurately." (Discussion, page 13)
more
"This Module is not about placing a value on artistic expression, but rather about capturing the true impact of Participatory Theatre for Change (PTC) and helping advance the field through the development of vetted best practice. As discussed in the Module, this starts with design, focusing on the i
...
mplementation context and understanding the full dynamics in which PTC activities will take place. The Context Analysis process will help determine how PTC can best be utilised in the implementation environment, how complex the programme should be, and if there are additional activities needed to ensure impact. The information from preliminary design thinking and the Context Analysis then supports the identification and contextualization of the key interrelated theories of change that apply to the programme, serving as a foundation from which the activities can be evaluated. Monitoring then allows for an iterative process of designing individual performances, engaging appropriately and in a context sensitive manner with the community, as well as determining if the PTC activities as implemented are trending towards impact. Regular monitoring helps with course corrections when needed, and leads towards more effective programming. The monitoring of the process and quality helps ensure that the performances are doing no harm, artistic expressions, and implemented in such a manner as to achieve Reach, Resonance, and Response. Monitoring provides the temperature checks to make sure the ecosystem of PTC is balanced. And finally, evaluation helps push PTC programming to the next stage. Strong evaluations provide the community with more information about changes that have happened, even self-reflection through participatory methods. Evaluations also enable the actors to learn about the bigger picture of what worked and did not work beyond the immediacy of the performance space and towards the longer term impacts. It enables informed follow-on activities, as well as learnings that can be shared with the broader field to enhance practice more globally. Lastly, evaluations provide evidence of the great work participatory theatre does as a nuanced, emotionally connected, and transformative approach to communications for development." (Conclusion, pages 38-39)
more
"The aim of this handbook is to provide civil society campaigners with the knowledge and skills they need to undertake high quality counter-narrative campaign evaluations. It presents model frameworks and case studies of successful counter-narrative M&E, as well as a range of useful tools and resour
...
ces that counter-narrative campaigners can use to support their M&E efforts, including: A description of key metrics and how to interpret them; An explanation of various research methods that can be used in online M&E; Guidance on potential risks and ethical considerations for counter-narrative M&E; A flexible M&E framework that can provide a starting point for evaluation planning; Three case study examples of high quality counter-narrative campaign evaluations; A glossary of key M&E terms; • An index of useful online M&E tools." (Page 3)
more
"This report presents an evaluation of the utility of social media for conducting research, monitoring and evaluation in the MENA region, using a case study approach. The Twitter based reaction to the reported cancellation/reduction of food provision by World Food Programme (WFP) for Syrian Refuges
...
was taken as the case study [...] The application of frequency analysis to the case study showed that information regarding ebb and flow of the popularity or awareness of the cuts to the WFP could be obtained from Twitter. Spikes in popularity or awareness on Twitter related to key reports in the media relating to the cuts. These spikes were subjected to further analysis using topic classification. Simple topic classification was performed on the spikes identified in the frequency analysis. Wordclouds are simple but effective methods for viewing the high-level discussion on social media. Words are sized in the wordcloud based on their frequency – the bigger the word, the more frequently it is used. The accounts @60Minutes and @bbcworld were dominant in the visualizations, confirming the broadcast reaction on Twitter. To further refine the analysis these Twitter accounts were visualised in retweet networks. Social Network graphs were used to visualise interactions between accounts posting about cuts to the WFP on Twitter. The size of nodes (accounts) in the graphs corresponded to amount of interactions with other nodes (accounts), indicating their relative importance in propagating information about the cuts to the WFP on Twitter. The graphs also identified nodes (accounts) that acted as bridges for information to travel from one Twitter community to another." (Executive summary)
more
"Drawing on more than 40 years of experience conducting applied social science research and program evaluation, author Michael Quinn Patton has crafted the most comprehensive and systematic book on qualitative research and evaluation methods, inquiry frameworks, and analysis options available today.
...
Now offering more balance between applied research and evaluation, this Fourth Edition illuminates all aspects of qualitative inquiry through new examples, stories, and cartoons; more than a hundred new summarizing and synthesizing exhibits; and a wide range of new highlight sections/sidebars that elaborate on important and emergent issues. For the first time, full case studies are included to illustrate extended research and evaluation examples. In addition, each chapter features an extended "rumination," written in a voice and style more emphatic and engaging than traditional textbook style, about a core issue of persistent debate and controversy." (Publisher description)
more
"This toolkit is for organisations planning to use 'participatory video and the most significant change' (PVMSC) to evaluate their social change projects. Part one gives an overview of the key stages and things to consider when preparing for a PVMSC process and includes perspectives and feedback fro
...
m partners and participants. Part two is a guide for facilitators as they carry out the games and exercises and includes case studies drawn from 10 years of InsightShare’s experience in a variety of contexts and countries." (Back cover)
more