"This guide is designed for anyone who wants to learn how communication can more effectively support evaluation: evaluation consultants, communication consultants, evaluation commissioners and programme/project staff participating in evaluations. The guide is structured around three broad phases of
...
the evaluation process: before, during and after. The guide aims to provide advice and practical hints on how to use communication for each of these phases. A communication plan of action template adapted for evaluation is provided at annex 1. Examples of innovative communication tools for evaluation are found at annex 2. A case study is included at annex 3 that can be used for learning purposes. Links to extra resources are found at annex 4. The guide explains how to use communication to support evaluation and readers can adapt and use these approaches depending upon the stage of the evaluation process, their own context and available resources." (Page 4)
more
"Since The Seed Alliance started in 2012, the three program partners (FRIDA, FIRE and ISIF) have supported 151 projects throughout Africa, Asia Pacific, Latin America and the Caribbean, helping to strengthen and promote the Information Society within these regions. From the beginning of the program,
...
a set of criteria was defined to evaluate the projects supported by Seed Alliance. The criteria work as a system of categories disaggregated into sub-categories which are evaluated both when the projects are initially submitted and when the final reports are presented by the supported organizations. The current report results from the consulting work by the FOCUS team, which was aimed at redefining Seed Alliance´s evaluation system (score card system) looking to improve the comparability among the different projects." (Introduction, page 3)
more
"In a tracer study, data is collected and analysed repeatedly over time. For monitoring and evaluation purposes, tracer studies are often designed to track changes at individual level, following a development intervention. However, they might also focus on other units of analysis such as communities
...
, organisations or policies. Tracer studies are most useful when change is intended to be long-term and significant." (Introduction)
more
"A review of eight projects with end-of-project and post project evaluation reports suggest that post project evaluations can contribute to better understanding of the sustainability of project impacts and identify unexpected and emerging outcomes years after project close. The review highlights som
...
e of the methodological issues that are unique to post project (as opposed to end of project) evaluation, and point to the advantages of planning for sustainability measurement from the outset of the project. In the majority of cases, the sampling approach adopted at the endline proved challenging to replicate post project because it was not designed to measure sustainability, or because of access to or demographic changes in the beneficiary population. Finding suitable projects for this review was difficult because so few post project evaluations are done, fewer are publically available, fewer still had comparable final evaluations and included local voices. Agencies that fund post project evaluations offer a range of reasons for doing so: to learn, to promote a success, to inform replication or scale, to provide justification for future funding, to promote accountabilities. What is less explicit is how findings are used within implementing and donor agencies, shared with partners, and influence future programming. As part of this report, Valuing Voices offers an evaluability checklist for doing a post project evaluation from the onset of the project, and a checklist for measuring sustainability through the entire project cycle." (Abstract)
more
"Process tracing is a qualitative analysis methodology. The main purpose of process tracing is to establish whether, and how, a potential cause or causes influenced a specified change or set of changes. This is done by applying formal tests to examine the strength of evidence linking potential cause
...
s to the changes. Process tracing also involves testing alternative ideas about how change might have come about." (Introduction)
more
"This report described a pilot study using the VOTO Interactive Voice Response (IVR) platform to 1) distribute content and 2) collect quantitative and qualitative monitoring and evaluation data regarding a media program in Rwanda. In addition to showing that the VOTO system worked well as a distribu
...
tion tool (though expensive) – our findings suggest great potential for the use of IVR as an M&E tool. Unfortunately, we cannot recommend a particular system as yet having the capacity to offer an efficient and cost-effective system for NGOs looking to collect qualitative and quantitative data cheaply and accurately." (Discussion, page 13)
more
"This Module is not about placing a value on artistic expression, but rather about capturing the true impact of Participatory Theatre for Change (PTC) and helping advance the field through the development of vetted best practice. As discussed in the Module, this starts with design, focusing on the i
...
mplementation context and understanding the full dynamics in which PTC activities will take place. The Context Analysis process will help determine how PTC can best be utilised in the implementation environment, how complex the programme should be, and if there are additional activities needed to ensure impact. The information from preliminary design thinking and the Context Analysis then supports the identification and contextualization of the key interrelated theories of change that apply to the programme, serving as a foundation from which the activities can be evaluated. Monitoring then allows for an iterative process of designing individual performances, engaging appropriately and in a context sensitive manner with the community, as well as determining if the PTC activities as implemented are trending towards impact. Regular monitoring helps with course corrections when needed, and leads towards more effective programming. The monitoring of the process and quality helps ensure that the performances are doing no harm, artistic expressions, and implemented in such a manner as to achieve Reach, Resonance, and Response. Monitoring provides the temperature checks to make sure the ecosystem of PTC is balanced. And finally, evaluation helps push PTC programming to the next stage. Strong evaluations provide the community with more information about changes that have happened, even self-reflection through participatory methods. Evaluations also enable the actors to learn about the bigger picture of what worked and did not work beyond the immediacy of the performance space and towards the longer term impacts. It enables informed follow-on activities, as well as learnings that can be shared with the broader field to enhance practice more globally. Lastly, evaluations provide evidence of the great work participatory theatre does as a nuanced, emotionally connected, and transformative approach to communications for development." (Conclusion, pages 38-39)
more
"This report presents an evaluation of the utility of social media for conducting research, monitoring and evaluation in the MENA region, using a case study approach. The Twitter based reaction to the reported cancellation/reduction of food provision by World Food Programme (WFP) for Syrian Refuges
...
was taken as the case study [...] The application of frequency analysis to the case study showed that information regarding ebb and flow of the popularity or awareness of the cuts to the WFP could be obtained from Twitter. Spikes in popularity or awareness on Twitter related to key reports in the media relating to the cuts. These spikes were subjected to further analysis using topic classification. Simple topic classification was performed on the spikes identified in the frequency analysis. Wordclouds are simple but effective methods for viewing the high-level discussion on social media. Words are sized in the wordcloud based on their frequency – the bigger the word, the more frequently it is used. The accounts @60Minutes and @bbcworld were dominant in the visualizations, confirming the broadcast reaction on Twitter. To further refine the analysis these Twitter accounts were visualised in retweet networks. Social Network graphs were used to visualise interactions between accounts posting about cuts to the WFP on Twitter. The size of nodes (accounts) in the graphs corresponded to amount of interactions with other nodes (accounts), indicating their relative importance in propagating information about the cuts to the WFP on Twitter. The graphs also identified nodes (accounts) that acted as bridges for information to travel from one Twitter community to another." (Executive summary)
more
"This guide shows, besides other things, how to implement the traffic light function of Excel. Aside from this function, it is in general very easy to visualize information with Excel. In this respect, the progress of indicators along the results chain of a project, but also the cash flow can be gra
...
phically presented [...] Another goal of this manual is to stress the point that Excel is much more than a static tool in which data can be handled in tabular format. By means of cross-linking data, the whole monitoring system becomes dynamic so that the whole data processing works nearly automated. Time and resource intensive double entries of information can be avoided, since the majority of data can be updated, calculated and visualized automatically." (Introduction)
more
"Drawing on more than 40 years of experience conducting applied social science research and program evaluation, author Michael Quinn Patton has crafted the most comprehensive and systematic book on qualitative research and evaluation methods, inquiry frameworks, and analysis options available today.
...
Now offering more balance between applied research and evaluation, this Fourth Edition illuminates all aspects of qualitative inquiry through new examples, stories, and cartoons; more than a hundred new summarizing and synthesizing exhibits; and a wide range of new highlight sections/sidebars that elaborate on important and emergent issues. For the first time, full case studies are included to illustrate extended research and evaluation examples. In addition, each chapter features an extended "rumination," written in a voice and style more emphatic and engaging than traditional textbook style, about a core issue of persistent debate and controversy." (Publisher description)
more
"This toolkit is for organisations planning to use 'participatory video and the most significant change' (PVMSC) to evaluate their social change projects. Part one gives an overview of the key stages and things to consider when preparing for a PVMSC process and includes perspectives and feedback fro
...
m partners and participants. Part two is a guide for facilitators as they carry out the games and exercises and includes case studies drawn from 10 years of InsightShare’s experience in a variety of contexts and countries." (Back cover)
more
"The paper focuses on exploring the application of the following ICTs: mobile technology, social media, big data, the digitization of surveys, and tools to better visualize data. Each section outlines a series of resources and some advantages and drawbacks to utilizing the new technology for M&E pur
...
poses. When appropriate, examples and case studies have been included." (Document summary)
more
"A practical guide to managing and leading evaluation efforts, this module is designed for use in evaluations within your organization and among your partners. Increasing local capacity to conduct and manage good program evaluation is a key goal of the Results and Measurement team at Pact ... Each c
...
hapter’s learning objectives and exercises relate to sections of Pact’s evaluation protocol, and each chapter builds on the previous one. By the end of the module, if you have worked your way through the exercises, you should have a comprehensive, written plan for your evaluation—that is, a full terms of reference (TOR). In addition, a TOR template can be found in Appendix 1 (page 90). The shaded boxes that introduce and give an overview of each chapter refer to relevant sections of this template." (Foreword)
more
"In this paper we describe how quality in reporting could be measured through content analysis. We show that this approach, although somewhat technical, is feasible. It can help projects to become better and more successful. As a suggestion for practitioners in media development we present three opt
...
ions for measuring quality of reporting for monitoring and evaluation purposes." (Abstract)
more
"A comprehensive, practical guide to the on-the-ground tasks of evaluating and monitoring democracy assistance programs, from planning and implementation to preparing and presenting evaluation reports." (Publisher description)
"Overall, this paper finds that peacebuilding practitioners have much to learn in terms of new media use from their humanitarian and development colleagues. New media is increasingly and often successfully used for needs assessments and beneficiary communication but only very few examples exist in w
...
hich new media are applied for (improving) monitoring and evaluation in peacebuilding contexts. In a sense, therefore, peacebuilders are latecomers to an action trend that is far more advanced in other fields." (Page 2)
more
"Der Artikel beabsichtigt, den derzeitigen Diskussionsstand zu Theorie und Praxis der Gruppendiskussion bezogen auf das Feld der Evaluation zusammenzutassen, die Stärken und Grenzen von Gruppendiskussionen in evaluativen Vorhaben zu skizzieren und eine weitere Fundierung der Gruppendiskussion in de
...
r Evaluation anzuregen. Nach einer definitorischen Klärung verwandter Begrifflichkeiten wie Gruppendiskussion, Focus Group und Gruppeninterview wird ein Überblick über die Entwicklung und Rezeption der Methode in der Literatur der Sozialforschung und Evaluation gegeben. Es wird herausgearbeitet, was Evaluation als Disziplin im Vergleich zur Sozialforschung allgemein ausmacht und welche besonderen Leistungspotenziale sich hieraus für die Gruppendiskussion in der Evaluation ergeben. Zwei idealtypische Formen der Gruppendiskussion werden kontrastiert und zentrale Gestaltungsparameter bei der Planung, Durchführung und Auswertung vorgestellt und diskutiert." (Zusammenfassung)
more
"The present paper explores approaches for assessing efficiency. Efficiency is a powerful concept for decision making and ex post assessments of development interventions but, nevertheless, often treated rather superficially in project appraisal, project completion and evaluation reports. Assessing
...
efficiency is not an easy task but with potential for improvements, as the report shows. Starting with definitions and the theoretical foundations the author proposes a three level classification related to the analytical power of efficiency analysis methods. Based on an extensive literature review and a broad range of interviews, the report identifies and describes 15 distinct methods and explains how they can be used to assess efficiency. It concludes with an overall assessment of the methods described and with recommendations for their application and further development." (Foreword)
more
"Mixed methods (MM) evaluations seek to integrate social science disciplines with predominantly quantitative (QUANT) and predominantly qualitative (QUAL) approaches to theory, data collection, data analysis and interpretation. The purpose is to strengthen the reliability of data, validity of the fin
...
dings and recommendations, and to broaden and deepen our understanding of the processes through which program outcomes and impacts are achieved, and how these are affected by the context within which the program is implemented. While mixed methods are now widely used in program evaluation, and evaluation RFPs frequently require their use, many evaluators do not utilize the full potential of the MM approach."
more
"The tools nonprofits need to measure the impact of their social media. Having a social media measurement plan and approach can no longer be an after-thought. It is a requirement of success. As nonprofits refine their social media practice, their boards are expecting reports showing results. As fund
...
ers provide dollars to support programs that include social media, they too want to see results. This book offers the tools and strategies needed for nonprofits that need reliable and measurable data from their social media efforts. Using these tools will not only improve a nonprofit's decision making process but will produce results-driven metrics for staff and stakeholders. A hands-on resource for nonprofit professionals who must be able to accurately measure the results of their social media ventures. Written by popular nonprofit blogger Beth Kanter and measurement expert Katie Delahaye Paine. Filled with tools, strategies, and illustrative examples that are highly accessible for nonprofit professionals. This important resource will give savvy nonprofit professionals the information needed to produce measurable results for their social media." (Publisher description)
more