"Quality assurance (QA) systems applied in educational contexts are generally concerned with inputs — how much money is spent, what staffing, resources and support are provided, what kinds of teaching and learning are involved, and so on. There is an assumption — not always fulfilled — that th
...
e higher the standards of the inputs, the higher the quality of the outputs. In this toolkit, we propose a different approach: the evaluation of the programmes’ outcomes, outputs and impacts. We examine the differences between informal and self-directed learning, nonformal education and training (NFE) and formal education; provide examples of NFE programmes using a variety of face-to-face, distance education and technology-based teaching and learning methods; examine the approaches to QA that are required in NFE; consider the outputs, outcomes and impacts that can be achieved in NFE programmes; propose the adoption of a rigorous but simple-to-use QA framework which is based on outputs, outcomes and impacts." (Back cover)
more
"The present text explains how the Media Sustanability Index has refined its instrument and procedures to better capture the impact of social media." (Abstract)
"The present paper explores approaches for assessing efficiency. Efficiency is a powerful concept for decision making and ex post assessments of development interventions but, nevertheless, often treated rather superficially in project appraisal, project completion and evaluation reports. Assessing
...
efficiency is not an easy task but with potential for improvements, as the report shows. Starting with definitions and the theoretical foundations the author proposes a three level classification related to the analytical power of efficiency analysis methods. Based on an extensive literature review and a broad range of interviews, the report identifies and describes 15 distinct methods and explains how they can be used to assess efficiency. It concludes with an overall assessment of the methods described and with recommendations for their application and further development." (Foreword)
more
"The study did not aim to carry out new surveys or collect information that was not already publicly available. Since the toolkit is aimed at providing media development organizations with a tool to assess media landscapes without having to carry out extensive new research, the Ghana study worked on
...
ly with materials, surveys, and other information available at the time of writing. In testing the toolkit in Ghana, the study found that the indicators selected and the methodology based on collecting direct information and materials already available is a useful approach to create a comprehensive media landscape assessment. It allowed us to provide an overview of the situation in which media in Ghana operate and to identify areas for future media development programs." (Methodology, page 3)
more
"The present article discusses how the new categoryof “citizen journalist” fits into the overall media environment and how the Freedom House Freedom of the Press methodology has been changed to incorporate this category." (Abstract)
"The European Commission has begun to integrate a political economy approach throughout the project cycle. This guide tailors the political economy approach to the media development sector specifically. Using a political economy media matrix (PEMM) as its basis, it explains how to integrate the PE a
...
pproach throughout the programme cycle." (Background, page 4)
more
"BBC Media Action has made strengthening the evidence base of reports, data sets, and analysis about the role of media and communication in democratic development a priority. Yet, the extent to which different donor organizations, foundations, think tanks, practitioners, and academics utilize resear
...
ch in general and BBC Media Action research in particular, remains uncertain. To shed greater light on this question, in October 2012, the Center for Global Communication Studies at the Annenberg School for Communication, University of Pennsylvania, conducted a quantitative survey followed by in-depth, structured interviews with 57 development stakeholders from four areas of development: governance, humanitarian response, health, and resilience. These research methods were used to investigate: (1) prevailing perceptions about and uses of media among assorted development stakeholders, (2) the extent to which these stakeholders are familiar with and utilize existing research concerning media development and media for development, (3) if and how funders are using this research to decide funding and policy making priorities, and (4) the specific impact of research and evidence initiated by BBC Media Action." (Executive summary)
more
"This case study of USAID media assistance program reporting documents (n=68) looks at specific monitoring and evaluation characteristics as reported over a 20-year period and how reporting documents make the link to democratization. The analysis found that although M&E activity has improved as repo
...
rted over the 20-year period of the study, 75 percent of the documents ranked in the lower half of cumulative M&E characteristics scoring. It also found that the relationship between democratization characteristics and media assistance are not clarified by the monitoring and evaluation data as reported. The study does show the USAID database to be a rich source of data about how media assistance programs have been implemented in different cultures, countries and political environments." (Abstract)
more
"In step with United Nations efforts to further promote universal standards and better protect people against human rights violations, this publication of the Office of the United Nations High Commissioner for Human Rights provides practical guidance for the development of quantitative and qualitati
...
ve indicators to strengthen the measurement and implementation of human rights, including the right to development. It contains a detailed description of the conceptual and methodological framework for human rights indicators recommended by international and national human rights mechanisms and used by a growing number of governmental and non-governmental actors. Concrete examples of indicators identified for a number of human rights—all stemming from the Universal Declaration of Human Rights—and other practical tools and illustrations are provided to support processes and stakeholders that aim to improve the realization of human rights on the ground." (Back cover)
more
"This article presents an overview of the emergence of sustainability themes in communication for development and argues that there is an urgent need for a framework of sustainability indicators for communication for development and social change projects around the world. It fills a crucial gap in
...
the growing body of literature by first synthesizing the most relevant data currently produced by global and local institutions, NGOs, UN-based organizations, academics, and professionals regarding assessment indicators for development projects, and second, produces a framework of sustainability indicators that can be used by a wide variety of people in the field to assess the sustainability of existing projects and the sustainable potential of planned ones. It then tests the framework in two representative cases." (Abstract)
more
"Behavior Change Communication (BCC) interventions have become an integral part of many HIV prevention programs. Monitoring and evaluation is expected to be included in the design of any BCC interventions from the beginning on. However, the know-how on how to demonstrate results and impacts of such
...
interventions in a methodologically sound way often lags behind. This document aims at enhancing program officials' understanding of the importance of evaluations as well as the most important steps to take and decisions to make in the course of an evaluation, and at enabling them to oversee the work done by the evaluators.
It is divided into three parts. The first part presents the main general aspects that need to be taken into consideration when monitoring and evaluating BCC interventions with a focus on HIV-related BCC approaches. More specifically, a general framework for monitoring and evaluation is presented, including among others information on the use of appropriate indicators, the development of an evaluation plan, different types of evaluation, study designs of summative evaluations, statistical analysis and dissemination of the evaluation results. Based on this theoretical background presented in part one, the second part draws conclusions, presents recommendations and gives practical advice on monitoring and evaluating HIV BCC programs. Three types of HIV BCC interventions, relevant to the work of German Development Cooperation (GDC), are also here discussed; these are the tool Join-in-Circuit, peer education programs and mass media campaigns. The third part presents four country examples of M&E of HIV BCC interventions implemented by GDC programs. These were discussed during a short-course on evaluation of BBC interventions offered by GIZ from 9]10 March, 2012 in Johannesburg, South Africa." (Executive summary)
more
"This first guidance note, Introduction to Impact Evaluation, provides an overview of impact evaluation, explaining how impact evaluation differs from – and complements – other types of evaluation, why impact evaluation should be done, when and by whom. It describes different methods, approaches
...
and designs that can be used for the different aspects of impact evaluation: clarifying values for the evaluation, developing a theory of how the intervention is understood to work, measuring or describing impacts and other important variables, explaining why impacts have occurred, synthesizing results, and reporting and supporting use. The note discusses what is considered good impact evaluation – evaluation that achieves a balance between the competing imperatives of being useful, rigorous, ethical and practical – and how to achieve this. Footnotes throughout the document contain references for further reading in specific areas." (Introduction, page 1)
more
"Since impact evaluations focus on results and what contributes to results, logically there should be a greater enthusiasm for making use of the findings. Impact evaluations also tend to cost more than other types of evaluation, implying that they should command significant follow up as well. It tur
...
ns out, however, that credible evidence and significant costs are not sufficient to ensure that impact evaluation findings are used. This guidance note looks to several additional factors that contribute to effective evaluation use. Using impact evaluations depends importantly on how one designs and conducts them. As was illustrated in Guidance Note 3 on mixed methods, the choice of evaluation method should follow from the questions one is trying to answer. But whatever questions are being tackled, whatever evaluation methods are employed, to make the best and proper use of evaluation findings we can apply a common set of practices and insights." (Introduction)
more
"The main aim of this study is to examine and document the current state of how communicators and evaluators share results of development co-operation and specifically the outcomes of evaluations, including ‘negative’ or ‘sensitive’ results. Related to this, the study will shed light on the
...
state of evaluation of the efforts undertaken by OECD aid agency communicators themselves: looking at whether, to what extent and how donors monitor measure and evaluate the effectiveness and impact of their communication strategies. First, the study will highlight key trends that have shaped the communication context for development evaluation. Second, and in separate sections, it will reflect on the evaluation and communication perspective around communicating results. For each of the two disciplines, and with reference to results of recent surveys, the paper will reflect on questions of definition, mandate, track record, challenges, and status of collaboration, and reference examples of emerging good and bad practice. Third, it will highlight the dynamics around communication of ‘negative’ or ‘sensitive’ evidence, identified by both evaluators and communicators as among the biggest challenges to be addressed. Fourth, it will look at how systematically agency communication strategies and initiatives are evaluated, exploring the extent to which evaluators and communicators work together in assessing these strategies. Fifth, the study will reflect on the experience to date in involving partner countries in communicating evaluation results, before concluding with a series of proposals aimed at improving collaboration between evaluators and communicators." (Introduction)
more
"When done well, evaluation for learning can help grantmakers, their grantees and their partners improve outcomes on the ground in real time. But doing it well requires that we work with key stakeholders to develop the leadership, the strategies and the systems that facilitate true learning.
1. LEAD
...
. Create a culture where evaluation is an everyday priority and where it supports and advances continuous learning. Build commitment to evaluation for learning from your board and staff leaders and create spaces for key stakeholders to reflect on your work. (Page 7)
2. PLAN. Develop a framework to ensure you, your grantees and your partners are “evaluating with a purpose.” Determine what your stakeholders need to understand in order to do a better job and develop ways that ensure everyone is gaining this knowledge on an ongoing basis. (Page 12)
3. ORGANIZE. Ensure you and your grantees have the necessary infrastructure to support your plan. This means establishing the right skills, processes and technology to make evaluation for learning an ongoing priority. (Page 16)
4. SHARE. Collaborate with grantees, grantmaking colleagues and others to ensure that evaluation is producing meaningful results. Involve grantees and partners when developing or reviewing strategies, share lessons on an ongoing basis with key audiences and engage in open relationships with grantees to support learning. (Page 23)
The goal of this guide is to provide grantmakers with ideas and insights so they can develop and strengthen their capacities in each of these four areas. Each section presents key action steps for grantmakers, along with examples of a variety of grantmakers engaged in this work. The fictional story of Anytown Foundation also illustrates how a foundation might build the four essential evaluation elements." (Publisher description)
more
"Mixed methods (MM) evaluations seek to integrate social science disciplines with predominantly quantitative (QUANT) and predominantly qualitative (QUAL) approaches to theory, data collection, data analysis and interpretation. The purpose is to strengthen the reliability of data, validity of the fin
...
dings and recommendations, and to broaden and deepen our understanding of the processes through which program outcomes and impacts are achieved, and how these are affected by the context within which the program is implemented. While mixed methods are now widely used in program evaluation, and evaluation RFPs frequently require their use, many evaluators do not utilize the full potential of the MM approach."
more
"This guidance aims to help improve programme design and management and strengthen the use of evaluation in order to enhance the quality of conflict prevention and peacebuilding work. It seeks to guide policy makers and country partners, field and programme officers, evaluators and other stakeholder
...
s engaged in settings of conflict and fragility by supporting a better, shared understanding of the role and utility of evaluations, outlining key dimensions of planning for them, setting them up, and carrying them out. This guidance is to be used for assessing activities (policies, programmes, strategies or projects) in settings of violent conflict or state fragility, such as peacebuilding and conflict prevention work and development and humanitarian activities that may or may not have specific peace-related objectives. This encompasses the work of local, national, regional and non-governmental actors, in addition to development co-operation activities. The central principles and concepts in this guidance, including conflict sensitivity and the importance of understanding and testing underlying theories about what is being done and why, are applicable to a range of actors." (Executive summary, page 8)
more