The impact evaluation in practice handbook is a comprehensive and accessible introduction to impact evaluation for policymakers and development practitioners. Data collection and analysis methods in impact evaluation page 2 outputs and desired outcomes and impacts see brief no. Our simplified and aggregate list includes 9 potential impacts of a food safety regulation. Handbook on impact evaluation foro consultivo cientifico. Although both experimental and nonexperimental methods are grounded in quantitative approach to evaluation, incorporating qualitative methods enriches the quality of the evaluation results. The theory of change should also take into account any unintended positive or negative results. The impact of the full range of development interventions can be assessed projects, programmes, sector wide support and donor country strategies and macro economic growth and programme aid support using qualitative or quantitative approaches or a mix. While quantitative research is based on numerical data analysed statistically, qual. Additional attention was drawn to the issue recently by a.
Matching methods are gaining visibility in impact evaluation because they use rigorous statistical techniques to build a control unprotected group of sites for comparison that are as similar as. Impact assessment methods this document outlines several common impact assessment methods, including. This guidance note provides an outline of a mixed methods impact evaluation with particular reference to the difference between this approach and qualitative and quantitative impact evaluation designs. Hence impact evaluation is aligned with results based management3 and monitoring the contribution of development assistance toward meeting the. Includes qualitative and quantitative data collection does not require advanced statistical methods. This book also details challenges and goals in other realms of evaluation.
The problem with seeking to advance a qualitative method of impact analysis is that impact is a matter of causation and a non quantitative approach to design is apparently not well suited to the task of establishing causal relations. Quantitative and qualitative methods in impact evaluation and measuring results 1 1 introduction there has been a renewed interest in impact evaluation1 in recent years amongst development agencies and donors. A basic understanding of quantitative research methods would be helpful. Students do not have to actually evaluate, but must be able to identify the correct. Understanding variation in treatment effects in education impact evaluations. C2960p november 19, 2018 evaluation briefs analyzing quantitative data for evaluation no. The book incorporates realworld examples to present practical guidelines for designing and implementing evaluations. This note describes a number of commonly used impact evaluation. Realism, subjectivism and the paradigm wars now we have defined quantitative research, it is a good idea to compare it with qualitative research, to which it is usually put in opposition. In particular it examines how qualitative methods could address some of the limitations of randomized trials and other quantitative impact evaluation methods. Using an experimental or quasiexperimental design to demonstrate a change or not is the most powerful programme evaluation for detecting changes in outcome. How to evaluate the programme world health organization. By identifying if development assistance is working or not, impact evaluation is also serving the accountability function.
Quantitative and qualitative methods in impact evaluation and measuring results. Impact evaluations and development no nie guidance on impact evaluation. Types of evaluation centers for disease control and. Descriptive statistics are performed by analyzing one variable at a time univariate analysis. This work is increasingly important as the global development community works to reduce poverty and achieve the.
Quantitative research methods nova southeastern university. The proper analysis of impact requires a counterfactual of what those outcomes would have been in the absence of the intervention. Using qualitative methods to assess impact jean ellis, ncvo charities evaluation services associate july 2015 1. Impact evaluation methods for youth employment interventions. Impact evaluation seeks to determine the longer term results that are generated by policy decisions, often through interventions, projects or programmes. The formal literature on impact evaluation methods and practices is. Research is a process of steps used to collect and analyze information in order to increase our understanding of a topic or issue. Needs assessment process evaluation impact evaluation to address the low literacy levels, the government will deliver free text books to schools to improve passing rates and increase employment prospects of school leavers. Section iv addresses some general concerns about impact evaluation, introducing them as myths and explaining why impact evaluation is less complicated than often assumed. The formal literature on impact evaluation methods and practices is large, with a few. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. Impact and outcome evaluations may be carried out using a variety of quantitative methods. Quantitative and qualitative methods in impact evaluation. In particular, qualitative methods not only provide qualitative measures of impact, but.
The choice of methods and designs for impact evaluation of policies and. Impact evaluations in low, moderate, or heterogeneoustransmission settings will likely use plausibility designs, and methods highlighted by the framework include interrupted time series. Pdf on apr 2, 2017, abhisek mishra and others published methods of impact evaluation. Students will learn to critically analyze evaluation research and to gauge how convincing the research is in identifying a causal impact. Formative evaluation ensures that a program or program activity is feasible, appropriate, and acceptable before it. Yet there is a need to put the theory into practice in a handson fashion for practitioners. Research process quantitative methods designs validity and research designs. Quantitative research, is defined as a the systematic investigation of phenomena by gathering quantifiable data and performing statistical, mathematical or computational techniques. Qualitative and quantitative methods can usefully be combined in mixed methods research to provide more comprehensive information for an evaluation. Outline of principles of impact evaluation part i key concepts definition impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended. Usaid definition of impact evaluation part 1 impact evaluations measure the change in a development outcome focused on higher level results outcomes rather than inputs and outputs e. Impact evaluation methodologies impact evaluation workshop, lilongwe, dec. Handbook on impact evaluation quantitative methods and practice.
Samad 2010 the international bank for reconstruction and development. There are several types of evaluations that can be conducted. Utilising quantitative and qualitative approaches in impact evaluation. A particular concern is the accessibility of health services to poor and vulnerable sectors of the population. Also learn about primary and secondary quantitative research along with techniques and types of. Quantitative and qualitative methods in impact evaluation gsdrc. The methods used will depend on the aim and the budget for the evaluation.
Introduction voluntary sector organisations are being urged to provide more convincing evidence of their impact and with good reason, given that funders and commissioners want. Expert judgment quantitative physical and mathematical models cumulative impact assessment matrices and interaction diagrams rapid impact assessment matrix riam battelle environmental evaluation system. This approach to evaluation focuses on the practicalities of defining successful outcomes and success cases brinkerhoff, 2003 and uses some of the processes from theorydriven evaluation to determine the linkages, which may take the form of a logic model, an impact. Understanding variation in treatment effects in education. All researchers perform these descriptive statistics before beginning any type of data analysis. Learn more about quantitative research methods along with types and examples, characteristics and advantages. The tools, while designed specifically for each type. Introduction to mixed methods in impact evaluation. Department of education, institute of education sciences, national center for education evaluation and regional assistance, analytic technical assistance and development. Measurement of impact of any developmental programme is a difficult task.
Research question evaluation design methods data to be collected, when. Analyze quantitative and qualitative data using biostatistics, informatics, computerbased programming and software, as. Impact evaluation in practice is an essential resource for evaluators, social programs, ministries, and others committed to making decisions using good evidence. Choosing appropriate designs and methods for impact. Impacts may be positive or negative, intended or unintended, direct or indirect. Impact evaluation provides the causal effect of a programme on the outcome. In response to this growing consensus there are a number of international initiatives on impact evaluation. While impact evaluations can be based on both quantitative and qualitative methods, this note focuses primarily on quantitative methods and introduces. Outline of principles of impact evalation discussion dra. Critically analyze impact evaluation research in economics and public health and gauge the validity of causal estimates understand evaluation design, including methods for designing randomizedcontrolled field trials learn how to develop meaningful hypotheses that are amenable to evaluation and test them. Using recent literature on the characteristics of such practices, we developed two tools to assess the methodological rigour and mixed methods integration of 40 ies and 7 srs, drawing upon previous approaches. Note 5 impact evaluation methods for youth employment.
Decision tree for selecting quantitative evaluation designs to deal. Handbook on impact evaluation quantitative methods and. Quantitative and qualitative methods in impact evaluation and measuring results this paper from the governance and social development resource centre grdrc provides a detailed discussion of using both qualitative and quantitative methods in impact evaluation. Evaluations using quantitative impact evaluation approaches 71 6. As you move across this gradient, the usefulness, potential questions answered, and results garnered change categorically.
300 1482 553 595 1622 755 1279 646 1516 935 1565 771 472 1040 966 888 453 974 737 1564 836 478 420 1325 515 1241 399 574 149