Impact evaluation

Shaima M.

Order the writing of a tailor-made Economy general Course material

Free quote online

Course material Format .docx

Impact evaluation

Download

Read an extract

Themes

challenges, economic development, impact, causal effect, information, counterfactuals, experimental methods

Reader
Abstract
Contents
Extract

Abstract

When you really do database business of development policy making, it makes it sound very idealistic to, for example, providing easy access to health, which means you intervening when someone is sick. In this case, you are intervening at the earliest stage (for example, providing clean water to avoid sicknesses, use vaccination in advance). And if for example, you see that there are 3 different projects that potentially can have a reduction in child mortality and suppose you have a limited budget, how do you allocate it? In practice you must make choices. At some point the question comes for the money that you spent. What is the biggest return in terms of improvement of your development objectives?

What is impact evaluation?
Estimating Causal Inference: How do we evaluate?
Counterfactuals: How do we construct a valid counterfactual?
1. Counterfactual criteria
2. Treated and Counterfactual
Experimental methods: randomized control trials (RCT's)
Quasi experimental methods
1. Difference in differences
2. Regression Discontinuity Design (RDD)

Get this table of contents for free after login.

Extract

[...] So, you don't know the alternative trajectory of what would have happen to these people in absence of the program. So therefore, many program that were evaluated based on comparing those who are enrolled and those who are not enrolled will making these mistakes by comparing not comparable groups. The voluntaries programs have the problem of self-selection. When you have self-selection, you should be careful interpreting as a causal effect. Experimental methods: randomized control trials (RCT's) Allocation of intervention through lottery or another random process generates two groups statistically identical. When do we randomize? [...]

[...] Impact evaluations have several steps: Step Challenge We start with a development problem. More than 10% of children in Paraguay suffer delays in their physical, emotional, or cognitive development Diarrhea is the second leading cause of child mortality School enrolment is Afghanistan is 54% Step Outcome We determine a desired outcome we want to achieve. (You quantify it). Decrease cognitive development delays by half Reduce diarrhea incidence by 20% Achieve universal primary enrolment Step Intervention We propose a potential solution, in the form of a program or policy. [...]

[...] and made the following experiment: there was a bull on sale and people wanted to buy it (there were 50/60 participants on there) and he said: could you please each of you tell me what do you think is a the weight of the animal? They gave a reasonably close number. So, he collected those information's from all the people around there. When he measures the average of the answers, the average was close to the real weight from 1 kg. It shows the power of low large number. That is the reason why you must take a lot of individuals in statistics analysis. [...]

[...] Therefore, it means that when you observe this but not observe this one you need to find a substitute, create an artificial comparison group against which you compare what happens under the treatment. How to construct the comparison group? that is all the story about how to do a valid impact evaluation. The key things are to have this counterfactual or comparison group which is valid. A valid counterfactual should satisfy some conditions: - One is, as compared to the treatment, the counterfactual must have essentially the same characteristics. Suppose you have two households in the village, one is on the treatment, and one is not. [...]

[...] In a project, you must first identify what are the relevant measure of what you would like to improve. To sum up, what you would like to estimate on your impact is what is the outcome under the treatment in the treatment group, the average outcome, what is the average outcome for the control group and then you take the difference, and that difference becomes your estimate of the impact. Two imperfect counterfactuals Before and after Same individual or group before and after the treatment - Those not enrolled Those who did not enrol in (were not offered) the program versus those who did (were) There are two imperfect ways of doing your counterfactual and many studies unfortunately use them: Before and after comparison (was long time used): the same individuals or same groups of people you get the measurement before the treatment and then after the program. [...]

docx