Do international aid programs work? Many of us send money to an organization like Heifer International, Mercy Corps, or Oxfam. Does the money these programs invest, either through direct aid, microloans, gifts of livestock, or other investment, improve the financial and psychological well-being of the recipients?
For a long time, we have not had definitive evidence that such programs improve people's lifestyles. According to this story in NPR, (which you can listen to here):
One problem was that no one was actually testing global aid programs — methodically — to see if they really changed people's lives permanently. "They haven't been taking the scientific method to problems of poverty,"
Dean Karlan of Yale University was a lead author on a study on these aid programs. The NPR story describes, first, the problem:
Take, for instance, a charity that gives a family a cow. The charity might check on the family a year later and say, "Wow! The family is doing so much better with this cow. Cows must be the reason."
But maybe it wasn't the cow that improved the family's life. Maybe it had a bumper crop that year or property values went up in the neighborhood.
a) What problem, introduced in Chapter 2, is being described in the passage quoted above?
Next, the NPR story explains Karlan's solution:
So [Karlan] and a bunch of his colleagues had a radical idea: Test aid with the same method doctors use to test drugs (that is, randomized control trials).
The idea is quite simple. Give some families aid but others nothing. Then follow both groups, and see if the aid actually made a difference in the long run.
An anti-poverty program in Bangladesh, called BRAC, looked like it was successful. It seemed to help nearly 400,000 families who were living off less than $1.25 each day. So Karlan and his colleagues wanted to test the program and see if it could work in other countries.
They teamed up with a network of researchers and nonprofits in six developing countries. They went to thousands of communities and found the poorest families.
Then they divided the families into two groups. They gave half the families nothing. And the other half a whole smorgasbord of aid for one to two years.
The focus of the aid was a gift of livestock, which could be used to make money. It also included home visits to provide training for the livestock, a savings account, mental health services, other health services, and food or cash (which prevented them from eating their livestock investment).
What about the results?
The strategy worked pretty well in five of the six countries they tried it in. Families who got the aid started making a little more money, and they had more food to eat.
"We see mental health go up. Happiness go up. We even saw things like female power increase," Karlan says.
But here's what sets this study apart from the rest: Families continued to make a bit more money even a year after the aid stopped.
b) This was an experiment. What was the IV in the study?
c) There were more than one DV in this study. What were at least two of the DVs?
Here's another comment about the study:
The effect of the aid was actually quite small, she says. Families' incomes and food consumption together went up by only a small amount — about 5 percent, on average, when compared with the control group.
d) What's the above comment addressing? Which validity?
Here are some comments by scientists about the outcome of the study, and its importance.
As you read this, reflect on some of the lessons from Chapter 14 about importance, generalizability, and importance:
"Moving poverty is hard," Baird says. "The fact that they [Karlan and colleagues] were able to move it, and it was sustainable after a year, I think is important."
The findings are a leap forward, she says, because it shows charities and governments a basic strategy that often works.
And even a little bit of extra money can make a huge difference in these peoples' lives, she says. It can help them send their kids to school. Or even just give them a little more hope.
What do you think about the ethics of this approach? Is it fair to withhold aid from families like this?
The main author described it like this:
One issue is that some families go home empty-handed, with no aid. So the idea seems unethical. But Karlan disagrees. "The whole point of this is to help more people," he says. "If we find out what works and what doesn't, in five years we can have a much bigger impact."
The above statement reminds us that one foundation of ethics--beneficence--estimates two things. First, it estimates the costs or benefits to a study's participants. But second, it estimates the costs or benefits to our society from the knowledge in the study. Karlan is emphasizing the benefit of this scientific knowledge in his comment.
Suggested answers
a) This is about confounds. When we observe experiences, lots of things change at the same time. In this case, we can't be sure that it's the charity that caused the improvement. Another name for this problem is internal validity.
b) The IV was receiving the package of aid, or not. The NPR story doesn't mention if the participants were randomly assigned to the aid group or control group. But you could look up the answer in the original journal article, linked here in the journal Science, to find out if they were assigned at random or not.
c) One DV appears to be family income. Other DVs apparently include family food input, female power, and mental health and happiness.
d) Statistical validity--specifically, effect size.
Many thanks to my student, Madeline Halkowski, for telling me about this story!
