So – here’s the dream – figure out how to reduce poverty, scientifically. Find ways to do this that you can show, with numbers, actually work. What a seductive idea. And how slippery and evasive. Let’s look at one of the flagship projects, the Millennium Villages.
They are the poster child of Jeffrey Sachs of the Earth Institute at Columbia, along with UNDP. The idea is to take Sach’s theories of poverty traps (if you need to brush up on this stuff it’s all in ‘The End of Poverty’ by Sachs), and design a series of pilot interventions that demonstrate, empirically, that an integrated approach to rural development with consistent adequate funding over time can achieve the goal of lifting communities out of the poverty trap (and meet the Millennium Development Goals while we’re at it).
There are 13 of the sites, in 10 countries in sub-Saharan Africa, plus a bunch of similar type projects all over the place. They started as a five year project in 2006 and just got renewed for another five. So – where are we as far as empirical proof of poverty reduction is concerned? Well, it’s, erm, a mixed bag, and sort of depends on how you look at it…
ODI did one of the earliest reviews of it in 2008 (read the full pdf here) – a little early to really show impact on poverty reduction, they do claim crop productivity increases (although only alongside additional agriculture inputs, so there isn’t really a lot of income increase). There does seem to be some malaria reduction, and that’s good, although it’s not entirely clear how cost effective this public health program is compared to traditional ones, since the whole thing is a big integrated package.
ODI signs out with some rumblings about sustainability of gains and issues of scaling up, but they are basically optimistic. Not so the Soils, Food and Healthy Communities group, who claim that their Ekwendini project villages that use sustainable agricultural techniques and crop diversification achieve better agricultural gains with far less cost.
Fast forward to this week, when we get the first independent evaluation of the MVPs, by Kenyan economist Bernadette Wanjala of Tillburg University. She looked at 236 families in the project and 175 control families in the same districts (read more here). She was hoping to find that the MVP project (which spends the equivalent of 100% of local per capita income per year on each beneficiary) would be able to show some empirical increase in recipient income. Unfortunately, she couldn’t find any evidence for it.
How can that possibly be? Well, let’s dig in a little more. She did find a 70% increase in agricultural productivity, which did tend to increase agricultural income in those families (at a substantial input cost). The problem was that those families that spent more time farming seem to be spending less time doing other profitable activities, and their income in those other areas fell, canceling out benefits.
The MVP group claims that ‘incomes are rising’, and that the project is ‘enormously successful’, but unfortunately doesn’t publish their data on incomes.
So where does this leave us? The MVPs are a bold attempt to test theories of change on a village level, and five years on we’re still arguing about what the data mean. It’s encouraging to see donors and academics entering this field of research, and perhaps not surprising that the answer is “it’s complicated – we need more research”.
Filed under: Development theory, Environment, If you only have 15 mins, measuring results
Started in 1981, the Farming Systems Trial (FST) at Rodale Institute is America’s longest running, side-by-side comparison of organic and conventional agriculture. The results of their 30 year trial were published this week (get the full pdf here).
The short story is that they claim that, on real farm size trials, over 30 years:
- Organic yields match conventional yields.
- Organic outperforms conventional in years of drought.
- Organic farming systems build rather than deplete soil organic matter, making it a more sustainable system.
- Organic farming uses 45% less energy and is more efficient.
- Conventional systems produce 40% more greenhouse gases.
- Organic farming systems are more profitable than conventional.
Filed under: measuring results
Esther Duflo, founder and director of the Jameel Poverty Action Lab (J-PAL) at MIT delivers a thought provoking lecture about how randomized control experiments, a cornerstone of J-PAL’s work, can prove that some development programs are more effective than others. Randomized control experiments have been popularly used to determine the effectiveness of health care options by holding a control group constant and then applying the treatment (or program) to a different group, afterwards comparing the results. Duflo believes that this methodology can also be applied to studying the effectiveness of aid programs. She specifically addresses 3 big development questions: how to immunize more children, how much to charge for bed nets, and how to get more children to stay in school longer. Think you know the answers? Listen to her perspective here. Interested in learning more about J-PAL and their methodology for evaluating program effectiveness? Click here.
Filed under: accountability, If you only have 15 mins, Interesting links, measuring results, organizational learning
For anyone interested in understanding the differences between qualitative and quantitative research methods this website hosted by the Web Center for Social Research Methods. All too often we get into fruitless debates about which approach is better, whereas both are equally valid approaches and more important is to determine which approach is better for the research you are looking at conducting. The bite size chunks and easy navigation make this is a useful site to peruse and then bookmark. Highly recommended!
This 6-page report by McKinsey has some excellent examples of how some large non-profits have got round the challenges of how to measure progress towards fulfilling their mission. You do have to sign up to read the full article, but it only takes a couple of seconds and is well worth it. The article takes the Nature Conservancy as an example of an agency that had two macro-level indicators (revenue generated and increased number of protected habitats) until they realised that neither of these indicators were telling them whether they were making progress towards the real heart of their mission which was increasing biodiversity. The authors claim there are three critical performance metrics needed by an organization:- success in raising resources, staff effectiveness and progress toward its mission fulfillment, with the last being the hardest to measure. They outline three ways of mission measurement: 1) through narrow definitions of success, through research, or through proxy indicators. They draw examples from several organizations which is useful and thought provoking.
If you only read one thing this week... 