Interventions & Institutions

.

The most recent draft of our Kenyan RCT results on scaling up proven education interventions (with Tessa Bold, Mwangi Kimenyi, Germano Mwabu, and Alice Ng’ang’a) is available here.

.

Gabriel Demombynes wrote a nice summary of the paper over at the World Bank’s Development Impact blog.  For a related discussion, see Roving Bandit’s post as well.

.

Lawrence MacDonald has posted a CGD podcast where we discussed this and another paper, and Kevin Grier has a nice discussion of the political economy underbelly of the paper over at Kids Prefer Cheese.

.

Fake Aid

.

Joint with Charles Kenny

.

Winning hearts and minds is a key part of the US Military’s counterinsurgency strategy in  Afghanistan, and a major rationale for USAID’s $15 billion investment in the country.  This strategy rests on Secretary Clinton’s visionthat defense, development and diplomacy are closely linked, mutually reinforcing goals — a win-win-win foreign policy love triangle.

.

Some development experts, channeling their inner Dr. Phil, have been skeptical of this model.  But much of the industry has been won over by the lure of Pentagon-sized budgets for real aid projects serving real development goals like rural development and girls’ education.

.

Meanwhile, from a defense and diplomacy perspective, the Kashmir earthquake in Pakistan in 2005 has provided one of the strongest pieces of empirical support for the idea that aid can in fact work to win goodwill.  A World Bank working paper by Jishnu Das and Tahir Andrabi suggested that “four years after the earthquake, humanitarian assistance by foreigners and foreign organizations has left a lasting imprint on population attitudes.”  Das and Andrabi argue this strong impact was based in part on the near unanimous sentiment by local people that aid workers were there for humanitarian reasons rather than to promote hidden agendas.

.

Which is why a new book by Marc Ambinder and D.B. Grady might make particularly interesting reading for residents of North-Eastern Pakistan.  They report that the CIA used the Kashmir Earthquake to send intelligence officers into the country.  “Using valid U.S. passports and posing as construction and aid workers, dozens of Central Intelligence Agency (CIA) operatives and contractors flooded in without the requisite background checks from the country’s Inter-Services Intelligence (ISI) agency.”

.

We’re in no position to judge the intelligence payoff to the agent insertion in Pakistan.  Again, perhaps the fake vaccination campaign orchestrated by the CIA in Abbottabad with the help of Pakistani doctor Shakeel Afridi (now nominated for a Congressional Medal of Honor) brought in vital information.  But two conclusions do appear reasonably safe.

.

First, fake aid and fake aid workers are worse than nothing from a development perspective.  Rumors that vaccination drives were actually a Western plot have been rife before –and distrust of aid workers as agents of neo-imperialism is as old as aid itself.  It is a lot harder to run a successful development project without any trust.  And, of course, running an aid project when it is assumed you are a spy can be dangerous.

.

Second, fake aid is a risky strategy from a military and diplomatic perspective as well, because it can undermine the goal of winning hearts and minds.   While beneficiary goodwill helps development projects run smoothly, building that goodwill is the whole point of aid as a weapons system or a diplomatic tool.  Deceiving beneficiaries undermines trust and thus the hearts and minds mission –at least if they find out about the deception.

.

So it is great if the military, diplomats and aid workers can work together to bring stability and prosperity to fragile states.  But using aid as a cover for intelligence operations is often going to end badly for all three partners in the defense, development and diplomacy triangle.  And much like traditional love triangles, that’s particularly true if those involved can’t keep a secret.

.

The Epistle of Gates and the Gospel of Agricultural Innovation

.

Bill Easterly stirred up the blogosphere earlier this week with a call for USAID to get out of the national security business, declare that “aid is for poverty relief and only for poverty relief,” and “build a firewall between USAid and the defence department.”

.

Nancy Birdsall replied on this blog that Easterly is too quick to jettison Pakistan.  But in CGD’s spirit of open debate, I want to argue that Easterly’s critique is much harder to dismiss in Afghanistan.

.

A few days ago Bill Gates released his annual letter to the world, opening with a discussion of how Gates-funded agricultural research can help Tanzanian farmers.  Coincidentally, before coming to CGD I did some agricultural research in Tanzania myself — funded in part by the Bill and Melinda Gates Foundation — so I was eager to compare notes.  Gates’ letter is so optimistic about agricultural innovation lifting Tanzanian farmers out of poverty, it feels almost impolite to point out that the main source of poverty reduction for Tanzania farmers in the past two decades has been essentially the opposite: leaving the farm.

.

.

Gates opens his letter by introducing us to Christina Mwinjipe — a small-scale farmer in Tanzania whose cassava crop is threatened by something called brown streak disease.   Sadly, cassava is not a big priority for commercial agricultural research.    Enter Dr. Joseph Ndunguru, a plant scientist in Dar es Salaam who is developing disease-resistant cassava.

.

Ndunguru embodies Gates’ core idea that agricultural innovation can help small-scale farmers in situ.  But this vision is at odds with the recent historical record of poverty reduction for Tanzanian farmers.  To build up that historical record, I and my co-authors (Razack Lokina at the Univeristy of Dar es Salaam and Måns Nerman at Gothenburg University)  pieced together all available survey data – including recent Gates-funded data – on Tanzanian small-scale farm productivity.  Here’s what we found:

.

• First, innovation is not very popular with Tanzanian farmers.  The share of farmers using any modern farming technologies — fertilizer, irrigation, improved seed, herbicide, pesticide, mechanization, etc. – is extremely low and stagnating across the board.

.

• Second, income levels have been stagnant within agriculture.  The same is true in the informal, non-farm sector.  But the latter offers much higher income levels — implying that farmers such as Christina Mwinjipe could achieve huge income gains just by leaving the farm.

.

• Slowly but surely, this is exactly what has happened over the past couple decades.  Tanzanians are responding to market signals and moving out of agriculture — primarily into self-employment in informal micro-enterprises (see the first chart below).

.

Putting all these pieces together, the biggest driver of household consumption growth in Tanzania from 1991 to 2007 was not agricultural innovation, productivity growth, or anything to do with agriculture whatsoever.  Over 40% of all consumption growth came from people getting out of farming (see the second chart below).

.

There’s a risk here of creating a false choice between innovation to boost productivity and policies to promote movement out of agriculture.  The idea that farm productivity growth is a prerequisite for industrial growth has a long and prestigious pedigree in development economics.  (Alas for this theory, in Tanzania agricultural productivity has stalled, yet people keep leaving.)

.

.

Am I suggesting that the Gates Foundation scale back its focus on agriculture innovation?  Not necessarily.

.

The Gates Foundation’s work on agricultural innovation is an unmitigated good thing, and the topic is well-suited to the talents and proclivities of a private philanthropic foundation.   It’s a great, relatively apolitical niche for a private donor to fill, leaving bigger and more contentious issues to public actors with greater political legitimacy.

.

Ultimately, I have two concerns with this year’s letter. The first mostly concerns rhetoric. I cringed at the reference in the middle of the letter to the “population bomb” and global food supply shortages. This strikes me as fear mongering that distracts from the main issue.

.

More substantively, the letter seems to assume that once Dr. Ndunguru finds a brown-streak resistant cassava strain, Christina Mwinjipe will spontaneously start using it.  Low adoption of seemingly profitable technologies is a huge obstacle. We need more attention and hard thinking about solutions on that front.

.

The most important message is for other donors.  Let Gates be Gates.  But there is a limit to how far the lens of innovation can guide development policy.  Not all social ills are susceptible to a technological fix.  Science and innovation are cool and politically easy, while economic reform and political compromise are messy and tedious.  But governments and donors must confront the messy issues that stand in the way of major structural transformation in economic backwaters like Tanzania.  Things like enabling large-scale foreign investment in agriculture without trampling farmers’ rights, or opening borders to easier trade within East Africa.

.

It’s still true that the majority of Tanzanian farmers are poor, and a majority of poor people in Tanzania are in farm households.  But it doesn’t necessarily follow that the solution to their poverty is in a new seed, or even on the farm at all.

.

Aid, WHAM, and Afghanistan

Bill Easterly stirred up the blogosphere earlier this week with a call for USAID to get out of the national security business, declare that “aid is for poverty relief and only for poverty relief,” and “build a firewall between USAid and the defence department.”

Nancy Birdsall replied on this blog that Easterly is too quick to jettison Pakistan.  But in CGD’s spirit of open debate, I want to argue that Easterly’s critique is much harder to dismiss in Afghanistan.

The rhetoric of aid to Afghanistan is about “winning hearts and minds,” aka WHAM.  Clearly WHAM is not development.  But the question remains, within a national security context, does the model work?

Can aid WHAM?

There is a growing body of evidence that giving people considerable sums of money and goods does in fact lead to gratitude – either for the government or the foreign donor.  (Shocking, I know.  But there are RCTs to back me up.)

Last week the Asia Foundation released findings from its annual “Survey of the Afghan People”, which found that sympathy for the motives of the insurgency plummeted from 56% in 2009 to 29% in 2011.  And there is solid evidence that aid deserves at least a sliver of credit for this victory in the war for public opinion.

Yesterday, a team from MIT and the World Bank released a new working paper summarizing results from a large-scale randomized evaluation of the Afghan government’s National Solidarity Program (NSP) – one of the largest and most highly praised conduits for aid monies.  The results are positive, if not exactly value for money: for a cost of roughly $200 per household the NSP raised the proportion of villagers who reported improvements in their economic conditions by 5%, and raised the share who felt government officials act for the benefit of all villagers by 4 to 6% across virtually all levels of government.  This is not an entirely isolated finding: World Bank research on aid relief to Pakistan in the wake of the 2005 earthquake also found a significant impact on public opinion, in that case benefiting foreign donors.

Is WHAM the point?

Note what the researchers did not find in Afghanistan’s NSP villages: any reduction in violence.  A massive influx of aid left people with positive feelings about the government.  But the rate of security incidents in and around the village was unchanged.  Indeed, a recent study out of UC Berkeley examining a similar World Bank aid program in the Philippines found precisely the opposite: aid projects may be a magnet for insurgents.  Municipalities just within the eligibility threshold for the Philippines’ flagship anti-poverty grant program suffered significantly higher casualties in the country’s longstanding civil conflict than municipalities just outside the cutoff for eligibility.

If aid is so bad at quelling insurgency, why does the U.S. military continue expand its aid model of “Money as a Weapons System”?  This initiative, known as the Commanders’ Emergency Response Program (CERP), doles out cash for small, relatively ad hoc reconstruction and humanitarian projects that have been derided as “quick impact, quick collapse”.  In 2010, CERP spending reached $1 billion in Afghanistan, roughly a quarter of all non-security related US aid to the country.

I know of no study that attempts to rigorously measure the development impact of CERP spending.  But that’s the point.  CERP is about military tactics, not development impact.  And on this front, a much touted study just published in the Journal of Political Economy (ungated version here) provides some evidence that CERP works.  Districts receiving more CERP funds in Iraq saw a significant decline in insurgent attacks, controlling for a host of other factors.  Notably, the authors conducted the same analysis using non-CERP reconstruction spending and found no effect, suggesting these small-scale, “quick impact” projects may be well-suited to the military’s needs.

So, returning to Easterly’s challenge to USAID: If WHAM seems to be missing the point, and the Pentagon is refining its own models of aid for its own goals, why shouldn’t USAID re-focus its energies on its core mission of development, and leave the war to the experts?

Over the next several months, my colleagues and I at CGD will be digging deeper into these issues and examining the effectiveness of U.S. assistance to Afghanistan — splicing and dicing data on USAID and Pentagon projects, socio-economic conditions, and the violence itself.   There’s some hard thinking about aid in conflict zones going on within the agency (see here) and those new initiatives deserve a closer look.  We’d love to have your ideas and comments on what you think the open empirical questions are in this ongoing debate.

Worms Held Hostage

I was dismayed to learn recently that school-based deworming in Kenya — one of the most celebrated and cost-effective successes in global development in 2009 — was not repeated in 2010.   The story of how that happened offers an object lesson in the gritty difficulties of translating evidence into policy.

.

Intestinal helminths, aka worms, infect roughly a fourth of the world population and contribute to anemia, malnutrition, and other illnesses — disproportionately affecting school-aged children.  Fortunately, simple, cheap, drugs have been shown to be highly effective against many types of worm infections.

.

Based in part on research by Berkeley’s Ted Miguel and Harvard Professor (and CGD non-resident fellow) Michael Kremer, this proven small-scale intervention had been scaled up to the national level.   But after just one year of nationwide implementation, funding for the deworming program was recently suspended.  So what gives?  The program’s suspension appears to have been unintentional collateral damage in a larger budget battle, despite broad support within the Kenyan government and donor community.  (Thus this is hopefully a temporary suspension.)  This saga raises concerns about whether donors and governments are committed to scaling up policies that have been proven to work in small-scale NGO pilots.  Depending on your perspective, however, it also raises some interesting questions about (a) the feasibility of turning small NGO pilots into manageable national policies, and/or (b)the unintended consequences of donors being tough on corruption.

.

The “Worms” model of policy change

.

School-based deworming is an initiative with almost iconic significance for many people in the development industry.  One of the first randomized impact evaluations in development economics, Miguel and Kremer’s 2004 study, “Worms“, showed that deworming had even greater public-health benefits than previously thought due to positive externalities on untreated children, and was as an extremely low-cost way to boost school enrolment in Western Kenya.

.

On the basis of these results, an international campaign emerged to champion the benefits of free deworming and provide technical assistance to governments wanting to implement school-based deworming programs.  Within Kenya, the education and health ministries formed a joint team to develop a plan for implementing deworming in all the regions of the country where worms were a major health problem. This was incorporated into the national education plan that was approved by the government and funded by donors and the Kenyan government.  In 2009 the Kenyan government implemented a nationwide school-based deworming program reaching over 3.5 million high-risk children at a total cost of just Sh. 70 million ($836,000).  Since then, a similar process in the Indian states of Andhra Pradesh and Bihar has led to the treatment of many millions more children.

.

For the new wave of ‘randomistas’ in development economics, deworming became a template for how evidence-based policymaking should work: a small-scale project subject to gold-standard evaluation contributed to an evidence base, and — combined with a concerted effort to disseminate the results and provide assistance in adopting and implementing the program — led to a massive scale-up reaching tens of millions of children.

.

What went wrong?

.

Kenya’s deworming program has become a casualty of a deeper dispute between donors and the Kenyan government over financial management within the Ministry of Education.  The celebrated deworming tablets were a tiny slice of a massive, 5-year, aid package for Kenyan education funded by DFID, the World Bank and other donors.  In late 2009, the renewal of this multi-donor aid package hit a major snag when an audit revealed that over a six-year period as much as Sh. 5.5 billion ($63 million) in Ministry funds for free primary education had gone missing  – funds that should have been transferred to schools’ individual bank accounts to pay for supplies.  The Education Minister was sacked, then reinstated; the previous Permanent Secretary was suspended, reinstated, then transferred to another Ministry.  At the end of the day, foreign donors and many domestic pundits felt that the relevant senior officials had dodged accountability. They therefore cut off support for the national education plan — including school-based deworming. . Whatever one thinks of the overall merits of the donors’ decision, Kenyan children suffering from worm infections are collateral damage. Advocates for deworming are seeking replacement funds, but in the short run at least, treatable infections are going untreated.

.

The scandal of Kenya’s missing education funds and the subsequent donor response suggest that — even when randomized trials identify cost-effective interventions in small-scale pilots — other weak links in the chain from Ministry financing to project implementation may hinder successful scale-up of ‘proven’ interventions.  (This issue of institutional and political economy as a constraint to scaling up is something I’ve been exploring in my own research with colleagues Tessa Bold, Mwangi Kimenyi and Germano Mwabu, working with Kenya’s Ministry of Education and National Examination Council to replicate another successful RCT using the Ministry’s own structures.)

.

Learning from others’ mistakes

.

Kenya’s struggles in managing education funds are not unique.   During Uganda’s introduction of centralized funding for “Universal Primary Education” in the 1990s, researchers Ritva Reinikka and Jakob Svensson found that 87% of capitation grants failed to reach individual school accounts.  There’s a reassuring lesson here though: Reinkikka and Svensson subsequently found that a government-sponsored transparency initiative, including newspaper ads and public meetings, dramatically improved the rate at which resources reached schools, with just 19% failing to reach schools by 2001.

.

Clearly, the existing approach to managing education aid flows in Kenya has proved inadequate, and deworming has been an unintended victim.  All the more reason to focus on fixing those weak links in the funding chain, and concentrating on a manageable set of proven tools — such as deworming.

.

In sum, rigorous evaluation (a la Miguel and Kremer’s worms study) and fixing the weak links in the governance chain (a la the transparency reform that Reinikka and Svensson report on from Uganda) are not substitutes; they’re complements.  In some cases, such as the nationwide randomized trial of Mexico’s Progresa program, rigorous evaluation can be conducted at full scale, exposing the project to all the relevant institutional and political constraints from the beginning.  This is ideal when it’s feasible.   But in other cases it makes sense to experiment at a small scale, and allow for more trial and error.  In those cases, scaling up successful pilots requires complementary intellectual efforts toward solving or circumventing the political economy  failures that led to the suspension of deworming.

.

Mexico’s Progressa program and Kenya’s deworming program are also different in another important way, in that the latter was financed with domestic resources and thus not subject to donor shut off.  For countries like Kenya with more limited resources it is not just ‘weak links in the chain’ that are to blame; relations between donors and national governments also need fixing, to ensure that successful programs are not held hostage by unrelated budgetary disputes.

.

New research by Berkeley’s Owen Ozier has shown that the original deworming experiment in Western Kenya had additional, previously unnoticed benefits for the early cognitive development of younger siblings who were spared infection by intestinal helminths during critical stages of development.  And a new follow-up study by Sarah Baird, Joan Hamory, Kremer and Miguel shows surprisingly durable benefits from deworming, measured in terms of adult employment and earnings a decade later.  It seems the more we learn about deworming in Kenya, the better it gets.

.

Ironically, just as state governments in India are learning from the Kenya’s success and scaling up school-based deworming, children in Kenya are caught in the crossfire of a dispute between the government and donors.  If only there were a pill for that.

.

Is the Malawi Maize Miracle a Good Example for Others?

In 2005/06 Malawi bucked the advice of the World Bank, and launched a massive nationwide program to subsidize fertilizer and maize seeds for small farmers.    Over the next two years, maize output doubled and real GDP growth jumped from 0 to 5% per annum.  Malawi was crowned an economic miracle, and copycat programs sprang up in Tanzania, Zambia, Ethiopia, Nigeria and Senegal.

.

I spent most of 2008 and 2009 working for the Tanzanian government to measure, among other things, small farm yields — just as Tanzania’s Ministry of Agriculture was launching its own replica of the Malawian program.  There was a lot of enthusiasm on the government side, lots of skeptical hand-wringing on the donor side, and precious little evidence about whether or not this would work on either side.

.

Unraveling the hype

.

Can the reality possibly live up to the hype surrounding the Malawian experience?  That’s an extremely high bar.  But recent research reveals tantalizing evidence that small farmers systematically under-invest in fertilizer — implying, in theory at least, big economic returns from exactly this kind of subsidy.

.

Matt Collin over at Aid Thoughts hosted a series of guest posts about recent claims, publicized byOxfam and others, that Malawi’s program raised real wages or contributed to economic growth.   There are many reasons to be skeptical.

.

First, claims of huge program impacts are based primarily on nationwide trends, with no credible comparison group or counterfactual.

.

Second, 2004/05 was a dry year, and rainfall improved dramatically in the two subsequent seasons that are used to evaluate the program. At least some of the increased productivity is clearly the result of better weather.

.

Third, even these crude estimates of the program impact are actually inconclusive, with a mostly positive program evaluation citing estimated benefit-cost ratios from 0.76 to 1.36.  That is, the program either led to 36% gains or 24% losses.   A fairly important difference.

.

More detailed research (here and here) examining maize yields in household panel data spanning the onset of the input voucher program has a better shot at establishing credible impacts.  Unfortunately, estimates to date are fairly noisy — jumping from positive to negative effects on output depending on the sub-sample and estimation method.

.

If fertilizer is so great, why do we need to subsidize it?

.

The best rationale for Malawi-style input subsidies is that small-scale farmers have profitable investment opportunities that they fail to exploit.  This logic is hard for economists to swallow.   Economists really only ever tell one joke, but it fits here:

.

An economist and his friend are walking down the street when the friend sees a ten dollar bill on the sidewalk.

.

“Look,” he says, “it’s a ten dollar bill”.

.

“Nonsense,” says the economist. “If that was a ten dollar bill, someone would have picked it up by now.”

.

By this logic, if fertilizer were profitable, farmers would be using it already.  Unless you can point to a clear market failure or some widespread failure of economic rationality, subsidies are just money down the drain.

.

Enter the behavioral economics revolution, which claims such failures of economic ‘rationality’ are in fact quite widespread.  Recent experimental work by Esther Duflo, Michael Kremer and Jonathan Robinson has been undermining economists’ traditional skepticism, and showing that impatience and poor planning may lead farmers to underinvest in farm inputs.  By randomly assigning farmers in Western Kenya to have fertilizer applied freely (and almost forcibly, it seems) on their plots, they reach startling conclusions.  On average, fertilizer is extremely profitable for their sample of small farmers in Kenya.  Yet even after seeing this fact demonstrated on their own plots, farmers still don’t use fertilizer once the free provision stops.

.

Interestingly,  however, a simple “nudge” is all it takes to tilt the scales: when offered the opportunity to invest in fertilizer at harvest time when they have cash on hand, farmers jump at the chance and reap profits in the following season.

.

If these results are right — and replicable in Malawi and elsewhere — the economics underlying input vouchers is much more promising.  The implication is that small farmers are leaving money on the table, so to speak, and vouchers could spur them to make profitable investments.

.

A word of caution

.

The counter-revolution to Duflo et al’s behavioral approach arrived almost simultaneously from another researcher, Tavneet Suri, also at MIT, also working on technology adoption among maize farmers, also in Kenya.  The abstract of her new paper in the journal Econometrica, states quite bluntly:

.

“This paper investigates an empirical puzzle in technology adoption for developing countries: the low adoption rates of technologies like hybrid maize that increase average farm profits dramatically. I offer a simple explanation for this: benefits and costs of technologies are heterogeneous, so that farmers with low net returns do not adopt the technology.”

.

Despite the fact that Suri focuses on seeds and Duflo et al on fertilizer, the results from these two studies sit awkwardly together.   Suri’s results imply that farmers not using hybrid seeds right now simply shouldn’t be: it’s not profitable for them.  And input subsidies would be a very inefficient way of helping such farmers.  Instead we should think of other forms of social protection or targeted transfers for such households (conditional cash transfers?)

.

Stay tuned

.

I’m anxiously awaiting new data from Tanzania’s first maize harvest under the input voucher scheme, which is now slowly trickling in.  Will the numbers validate the Malawian miracle?  If yields are strongly up, it would seem the burden of proof in the subsidy debate might begin to shift.

.

The holy grail will still be an evaluation — in Malawi, Tanzania or elsewhere — showing that input vouchers not only stimulate yields, but stimulate them up to a level of profitability that justifies input subsidies vis-a-vis some other form of transfer or social protection.   But if Tanzania is successful even in meeting its operational targets of delivering valuable commodities to the rural poor in over 50 districts (never mind its ambitious impact goals), the voucher fad would seem  to merit more serious attention from aid donors and policymakers concerned not only with agriculture, but social protection more broadly.    Stay tuned.

.