Home > Uncategorized > Beyond Poverty Reduction “Inputs” to Outcomes and Impact

Beyond Poverty Reduction “Inputs” to Outcomes and Impact

Fonkoze is unique among microfinance institutions (MFIs) in terms of the human and financial resources it devotes to evaluating the effectiveness of its programs, and then using those findings to improve services to Haiti’s poor.  Its annual social performance report is a model for other institutions.  While Fonkoze’s approach goes beyond most MFIs, it is part of a broader trend of measuring outputs and impact, rather than simply inputs (such as loans disbursed in microfinance, or number of schools built as a measure of promoting better education), when tracking the progress of poverty alleviation efforts.

This is a healthy development.  Academics are making significant contributions to this new trend.  One of the leaders of this in the research community is Dean Karlan, who works hard to make information about impact understandable and actionable for the general public who donate generously to poverty alleviation efforts worldwide.  His new book Beyond Good Intentions, co-authored with Jacob Appel, helps accelerate this trend.

Karlan is a serious thinker who is committed to effectively combating poverty, but who has a tendency to come a bit unhinged when speaking and writing about microfinance (even though he makes some excellent points about even that issue).  His strengths have prompted Grameen Foundation, which I lead, to engage him and his organization Innovations for Poverty Action to conduct research on our work also serve as an advisor.  We are a better organization as a result of his engagement with us, and our social performance management center is taking his ideas and others and applying them across the microfinance industry and beyond. (I have written about the half of the book dealing with microfinance elsewhere.)

What did I like about the book?  Let me first describe its structure.  Each chapter takes on an important subject related to poverty reduction and starts with a thought-provoking anecdote (usually involving Karlan’s co-author and someone he met in Africa or Asia), poses some interesting questions, describes a randomized control trial (RCT) analysis to test them, and then shares the results – regardless of whether they are definitive or not, or whether they agree with past studies or the authors preconceptions or not.  All of this is done in clear prose.

In other words, the authors take us through a journey of learning about what actually works in poverty alleviation, at least in certain contexts, and for the most part they include caveats about how much individual findings can be generalized and what remains unanswered.

The book has some remarkable strengths and findings.  It very effectively emphasizes the power of behavioral economics in crafting potent anti-poverty programs.  It details, among other things, how people (whether rich or poor) often do not act rationally, even though many poverty programs are built on assumptions that they are.  In one case, advertisements for consumer loans worked better when prospective clients were offered one option rather than four, leading to the surprising conclusion that “…presenting more options actually drove away customers [which] directly opposed standard economic theory…”  This may not be news to those in marketing in the business world, but it certainly is to many who design poverty reduction programs.

The authors describe research that puts to rest the long-standing debate about whether it is more cost-effective to give away insecticide-treated bednets compared to subsidizing their cost.  (The idea that if people get bednets for free they don’t value and use them as much as if they pay for them turns out not to be correct.  However, charging for it does decrease uptake significantly since many people choose not to buy it even at a subsidized rate, but will accept one if it is donated.)

Another important observation is related to research technique.  The authors explain that people are unlikely to reveal “sensitive truths” (related to, say, their sex lives or their incomes) if asked directly.  (In fact, in my experience, they are likely to lie.)  However, there is usually a way to get this information.  “The trick,” the authors write, “is realizing that people are willing to reveal sensitive truths as long as they can hide them in a cloud of mundane ones.”  They go on to explain how they have done this, and in so doing raise the confidence level of the reader in the research presented throughout the book.  As someone who has spent more than two decades interviewing microfinance clients and other poor people, from both my mistakes and learnings I can confirm this simple but powerful observation.

In another case, they studied the power of making minor modifications to delivering or marketing a service.  For example, tax filers in the U.S. who were offered a match on their IRA contributions took up the offer at almost double the rate as those who had the same opportunity but had it presented to them as a rebate.  In a world where the poor often befuddle development planners by not taking up opportunities designed to help them, this is a profound insight.  (Though, equally as important, in another case they cite research showing that African farmers decision not to buy and use fertilizer recommended by their government is entirely rational and in their best interests.)

They found business training for microentrepreneurs, much maligned for years but now experiencing something of a comeback, can be effective, especially when “rule of thumb” training techniques are used instead of standard accounting training – an idea being championed now by the organization ideas42.

The authors cite other research showing that the timing of providing a coupon to subsidize fertilizer purchase, or the location of a dispenser to put purifying chlorine drops into drinking water that village women collected, significantly improved uptake.  Finally, in their effort to find things that improve educational performance, the authors sift through the available research and find that by far most cost-effective way to improve learning (at least in western Kenya) was to make deworming pills available to students for free, though other approaches appeared to work well even though their costs were higher.

In another study, simple reminders sent by SMS or snail mail to encourage people to save were remarkably cost-effective: the incurred very little expense by the financial institution, while causing clients to save more, met savings goals more frequently, and in the process grow the institution’s deposit base.  And in a finding that could have profound implications for MFIs seeking to customize products cost-effectively, simple tests of cognitive ability proved to be “a strong predictor for high business returns.”  And in the final chapter the authors helpfully make the case that the percentage of a non-profit organization’s funds spent on administrative costs is “a really bad metric”.

Accountability for results in international development is still more talked about than practiced, but it is likely here to stay.  That is a good thing.  More Than Good Intentions is an important contribution to this process.  Its ideas are most powerful when they are used and informed by practitioners who are themselves deeply curious about their own (and others) outcomes and impact so that they can rapidly move from things that do not work to things that do, and from things that work reasonably well to things that are pro-poor “home runs”.  Fonkoze can proudly stake a claim to being one of those organizations.

Categories: Uncategorized
  1. August 22, 2012 at 12:00 pm

    I got this message from Anne Hastings of Fonkoze about this blog post and she agreed to let me put it in the comments section of this blog:

    Thanks so much for this review of Dean Karlan’s book and for showing the link between Fonkoze’s measurement of outcomes and Dean’s philosophy of looking beyond good intentions to actual outcomes. I had been aware of Dean’s book, but your review definitely made me want to read it. He covers such a broad array of interventions – and I know he is a very clear thinker and therefore clear writer.

    Dean and Fonkoze have been working together over the past year to identify financing for us to do a comparison of the outcomes of cash transfer programs as compared to those of our CLM graduation program. We have the proposal written (Dean’s colleague Nathanael took the lead in writing it), but we haven’t yet found the funding.

    Dean and IPA have been leaders in evaluating the graduation program with the support of the Ford Foundation. And to this point, they have found the positive outcomes to be measurable and definitive. At our recent global graduation meeting in Paris, Dean was there and together with Oriana Bandiera (London School of Economics), Shamika Ravi (India School of Business), Esther Dufeo (MIT) and Karishma Huda (Development Pathways) convinced the audience of practioners, theorists of social protection, and NGOs involved in livelihood development that, although context does matter and can interfere with results, on the whole, there is very strong evidence to suggest that graduation programs do lead to many positive outcomes in the lives of the beneficiaries including: (1) happiness where there used to be hopelessness (2) food security (3) assets (4) savings.

    We are hopeful that someone will see the benefits of financing a study to document the benefits of the CLM-style graduation program as compared to those of cash transfers with or without conditions. We look forward to Dean and his excellent team of researchers leading that effort.

    In the meantime, I’ll definitely read his book!

  2. August 22, 2012 at 1:31 pm

    This comment is from Meredith Kimbell, which is extracted from a message she sent to me this morning related to my post:

    Congrats on your continuing investment in your blogs. You’re finding substantive materials and important issues and writing well. I thought to offer another very different perspective on measuring outcomes that I found powerful. There is a book by Atul Gawande called “Better: A Surgeon’s Notes on Performance.” In it he wrote a chapter called “The Bell Curve” It is fairly short but oh so powerful.

    If you could find a used copy on Amazon and read through it, I think it would enrich your thinking. All the best and thanks for continuing to make awesome contributions in the world,

  1. October 29, 2012 at 12:44 pm
  2. October 30, 2012 at 2:23 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: