Home > Uncategorized > The Start of a Movement for “Lean Research”?

The Start of a Movement for “Lean Research”?

As I indicated in my last blog, I joined a stimulating gathering last Friday on the topic of “lean research” convened by the Fletcher School at Tufts and D-Lab at MIT.  The proposed definition of lean research that the group considered and used as a guide was research that is relevant, respectful, rigorous, and right-sized.  (See this blog with helpful definitions of the four proposed criteria for lean research.)

Kim Wilson of Tufts led off the day with a speech that included three examples of what one might call “research run amok.”  The first was a case where she had worked for a humanitarian organization that collected a huge amount of data through repeated surveys of poor people, but that used almost none of that information. Another story involved Fonkoze: she explained how she proposed to put a bunch of Fonkoze clients who had survived the 2010 earthquake through a 100-question survey, and how Anne Hastings (then the CEO of one of the Fonkoze organizations) refused to allow it, saying something characteristically blunt and principled like, “My job is to protect my clients, and your job is to extract [information] from them.”  In retrospect, Kim saw that Anne was right.  Her third major insight prompting this gathering was my blog on the researcher/practitioner interface, which I of course found encouraging.   

As I am not a researcher per se, occasionally the conversation last Friday strayed a bit over my head.  But it was a diverse group including academics, funders, practitioners, advocates and others, and I learned a lot and felt the vibe was constructive and action-oriented (which I liked).  I was asked to give closing comments alongside Amy Smith, the founder of the D-Lab (with which Grameen-Jameel Microfinance Ltd. is working on some exciting action-research in Morocco) and Juan Carlos Rodriguez of USAID’s new Global Development Lab

Below I have reconstructed (and slightly augmented) the core of my remarks, which Kim encouraged me to make as provocative as possible. 

In these closing remarks, I am going to touch on some things that Grameen Foundation and that I can do, and also what I think others should do, related to the discussions we have had today.  I will also mention some ideas I have been pushing for years, and others that popped into my head for the first time today.  You will notice that my ideas will tend to be fairly simple and practical, though I hope not simplistic – but you, rather than I, will need to be the judge of that.

Why Conduct Research?

My bottom-line framing of this entire issue is related to a basic question: why we do this kind of social science research at all?  In my mind, we do it for one reason: in combating societal problems such as poverty, we want to do more of what works well, and less of what doesn’t work, or works less well.  Period.  If research contributes to that, I will call it successful and effective.  If not does not, I will call it a failure – no matter how elegant the research design or how smart the investigators.  This is my take on the design principle of “relevance” which we have been discussing today.

I concur with someone who said that research has become too “elite” in that we mostly pay attention to studies that come out of “the best” universities and that follow particular research designs that are in vogue at the moment, designs that some call “gold standard.”  I think we need to recognize that valid research can be done through a wide variety of methodologies and sometimes by people with limited training in research methods.  The main question in my mind is whether we can learn something useful from research that can help us improve practice or policy.

If we are to deem some research to be the gold standard, that designation should be based on the degree to which the research has led to improved practice or policy, regardless of any other aspect of that research.

Making Research More Accessible

I think it is critical that we demystify research and present it in easier to digest forms, so that those outside the academy, particularly busy practitioners and policy-makers, are more likely to understand and apply the lessons of research.  One thing Grameen Foundation has done on two occasions is to commission an academic to analyze the totality of existing research on microfinance and financial inclusion, and then summarize it in a paper running about 30 pages, written in plain English.  This has allowed non-specialists, practitioners, policy-makers and others to get a sense of what research says generally about this social innovation, rather than relying on a single statistic, perhaps exaggerated or provided without context, by the media or an overzealous advocate or critic.

Grameen Foundation is going to come out with the third in this series 2015.  This gathering reaffirms in my mind the importance of publications like this that make research accessible to decision-makers outside the academy.  I encourage others to do so for other fields and other social innovations, as I believe they can help ensure that quality research translates into improved practice and policy.

Another publication I have been advocating for is one focused on “lessons for practice” for those who are providing financial services to the poor.  It turns out that many recent studies that looked into the question of whether financial services benefit the poor stumbled onto important insights about how those services can work best.  (This insight struck me when reading Dean Karlan and Jacob Appel’s book More Than Good Intentions, which I reviewed on the Center for Financial Inclusion’s excellent blog.)  So I have been pleading with the research community to boil those insights down into a single publication that could be read by senior executives and board members of financial service providers to the poor and hopefully influence their service delivery approaches.  Finally, one researcher, Tim Ogden of NYU, saw the need and agreed in principle to write it.  Still, it remains in limbo for lack of funding – but I am even more committed to getting it done after our meeting here today.

In general, something seems missing in the ecosystem that comprises researchers, practitioners and policy-makers.  That “something” would be a person or organization or loose network that saw it as their job to ensure that research makes a tangible, positive difference in the real world.  I feel compelled to fill the vacuum in a few cases, and I am sure others are doing their parts.  But I think there needs to be a systemic solution.  I am not sure what it is exactly.  I welcome your ideas.

A Simple Poverty Index

One other insight from today is about the power of a tool that we have helped develop, called the Progress out of Poverty Index®, which has been referenced a few times today.  It is a survey tool built to be very lightweight in terms of the demands it places on those being surveyed, and those conducting the surveys.  It invariably has ten questions and can be conducted in less than ten minutes.  Quite advanced statistics and mathematics are “underneath the hood” of this tool thanks to our talented partner Mark Schreiner.  In fact, it is that sophistication which allows for its simplicity in terms of how those who collect and contribute data interact with it.

More tools like this, which are carefully built for robustness and simplicity, are clearly needed.  At the same time, we need to leverage existing tools like the PPI® rather than reinvent the wheel, over and over, in terms of survey tool development.  Certainly, today’s discussions have increased my commitment to not add more questions to the PPI, as advocated by some.

Four Ideas for Making Research More Relevant

Let me close with four ideas, three that occurred to me just today.

There was talk about giving voice to those being surveyed and studied.  I am not sure that involving them in research design and so forth is practical, but I was reminded of something from my years working with Grameen Bank in Bangladesh.  Professor Yunus invited many, many researchers in to study Grameen, perhaps viewing their work as a way of augmenting its own monitoring and evaluation capabilities, at little additional cost.  But he always insisted that the researchers agree to publish, as part of their final paper, any rebuttal or comment that Grameen wanted to contribute.  To my recollection, Grameen never exercised that right.  But perhaps it kept the researchers honest from exaggerating their claims or not providing balance and context.

Today, I was wondering if it would be a good practice in published research to not only give organizations being researched the right to do that, but even to insist on it.  Let the reader know how the researched institution felt about the process of being studied.  And why not take this one step further?  What if, as a good practice in respectful research, at the conclusion of the process a representative group of “human subjects” would be assembled and explained what the study found, in language they could understand, and then be asked for their comments on the results and the process.  And then their comments would be printed verbatim as an appendix to the study itself.  How might this change some of the power dynamics involved in research?

Another idea, which I have proposed before, is to hold researchers to the standard that they rightly hold social innovations and programs to: measure whether they have positive impact, negative impact, or no impact at all.  One would assume (or at least hope) that a social program or sector or innovation that had been studied by researchers would be more likely to perform better post-research than pre-research, or compared to programs/sectors/innovations that had not been studied.  (If not, why conduct the research?)

Well, if this is our hypothesis, there must be some way to measure whether it holds true in specific cases where research has been conducted.  That way we could know what is the impact of impact research.  My hope is that it would motivate researchers to spend more time ensuring that their research was being digested and applied before setting off to conduct the next study.

My third idea is to have some credible authority commit to publishing an annual list of the top ten examples of research positively influencing practice and/or policy.  Perhaps there could be a cash prize for the top example.  Maybe a competing ranking system could spring up if people thought the original list was in any way flawed.  My hope is that this could spark new dialogue about how to make research more practical and impactful, by drawing attention and realigning incentives.

Finally, I would like to suggest that this group develop a concise and, to the extent necessary, provocative “declaration” coming out of this meeting.  It would state how we feel research should be recast to be leaner, which is to say strike a better balance and application of the principles of being relevant, respectful, rigorous, and right-sized.

Thanks for considering my ideas and thanks especially to Kim Wilson, Kendra Leith and Elizabeth Hoffecker Moreno and their colleagues at Tufts and MIT for their leadership in pulling this important gathering together.

(Update: I strongly suggest people take a look at Kim’s just-published blog on lean research and the August 1 gathering.)

Categories: Uncategorized
  1. August 6, 2014 at 1:26 pm

    Dear Alex –

    Thank you for this detailed and colorful summary of the day. Your idea on having subjects comment on the research findings is a good one. There may be many reasons why the research findings don’t match with subject views yet 1) this itself would be an interesting finding 2) it gives researchers a chance to be relevant to the subjects and it’s the respectful thing to do. I am not sure publishers will be instantly motivated to do this. It may take some time. Likely, it would be a “lean research” author who submitted with a Response Addendum. Ideally, it would be a different research team who pulled this part of the paper together, perhaps a local NGO.

    Thanks for all you time and help in making the day a success.

    • August 7, 2014 at 11:19 am

      Kim, thanks for your comment and even more for your leadership on this issue. I certainly sense that some of the ideas I put forth are not going to be adapted wholesale quickly, but rather by a few researchers who share the vision and start a slow process of industry-wide change. I am not sure that it would need to be a different research team, but I can see advantages of that. Bringing in a third party could make the process more complex and cumbersome, so for now I am going to be open to various approaches if anyone takes this idea up as part of this emerging movement.

  2. August 6, 2014 at 3:05 pm

    Sounds like it was a very interesting and fruitful discussion indeed, I´m sorry I missed it. I really like the idea of systematically allowing for commentary or rebuttal from researched organizations and subjects. It´s a very specific, measurable and actionable objective for this budding movement.

    • August 7, 2014 at 11:16 am

      Ignacio, thanks for your comment and feedback. I am not sure how practical my idea is — it’s easy to throw things out as a non-researcher — but it does make intuitive sense to me and I appreciate your feeling that it is specific and actionable. I encourage you to get involved with this group that Kim and others have assembled. I think it is going to lead to some exciting and tangible outcomes over time. Your voice and ideas are clearly needed. -Alex

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: