Revisiting the evidence on impact

by Jeanette Thomas : Thursday, July 2, 2009

For many years we’ve been bemoaning the paucity of impact studies in our field. But now comes news that even those few studies that did exist may have overestimated the good that microfinance does (at least what’s measurable).

The Center for Global Development’s David Roodman and NYU economist Jonathan Morduch have just published a working paper  that revisits impact data based on household surveys  from Bangladesh in the 1990s. And it makes pretty sobering reading.

It’s a heavy-going piece for those of us who aren’t economists. But the conclusion is clear: “30 years into the microfinance movement we have little solid evidence that it improves the lives of clients in measurable ways.”

It’s a heavy-going piece for those of us who aren’t economists. But the conclusion is clear: “30 years into the microfinance movement we have little solid evidence that it improves the lives of clients in measurable ways.”

It’s a heavy-going piece for those of us who aren’t economists. But the conclusion is clear: “30 years into the microfinance movement we have little solid evidence that it improves the lives of clients in measurable ways.”

Roodman and Morduch draw the opposite results from the original 1998 Pitt and Khandker  finding which spawned the widely-quoted claim that microcredit in Bangladesh lifts 5 percent of its borrowers out of poverty each year. And yet they still don’t say microcredit harms.

So does it matter? With CGAP estimating total public and private money invested in microfinance today at around $11 billion (portfolio outstanding as of December 2008), and still growing despite the financial crisis, it seems obvious. What other opportunities for improving lives might be missed, if it turns out microfinance isn’t doing half the good we think it is?

Of course the lack of robust evidence that microfinance results in increased household expenditure doesn’t mean microfinance does no good. It just means there’s not much evidence (–yet?). But the paper sure reinforces the case for why we need some hard evidence on what microfinance does–and doesn’t–do.

The good news is that a series of studies based on randomized control trials now beginning to trickle out from the Innovations for Poverty Action may help fill in some of the gaps.

TAGS: , , , , , , ,

Comments: Comments and trackbacks are open.

7 Comments RSS 2.0

  1. July 3rd, 2009 at 1:40 am, S.Santhanam ()

    Impact of micro-finance can be judged by the amount of employment, both direct and indirect, it has created over years. It has assumed the industry status in countries like India. Though there is no centralised database on the number of MFIs in India, estimates have put it anywhere between 800 and 1,200. There are about 50 MFIs functioning as Non-Banking Finance Companies (NBFCs). Besides, over 94,000 SHG Federations are reportedly operating in different parts of the country. Over 5 million Self-Help Groups (SHGs) were having bank accounts and 3.48 million SHGs have aviled loans from banks for various income generating activities. It means about 58 million rural poor have availed micro-finance and atleast 50% of them would have turned as micro-entrepreneurs over the last decade. Besides, large number of Joint Liability Groups and Grameen Groups are also functioning in India.

    In providing various services to these groups and institutions, over 1 million people would have got employment opportunities in the MFIs, Federations etc. It has created employment opportunities in micro-insurance sector as number of micro-insurance products have been introduced by the insurance companies in India. Further, Banks have started using the services of Banking Correspondents and Banking Facilitators. Information Technology has also penetrated the micro-finance sector with employment opportunities created for hardware and software development.

    At a time when countries are discussing recession impact on employment, micro-finance sector in India is still flourishing.

  • July 3rd, 2009 at 6:11 am, Sheriff Alabi ()

    Ah, yes – afterall, that’s the essence of accountable, responsible (dare I say, responsive) and sustainable investing. Perhaps, a Global Microcredit Impacts Monitor is warranted – to determine economic, social and environmental impacts of the activities of the sector.

  • July 4th, 2009 at 2:22 am, V.Rengarajan ()

    In the context of proving the sustainability of impact of Micro finance at client level without bias in a given sample over a period of time, a ‘revisit’ assumes significance. In the parlance of a social researcher, it could represent ‘ longitudinal design ‘ which involves two stages – 1) Base line data collection initially and 2) after a period of 2 to 3 years, second round of data collection on the same sample of the project, covered in the first round. A comparison of these data from two stages provides evidence of the actual impact status. A revisit may be made after 5 to 10 years.

    From the posting it is shocking to read “ 30 years into the micro finance movement we have little solid evidence that it improves the lives of clients in measurable ways.” It is high time to recognize the importance of the ‘revisit’ having longitudinal design for appreciating real impact of micro finance on the poverty canvas and its sustainability status over a period of time with or without micro finance, at client level. and for taking actions for improvement.

    Besides direct impact, micro finance shows some indirect impact also remaining hidden. Santhanam has well brought to light the indirect impact of Micro finance in terms of employment due to introduction of various micro insurance products in micro insurance sector and hard ware and soft ware development in IT sector for MF industry. Similar employment impact also appears to be visible in Mobile phone industry as this technology has outreached MF clients. (GB-Bangladesh)

  • July 6th, 2009 at 6:13 am, Alexandra Kobishyn ()

    Even though the microfinance movement was lethargic to conduct rigorous impact evaluations, we must not have a knee-jerk reaction to evaluations as a verdict on the entire industry. Rather we must piece together results, synthesize where appropriate, and critique where appropriate as well. For example, how exactly do we measure impact of microfinance? Is it a question of household poverty and expenditures? Is it a question of intangibles like empowerment and gender equity? Since microfinance is simultaneously both a social and financial intervention, quantifying its impact is a challenge.

    The Centre for Micro Finance (http://ifmr.ac.in/cmf), in partnership with Professors Esther Duflo and Abhijit Banerjee at MIT, has recently concluded one of the first randomized evaluations of a microcredit programme. The evaluation looked at the impact of Spandana’s expansion into the slums of Hyderabad. The research team revisited slums that had received Spandana credit and those which had not (the Spandana-treated slums and the non-treated slums were statistically similar in enough ways to make them comparable) one year after Spandana had entered.

    The research team found that the impact of the Spandana expansion was most acute on poor women who already owned businesses. Part of this impact included a shift in household expenditure patterns, with detectable shifts from temptation goods (including paan, chai and alcohol) and more on durables.

    For more on this innovative study and its results please visit:
    http://ifmr.ac.in/cmf/research/ieumc.html

  • July 7th, 2009 at 4:07 am, Jeanette Thomas ()

    Thanks for the excellent comments.
    By highlighting one study I certainly don’t mean to suggest it provides anything like a conclusive answer to the complex question at hand – I agree wholeheartedly with the point that we need to take into account a broader picture and results coming out from a range of different studies. (To be clear, the authors say that nothing in the paper actually contradicts the core ideas about what microcredit contributes in terms of poverty reduction, or in helping families smooth their expenditures, lessening the pinch of hunger and need in lean times. What they do say is that “decisive statistical evidence in favor of [these effects] is absent from these studies and extraordinarily scarce in the literature as a whole.”) The main point I take is simply that we need more hard evidence.
    Thanks for sending the link to the Banerjee/Duflo paper, which is an important contribution, and will be the subject of another post on this blog. This study will be followed by more from the ‘randomistas’ group. We shouldn’t expect any one of the studies to be conclusive, but collectively they should help shed some light on these issues.

  • July 9th, 2009 at 1:06 pm, David Roodman ()

    I’m one of the authors of the study blogged here, and I agree with much the commenters have said. The paper does not show that microfinance does not reduce poverty (on average). It just that we have had very little data with with to credibly test that proposition. Randomized trials just emerging should shed more light on the matter. Meanwhile though, I am also convinced that the “verdict” on microfinance should not rest solely on such studies. In my book (blogs.cgdev.org/open_book), I am also looking at whether microfinance increases freedom (on Sen’s definition of development as freedom) and whether it enriches the institutional fabric of societies—which it clearly has, and which is development in the vision of Joseph Schumpeter, who popularized the term “creative destruction.”

  • July 24th, 2009 at 5:51 am, V.Rengarajan ()

    Dear all
    After seeing some interesting comments in the earlier postings, I share my views on the direction of research or evaluation studies undertaken under the popular brand name ‘Micro finance’
    While most of the research/critic /impact /evaluation studies which appear in this blog, focus on the issues pertaining to supply side ( e.g. recovery, impact of credit , cost, interest, profit, technology, sustainability to MFI etc ) or methodology front for conduct of such studies ( e.g. randomization, control group , treatment group ), it appears that none of them dwells on the conceptual side of the Micro finance holistically and the level of adherence in the practice in the real world. In the process, these values of MF concept are lost with dominance of ‘micro credit’ ruling…Here are some questions to probe on the practice of concept MF. ( pl see my definition and conceptual clarity on MF made as comments in this blog under posting heads “Growing, Changing, ,Gone … , ‘What is Micro finance’, ‘New Micro finance’)
    1. How far the ethical and social considerations are examined correspondingly with other evaluative criteria under Micro finance arena in the context of reduction of poverty? What makes difference between Micro finance and other finance or are they one and the same except the word Micro prefixed ?
    2. To what extent the deviation takes place in the adoption of the globally conceived concept ‘’Micro finance’ ?
    3. How to ensure ‘inclusive’ Micro finance inclusive of all its components (savings, credit, insurance and others) to the given sample ? Is holistic adoption of the concept MF ethically and socially justified in the project ? If Micro finance is expected to reduce poverty , have all the MF inputs besides micro credit been provided in an integrated manner to the same poor clients? Does micro credit alone serve the said purpose for the most vulnerable groups?
    4. If the project is limited to micro credit only, what is the impact in the project area(partly treated group) comparatively with the impact with the provision of all the components of MF (fully treated group).? Will there be any difference in the level of impact in the above comparative study ?
    5. If the project or MFI is unable to provide all the components of MF to their target group, how to ensure the provision of MF services to the same poor clients besides credit for ushering in a sustainable development at household level.?
    6. Does micro credit alone serve the said purpose in the context of priority development to the poorest and vulnerable groups under Micro finance gamut. ?
    7. What are the lessons /‘suggestions /strategies for creating enabling environment for arrangement of all components of MF to the poor clients besides micro credit?
    8. Regarding target group, ethically speaking MF focus is more on the poorest and the woman (CGAP/ WB)..But Non poor are also (MIX data) invariably covered under MF concept. How a researcher in MF field should address this issue in his study framework? Is the issue justifiable for further probing under MF concept in the context of achieving millennium development goals?
    9. In respect of outreach, by and large the MF focus is on the poorest and women in rural area who lack financial accessibility and face lot of vulnerabilities . In real world, MFI also extends to micro credit (only) to the poor in Metropolitan city or Urban area ( business potential and easy accessibility to inputs ) running with petty/ small business. How to give credit or judge their performance taking into consideration of the focus and priority conceived in the concept Micro finance?
    10. Conceptually, the word ‘Micro’ in Micro finance has other inherent ramifications bearing ethical and moral values and social sensitivities. which demand ethical based social engineering also besides financial, .is it not necessary to design pro poor based products and services and demand oriented process in Micro financing to the poor ( loan amount, purpose, interest rate, collateral, repayment schedule, linking with non financial services including capacity building)
    11. Under widely popular MF-SHG system many groups are formed for only purpose of micro credit and the fact on mortality of the group, defunct group and drop outs of the poorest or weakest ( anti inclusive phenomenon) debt trap, has been brought out evidently in a few studies only In this context on social ground, what is the long term impact of the above adverse phenomenon in MF industry and gender development of poorest women ? Is this situation due to absence of ethical and moral approach or presence of commercial business approach in the process of Micro financing to the poor?

    Verdict on Micro finance is therefore possible in any places/context if the research or evaluation study examines the practice of the ethical and social values of the concept Micro finance holistically as the basic criteria along with other evaluative criteria and suggest ways and means for improvement in such practices with all MF inputs for a sustainable poverty reduction. Otherwise, any study confining impact of micro credit only, being a partial analysis, under the banner ‘Micro finance’ may not facilitate a change from ‘lethargic status’ in MF sector even after 3 three decades.,..
    Regarding the observation of Roodman, I also believe that Micro finance has potential to increase freedom( as development) both at individual and society level as well Here again the presence of ambiguity on the noble concept of MF and unethical practices in the modern market economy retards the positive development in the poverty sector. In this context it is relevant to quote Amartya Sen “Another surprising feature is the contrast between the self-consciously ‘non-ethical character of modern economics and the historical evolution of modern economics largely as an off shoot of ethics’ In another context, he says “ But I would like to argue that economics, as it has emerged , can be made more productive by paying greater and more explicit attention to the ethical considerations that shape the human behavior and judgment.” These two factors viz., ethical attitude and moral behavior are the basics in the modern economic approaches for ushering in freedom even through Micro finance also with distributive justice This requires a lot of spade work (research and innovation) with the policy support and coordination among the players. CGAP /WB may think of such initiatives for making a tangible dent in global poverty canvas through Micro finance platform

    • Leave a Reply