Archive for: Impact of Microfinance

The most rapidly obsolescing part of my book, Due Diligence, is chapter 6, which reviews the statistical evidence of the impact of microfinance on poverty. Since I put the text to bed, working papers have appeared that test microcredit in Mongolia and Bosnia & Herzegovina and microsavings in Malawi and Chile (though the latter is marked “do not cite or circulate”). There’s also the Morocco microcredit study, which I didn’t catch wind of until too late in the book production. Add all these to the trials of microcredit in India and the Philippines and of microsavings in Kenya—the one that initiated this wave of research in 2009—and we have five credit studies and three savings ones. Read the rest of this page »

It seems extraordinary that after 50 years of international aid, there is still no consensus on whether it works. Zambian economist Dambisa Moyo (Dead Aid) has argued that aid is not only ineffective, but is actually detrimental to development.  Bill Easterly (The Elusive Quest for Growth) says that ‘trillions of dollars’ of aid have had little effect.  Others, notably Jeff Sachs (The End of Poverty) and Roger Riddel (Does Foreign Aid Work?), have argued that there is plenty of evidence of the success of individual aid projects, and that it has brought about substantial improvements in people’s lives.  If we cannot even agree on whether aid works at all, how can we address the more important and nuanced questions such as how to make that aid more effective?

At the heart of this disagreement is not a dispute about the impact of aid but about what we mean when we ask whether ‘aid works.’ Read the rest of this page »

Many MFIs that want to increase their impact on the lives of their customers and families have developed non-financial products and services, such as schools, healthcare clinics, and vocational training. Adding on such a service is sometimes referred to as “microfinance-plus.” It  is not only important for MFIs with strong social missions, but also can help increase customer satisfaction and retention, and so may also be good business sense in some cases.

But is there a limit to how far MFIs should stretch their core operations to address client needs, and at some point do you risk losing focus on what made the MFI successful in the first place? Read the rest of this page »

Rich Rosenberg blogged yesterday about an Atlantic article that profiled John Ioannidis’s critique of medical research. The article reminded me of a meeting held in Washington a few years ago.  Consumers and producers of microfinance impact studies were brought together to discuss the research agenda.  One participant, who is not a researcher, concluded dolefully that microfinance research lags far behind medical research.

My immediate thought was that that claim was probably false: not because microfinance research is so far ahead, but because much medical research seems full of problems.  If you read beyond the newspaper headlines, it’s usual to see simple correlations between health conditions and a given diet/activity/lifestyle quickly – and falsely — assumed to be causally determined.  Sample sizes are small.  Lots of hypotheses get tested, but just a few get published.

According to Ioannidis, it doesn’t get better when you scour the academic medical literature.  The willingness to broadcast that fact has made John Ioannidis a rock star in the health statistics world.  Health researchers routinely test many, many hypotheses; often rely on small samples; and face fierce competition to get published in top journals.  One result, Ioannidis argues, is that the pressure to publish, combined with editors’ penchants for publishing striking (often counter-intuitive) results, means that a lot of results get published that wouldn’t hold up if the sample had been larger or the tests more robust.  Ioannidis’s analysis holds especially strongly in non-experimental studies (his simulations suggest that 80% of “results” from non-randomized studies are in fact wrong).  Another big result is that randomized controlled trials do much better (25% of health RCTs are wrong, according to Ioannidis).

Of course, being wrong just one-quarter of the time is no cause for celebration.  But it does point to a real strength of RCTs – which is even more true of RCTs in microfinance .

The RCTs do better because they are designed from the ground up to test a particular (and narrow) set of hypotheses.  That greatly curtails the opportunity for “fishing expeditions” and the chance that one of your 57 hypotheses happens to be statistically significant by a fluke.  The questions tend to be far more focused in the microfinance context.  Does access to microfinance increase business profit?  Business investment?  Household consumption? Those microfinance  hypotheses usually stem from a clear theoretical model and should show up in clear patterns.

That’s far different from many medical studies, in which a much greater range of plausible hypotheses exist (along with a greater range of incorrect hypotheses).  The situation persists because the specific pathways that link diet/activity/lifestyle to health conditions remain poorly understood.  So lots of stuff gets tested in the medical literature, and “effects” may emerge that pass standard levels of statistical significance but which are caused by odd outliers or other features common to small data sets – and which turn out to be wrong.

So, on this score, Ioannidis’s criticisms are far less of a concern when it comes to microfinance.  We have tighter theoretical understandings, a smaller set of hypotheses, and, usually, bigger samples.
But don’t relax completely.  Microfinance research has its own set of concerns.  Here are a few:

1) Replication.  We don’t (and can’t) replicate studies in the sense that medical researchers can.  Medical researchers replicate by trying a similar test again with a different, similar sample.  It can be a big help in determining the robustness of the initial finding, over-riding results that do not hold when repeatedly replicated. But in microfinance, to the extent that we replicate, we do it to test the same idea in very different settings. Sure, it worked in Bosnia, but will it work in East Timor?  Argentina?  What we get is a mapping of the landscape, learning how financial mechanisms work in different contexts.  That’s helpful to understand, but because contexts vary so greatly, a positive finding in one place rarely has power to knock out a negative result elsewhere.  Replication is crucial, but not for the reasons that replication is crucial in the medical context.

2) External validity.  The problem of replication above is tied to a more general problem in extrapolating from one context to another.  This is hardly a problem unique to RCTs: it is a generic problem of evaluation.  On this, researchers could do a better job of explaining who’s in their samples and how the populations relate to communities in other regions or countries.

3) But is it an interesting parameter?  When there’s a debate about RCT results, it’s usually not about whether the finding is wrong or right, but about whether it’s interesting.  Did the experimental design yield an estimate of an aspect of microfinance impact that matters most?  Researchers deserve much credit for measuring the short-term impact of new urban microfinance branches, say, but if we had magical powers we’d really like to measure the impact of established branches in more typical settings, and we’d want to see longer-term impacts.  It’s not fair to downplay the findings that we have just because we lust after an idealized (and unmeasurable) set of parameters.  Still, we need to accept that a given study might not give us everything we want to know.

The problem with imperfect studies is that you don’t know how big the bias is (but you know the bias could be really big).  RCTs have taken us a huge step forward, and promise to deliver clean estimates of various slices of microfinance impact.  In the end, the big question is not the one Ioannidis asks (are the results apt to be right or wrong?).  The big questions concern how the particular results matter to our understanding of microfinance broadly.
—- Jonathan Morduch

David Freedman in the November issue of The Atlantic  profiles John Ioannidis, one of the world’s most respected and sought-after experts on the credibility of medical research.

Ioannidis argues that much of what biomedical researchers conclude in published studies is misleading, exaggerated, and often flat-out wrong. “Gold-standard” randomized controlled trials (RCTs) are not immune: 25 percent of published RCT results are subsequently convincingly refuted. When Ioannidis narrowed the focus to the very pinnacle of the research pyramid—49 of the most highly regarded medical research findings of the past 13 years—he found the same problems.

The causes are varied and complex, including researcher bias (and sometimes even outright fraud), publication bias that highlights certain types of findings and buries others, inadequate statistical procedures, and of course financial conflicts of interest when pharmaceuticals are tested.

“Medical research is not especially plagued with wrongness,” says Freeman. “Other meta-research experts have confirmed that similar issues distort research in all fields of science, from physics to economics (where the highly regarded economists J. Bradford DeLong and Kevin Lang once showed how a remarkably consistent paucity of strong evidence in published economics studies made it unlikely that any of them were right).”

Does any of this bear on the credibility of microfinance RCTs? I could speculate about some differences.  For instance, I wouldn’t expect financial conflicts of interest to very much of a problem in current microfinance RCTs. And publication bias may be lesser; after all, a major reason that Dean Karlan and his buddies started IPA was to carry out research that academics would have a hard time getting published.

But I’m a statistical illiterate with no business opining on these issues.  Watch this space tomorrow for Jonathan Morduch’s much more informed reflections on the matter.

In the month of October, this blog ran a series of posts on the SKS IPO. It was meant to spur discussion and present a variety of voices on the issue. It did, thanks to the authors (Rasmussen, Harper, Sen Sarma, Waterfield, Crawford, and Danel) as well as the lively comments by readers.

From the rich conversation that was generated, here are the main issues that emerged and some of the many highlights from authors as well as readers.

There has been an explosive growth in microfinance institutions in the last decade, with many of them transforming themselves from nonprofit to for-profit financial institutions. While this trend is viewed by many as facilitating a sustainable business model that can provide financial services to more poor at lower costs, it has also invited criticism from pioneers of this field such as Mohammed Yunus who argues that the term “microfinance” should not be used for commercial MFIs.

In the series’ kick-off piece, Stephen Rasmussen asked six questions to SKS about how this IPO would benefit the poor:

“Will clients be better served by an expanding the range of services, higher quality services, and more affordable services? Will the clients retain a strong voice in the affairs of the company to help it sustain a direction that serves their interests first and best? Will shareholders understand that doing what is best for the customer is fundamental to sustaining long term shareholder value?”

Malcolm Harper’s provocative piece argued that SKS should be treated like any other commercial enterprise that is part of a global capital market, like Wal-Mart:

“Is SKS any different from Wal-Mart? And does it matter if it isn’t? Because when the promoters of SKS make millions out of a business whose original objective was to serve the poor, that’s capitalism. Cooperative banks, which don’t make a lot of money for any one individual but do provide safe and accessible savings products to poor people. In spite of some notable exceptions, however, that’s not the dominant paradigm. It’s more profitable to keep people in debt than it is to help them save.”

In his view, an MFI should be judged on the basis of whether it can keep a customer, “MFIs certainly do that, and if we don’t like it we should maybe go back to Marx rather than nit-picking over the details,” he concluded.

Since the SKS IPO was floated, there has been much speculation in the media and by experts around the globe about whether this trend of commercialization and tapping capital markets will lead to a drift away from the original promise of microfinance – helping the poor.

Moumita Sen Sarma’s post pointed out that the current policy context in India makes it even more imperative that microfinance’s impact rather than profit is the priority:

“With an already nervous RBI (Reserve Bank of India) and Ministry of Finance wary of the path being charted out by the Indian MFIs and now SEBI (Securities and Exchange Board of India), the market regulator asking SKS unpleasant questions, it’s perhaps a time to pause and introspect. In the end, shouldn’t the true measure of an MFI’s success be when people on the street instead of asking in surprise ‘So is microfinance really that profitable?’ as they did in the wake of SKS IPO, ask rather ‘Is microfinance really that impactful?’ We need to shift our focus away from the means and on to the end.”

Among the many comments concerning mission drift, here is one:

“Will SKS diversify their portfolio suited to the clients’ needs in a manner to generate an equally balanced income from both interest based source ( micro credit ) and non interest based source like fees & commission (micro insurance) for sustaining financial impact and social impact as well?” –V.Rengarajan

The only other “pure” MFI to go public in the world was Compartamos in 2007. Carlos Danel’s piece argued that Compartamos had not drifted and that “if you do an IPO, you’re not inevitably doomed to drift.” As evidence, he provided some data on the MFI’s growth and productivity post-IPO:

“In the past 42 months, Compartamos has grown from serving 614,000 clients to serving 1.75 million, a 191% increase in outreach. Of those, 98% are (still) women and loan size has increased marginally 0.002% (in pesos), currently at 391 USD per borrower.  Our internal culture has been sustained, if not improved, from being named the fifth best company to work for in Mexico in 2007, to number 1 in 2010. Transparency is now not only a voluntary trait, it’s a regulatory requirement.  Efficiency is 17% better with a cost per client of 125 USD, still well below the average of  176 USD for Latin-American MFIs, and pricing has continued to drop steadily, as it did before the IPO, to 68% average APR (plus VAT).”

Chuck Waterfield’s post analyzed whether the IPOs signal the coming of age of microfinance markets. Asking an essential question, “How much profit is too much profit?”  He compared the two IPOs by SKS and Compartamos, asking:

“Why should we make significantly higher profits loaning to the very poor than commercial banks make from the middle-class and wealthy?  The Compartamos IPO resulted in a 300-to-1 return on initial investment to the original investors.  The SKS IPO resulted in what looks like a 12-to-1 return to the original equity investors.  IPO returns were much lower for SKS, but the SKS returns are still higher than the commercial business world, and the public still wonders if we have any limits.”

On the other hand, Gil Crawford argued that “good business is good development” and that commercial microfinance leads to better governed institutions. In his view, the most sustainable way to attract capital is through commercial markets. He cautioned against capping interest rates as that might be counterproductive:

“It has been suggested that limiting the profits of MFIs is a way to prevent exploitative profiting off the poor.  The only realistic way that we can see to limiting profits is to cap interest rates.  This has been counterproductive, most recently in Nicaragua and Venezuela. I believe that any implementation of interest rate caps will perversely and significantly hurt the working poor.  MFIs will be forced to focus on a very specific, targeted market that is cheap to serve and reliable in repayments.  Incentives for innovation will be destroyed, as will the desire to offer small loans to the poorest of the poor or to expand to rural areas that are more expensive to service. At the end of the day I fear that we would see MFIs begin to mirror the worst attributes of large utilities: minimal competition, low levels of efficiency and a lack of product innovation.”

The issue of governance was a concern for many readers. For example:

“How appropriate is it for a financial services institution like SKSML – especially, one that is meant to service poor clients – that had significant public money (SIDBI’s investments) and client money (MBTs), to lend (interest free) to its own founder director to buy shares in the same company? Is this a good practice of Corporate Governance? Should this be allowed in MFIs?” –Ramesh Arunachalam

In the recent weeks, as the events in Andhra Pradesh have rapidly unfolded, the questions and issues raised in this series resonate loudly. We thank you for your participation and invite you to our next series on the Andhra Pradesh crisis starting today.

The series will feature many experts deeply familiar with the Indian context, starting with Justin Oliver’s post with fresh data from a new study by IFMR on rural households and their access to finance in Andhra Pradesh.

You can access the whole series on SKS here and continue to leave your comments.

There are encouraging signs from new randomized impact assessments of pilot projects in the CGAP-Ford Foundation Graduation Program presented at the Microfinance Impact & Innovation conference recently held in New York. The initial results coming out the Bandhan study show that households who participated in the program have experienced a 25% average monthly increase in consumption—with important increases in nutritious food such as fruit, nuts, diary, eggs, and meat. The early results from SKS’s Ultra-Poor Program are less clear-cut, but show that participants are saving more and are less likely to borrow from money lenders than the control group.

The buzz around early results from these two studies is growing in the blogosphere: on the Financial Access Initiative blog Dean Karlan, President of Innovations for Poverty Action and Assistant Professor of economics at Yale, writes that that the results are proof “you can have an impact on the poorest of the poor.”  Jonathan Morduch of New York University and  Managing Director of the Financial Access Initiative (FAI) also referrers to the “staggeringly large” results from the Bandhan pilot, and suggests the next step will be to examine costs and sustainability (see FAI’s web site). In another recent post, Timothy Ogden, the editor in chief of Philanthropy Action, an online journal for high net worth donors, also put up a more detailed review of the results from Bandhan and SKS revealed at the New York Conference.

These recent posts suggest that the results coming out of the Graduation Pilots impact assessments could get a lot of attention in the microfinance community and the development sector more broadly. The full results of the two India studies are due soon… stay tuned!

This is the first time that I have knowingly contributed to a ‘blog’; hence I am not familiar with medium’s etiquette.  Am I to oppose, to concur, or to add? I’ll try to do all three.

Steve Rasmussen poses a number of important questions; they are mostly about the future, and about clients, which is surely where our focus should be.

I shall not comment on the rights or wrongs, legal or ethical, of the ways in which the shareholdings of the SKS clients’ Mutual Benefit Trusts were handled; Professor Sriram has already covered that issue, very well.

I shall start by making some general assertions which I believe to be true, or perhaps at least to be provoking, which may be more useful than being true.

First, the impact of microfinance, for good or for ill, is exaggerated.  It is no more than second-rate retail banking for people who cannot afford the relatively decent services which all the readers of this ‘blog’ enjoy. Our banks serve us, more or less effectively, but they are much less important to us than our schools, our health care, our employment, our security, our communications… Let’s not fall into the trap of believing that microfinance, or the institutions which provide it, or still less the parasites which surround it, academics, commentators, consultants, researchers and so on, are so very important.

Second, the number of people who are still un-served by microfinance is also exaggerated. The figure of 150 million people omits the sixty odd million members of self-help groups in India, whose groups are served by commercial banks, the millions who have individual accounts with banks of all kinds, particularly cooperatives and credit unions, and the many more who are rather well-served by informal but often very sophisticated traditional institutions.

And every human being does not need financial services. Most households do, and there are not three billion un-banked households.

Third, the commercialisation of microfinance is a ‘done deal.’  To misuse a sentence from Abraham Lincoln, “the world will little note nor long remember what we say here.” When BASIX was started in Hyderabad our mission said we hoped to “access mainstream capital.” Now microfinance is a mainstream business, it’s been “Wal-Martised,” whether we like it or not, Compartamos and now SKS have shown the way, and soon many more will follow. The clock cannot be put back.

So, let’s look at microfinance, and SKS, through the same lens as we look at any other business. We don’t expect Wal-Mart or their peers not to rip off the poor because they want to help them. We hope that antitrust laws, other regulations, public opinion, and above all competition will stop them. Generally, albeit imperfectly, it works.

Mr. Sam Walton and his family have made vast sums of money by successfully satisfying the needs of millions of people, particularly not-so-wealthy people, and we don’t seem to grudge them their riches. If a donor had subsidised Wal-Mart’s entry into some apparently unattractive market; rather as DFID so cleverly subsidised Vodaphone’s MPESA in East Africa, and Wal-Mart had done very well out of it, as MPESA apparently has, would we complain? I think we would congratulate the donor for effective promotion of ‘market access for the poor.’

So although many people (and I include myself), find it distasteful, or may even think it stinks (are we perhaps secretly a little jealous?), when the promoters of SKS make millions out of a business whose original objective was to serve the poor, that’s capitalism.

We may prefer co-operative banks, which don’t make a lot of money for any one individual but do provide safe and accessible savings products to poor people. In spite of some notable exceptions, however, that’s not the dominant paradigm.

It’s more profitable to keep people in debt than it is to help them save.

As Peter Drucker wrote, “the purpose of business is to create and keep a customer,” MFIs certainly do that, and if we don’t like it we should maybe go back to Marx rather than nit-picking over the details.

— Malcolm Harper

Watch this space for a new post every week on the SKS IPO. Next week we feature Moumita Sen Sarma.

This post is the second in a special blog series on the SKS IPO. Over the coming weeks we’ll be featuring a variety of voices on the issues raised by the IPO. We welcome your participation in this discussion through comments.

This was one of the questions raised in a lively discussion that CGAP organized on “the Impact of Microfinance” early September with about 60 French microfinance stakeholders and experts from the AFD, CERISE, and the Poverty Lab. The debate kicked off with an overview of three main approaches to impact measurement and monitoring in microfinance: quasi experimental and qualitative studies, social performance measurement, and randomized control trials (RCTs). Having three different discussants from a donor agency, a promoter of social performance measurement, and a researcher at the same table was a unique opportunity for establishing common ground and debating these three trends.

The discussion included animated debate about the degree to which one can draw lessons from the three existing RCTs on microfinance. Participants raised several questions such as: “is the time span used by RCTs (1 to 2 years) sufficient to measure impact on household poverty?” or “if an RCT measures the impact of a type of credit in selected remote areas in Morocco, what does it tell us on the impact of credit in the entire country?” One RCT supporter responded that microentrepreneurs make quick returns on small investments based on recent research but reckoned that it would be wrong to extrapolate local impact results in a remote area of Morocco to the national context. Some participants also argued that it was unethical to keep a population excluded from the services for two years. Finally some donors mentioned that RCTs were expensive (about 400-500,000 $US each) given that they do not provide them with full answers on how they can improve their microfinance projects. One randomista responded “is this amount a lot when we know all the money that goes to development without knowing whether interventions actually work?”

There’s plenty of appetite to further this discussion so that we can learn more about the value that different types of research approaches can bring to answering some fundamental questions.

CGAP is sponsoring a conference on Microfinance Impact and Innovation together with academics, practitioners and investors in NYC 21-23 October.

–Eric Duflos

In an August 22 post on DevFinance Dale Adams opined that my CGAP paper on impact “dismissed” expressed demand (also called “revealed preference”) as a way to find out if microcredit helps borrowers:

“In an otherwise excellent note [thanks, Dale!], Rosenberg dismisses expressed-demand as an indicator of the usefulness of microloans: (CGAP, Focus Note No. 59, “Does Microcredit Really Help Poor People?”). This leads him to conclude that more rigorous (and costly) studies are needed to measure the benefits of microloans…. 

My first concern is with the comparison he uses to justify dismissing the votes-people-make-with-their-feet as a measure of the usefulness of loans to borrowers.   To support his claim he says that ‘…repeated use does not by itself prove that a service is benefiting users.  No one would make this argument about repeated use of heroin, for example.’  He goes on to mention that borrowers might be caught in a debt trap and seek loans to keep their heads above water.

Using comparisons, analogies, and parables can be useful (and tricky) in making a point, but equating loans to a habit-forming drug is a stretch.  Does CGAP really want to take the position that microloans are habit forming and therefore dangerous to the health of borrowers?   Even if a tiny percentage of borrowers do fall into debt traps, is it useful to extend the habit-forming analogy to cover all microborrowers? Should we think of the millions of women who take loans from the Grameen Bank as addicts?”

That’s how Dale read it. On the other hand, a May 17 New Yorker article quoted prominent researcher Esther Duflo as saying that our position was “moronic” precisely because it embraced (!) the expressed-demand argument. 

(Were Drs. Adams and Duflos both reading the same paper?) 

I thought the paper expressed a middle position. On the one hand, I used the admittedly extreme example of addictive substances to make the point (a correct one, I think) that you cannot automatically assume that repeated use means something is good for someone.  However, the whole discussion both before and after the sentences that worried Dale was a series of arguments about why we should take the revealed preferences of micro-borrowers (and especially micro-repayers) seriously as evidence that they benefit from their loans. I ended with the point (also correct, I think) that the revealed-preference arguments I was making do not conclusively settle the matter. And that therefore further research would be a good idea.

Rigorous impact studies are fairly expensive–the only on whose price tag I know was high six figures. But I find it a little hard to take this seriously as an argument against more studies.  As a percentage of the mutiple billions that donors and socially-oriented investors have already spent on microfinance, the cost of impact studies is inconsequential. Given how much we still don’t know about what kind of impact all this investment is producing, I think further research makes pretty good sense.

What’s the moral of this little story?  That impact is a really complicated topic? Or maybe just that I should work at writing more clearly?

Richard Rosenberg