Philanthropy Action

Analysis, Interviews, and Reviews

Archive

Since the 1970s, development practitioners have believed that providing financial programs for the poor – mostly in the form of small loans – can have a positive impact. In some corners, proponents say that micro-loans alleviate poverty by increasing incomes. This belief has been backed up by significant on-the-ground observation, but no rigorous evidence – until now. In the past few months, the results from the first two randomized controlled trials designed to measure the impact of microcredit programs have been released.

The first study was conducted in Hyderabad, India by researchers Abhijit Banerjee, Esther Duflo, Rachel Glennerster and Cynthia Kinnan from the Abdul Latif Jameel Poverty Action Lab (JPAL) at MIT. This study aimed to measure the household impact of micro-loans given to individual borrowers by Spandana, a microfinance institution in India. We wrote about the initial results of that study in October of last year.

The second study was designed by Dean Karlan and Jonathan Zinman of Innovations for Poverty Action (IPA).  Karlan and Zinman worked with FNB Bank in the Metro Manila area of the Philippines to offer individual loans to customers that fell just below the bank’s standard risk level. The study measured the impact of loans on the borrower’s income and consumption.

The studies were by design very focused on identifying a few narrow results around income and consumption. Neither study showed significant income gains from borrowing over 11 to 22 months.

Curious about what these findings might mean for donors, Philanthropy Action editor Laura Starita sat down with Nathanael Goldberg, Project Director for microfinance research at IPA, to ask for his interpretation of the study results.

Philanthropy Action: Can you talk briefly about why these two studies are important?

Nathanael Goldberg: A few years ago I catalogued for the Grameen Foundation what we knew about the impact of microfinance and I found that there was a ton of evidence showing it seemed microfinance was working. And so on the one hand we could feel really good about that. But on the other hand it was all sort of murky. Depending on how sure they wanted to be, people who wanted to be confident in knowing that their investment in microfinance was going to create a positive impact on the participants didn’t necessarily have enough to go on. A major reason why those studies taken together were murky was this problem called selection bias.

When we compare microfinance participants – borrowers usually, since most of the time when people talk about microfinance they are talking about microcredit – when we compare participants in microfinance programs to non-participants we have to ask what that comparison means. It would be easy to say, “Oh, well look, they are better off, so microfinance is great.” The problem is we don’t know what type of person is joining a microfinance program. If we think they are either more motivated to improve their lives or they have more resources at their disposal to find out about the program, or they have a good business idea, or they are more likely to get approved by the program – all these types of things might lead us to believe that the kinds of people who participate might be better off anyway, with or without microcredit.  So that means if we see people doing better in those earlier studies compared to non-participants we are not exactly sure whether that is saying something about the program itself or the people who participate in it.

So for the last few years some of us in the field of development have been saying, we really need randomized trials of the impact of credit. Studies that can answer the question, if people participate in these programs and borrow to start a little business, what effect does that have on their welfare and on the welfare of their family?

Those kinds of studies are also important for the ways in which they establish the counterfactual, which is to say, what would have happened in the absence of this program. When I catalogued the available studies for Grameen, the research pointed to a trend of positive impact from microfinance, but the studies had for the most part not established that counterfactual with a high degree of certainty. People who say that we already know everything we need to know, I don’t think are reading the research correctly.  More relevant is the fact that, since I conducted that review, some of the more rigorous work has been dismantled. For example, David Roodman and Jonathan Morduch have recently shown that the results of one of the most widely cited studies on the impacts of borrowing, conducted by Pitt and Khandker in Bangladesh, were not credible.  Most of the remaining studies I catalogued were smaller, less rigorous work.

PA: The fact that the term randomistas has evolved in the development lexicon to describe researchers who do randomized controlled trials suggests a certain mockery of the field. If there is a criticism of this kind of research what might it be, and what is the trade off?

NG: I prefer to take the term randomistas as gentle teasing of the popularity that randomized trials have acquired over the past couple of years. My assumption is that even those who are using the word randomistas will be turning to studies like this to see what in fact the impacts of microfinance programs are, because they know that these studies are the ones they can actually trust the results from.

So I would say there is no operational disadvantage – this is the right way to be doing research. But there are certainly limitations to it. The kinds of limitations that apply to randomized trials are the same limitations that apply to research in general.

Randomized trials solve what we call internal validity, which basically means that the study was well done and measuring what it intends to measure. With a randomized trial, as with any research, you have to pick a narrow target – a well defined product offered to a particular group of people in a particular place. You also have to define a starting point and an end point for the project.

All research is limited in its external validity, and that is the extent to which we think the results about a particular type of product might say something about another microfinance service that is being offered somewhere else in the world to different target clientele. Everybody will have to decide for themselves the extent to which they want to apply the results of our research to other situations.

PA: So how are these two studies designed to solve internal validity?

NG: They take two actually quite different approaches to finding a comparable group of participants vs. non-participants. They are both randomized trials. They are both randomly assigning some people to participate in the program and some people to serve as a control group. Because everyone was randomly selected and there was a large enough population for the study, the researchers can establish with confidence that the people who participate are just like people who don’t participate, except for the fact that the treatment group has been given access to this program.

For the one that was done in India, the researchers randomly divided Hyderabad into 120 different slums. They then randomly picked half of the slums to have access to microfinance through the microfinance institution Spandana and the other half to serve as a control group and not receive offers of Spandana loans. What that gets rid of is any kind of selection bias in terms of who participates, because they’re measuring the average impact across the entire slum.

What they ended up seeing is that Spandana going into the treatment slums created an eight percentage point increase in microfinance participation. So the impact on the population represented within that eight percentage points is going to be averaged out over all the households in the slums.

PA: Could you put into context what 8 percent means for this kind of a program. Is eight a large number or not so large?

NG: It is large, especially when you consider that these are percentage points, not percentages. The baseline data from the study showed that 18.7 percent of the population was already participating in some form of credit program, formal or informal. So an increase of eight percentage points, from that 18.7 up to 27, translates to something approaching a fifty percent increase in borrowing – so that is a huge increase, actually.

What we are finding across the world when people start to really document what the take-up rates might be of credit among micro-entrepreneurs, it is much lower than people had originally assumed. In some places we see take-up as low as four percent. So to achieve those eight percentage points across a whole slum shows a fair bit of demand for this type of service.

PA: What was the structure of the Philippines study?

NG: The Philippines study was individual randomization – somebody applies for a loan and they are either accepted or rejected. The way this individual randomization was achieved is very operationally convenient. There was a partner bank, FMB, a bank in the metropolitan area of Manila in the Philippines. FMB had a credit scoring system, where the profile of the applicant would be considered and the applicant would be assigned a score. If the applicant was above the baseline score required by the bank they would be approved for a loan and if they were below they would be rejected. There is nothing necessarily magic about that break point between being accepted and rejected. The bank can’t say for sure that the customer was going to repay their loan if they were above it and they were going to default if they were below it.

So the researchers worked with FMB to push that bar down a little bit and let in some marginal applicants who would have been rejected under the regular credit system. But we flipped a coin on this set of borrowers who would have been rejected and let some of them into the program. That is the target group for the study – marginal applicants.

The results were then measured just across this marginal group. That is essential, because we always want the treatment group to be exactly like the control group. If the researchers hadn’t only measured results in this marginal group, they would have had a biased sample, since the bank would have only let in the best performers, the ones they are always going to really want to lend to, and would have rejected the ones they thought had no chance of repaying.

This middle zone which the bank wasn’t sure was going to be a good credit risk or not was kind of an unknown case, business-wise. So among that group the researchers could randomly assign them to be offered credit or not, and the ones who were offered credit were just like the ones who were not offered credit, except that they were given this access – kind of like a second chance – at this loan.

PA: And this study found some surprising things about the impact of micro-credit on this group, correct?

NG: Yes, quite surprising. The authors found an increase in profits, but not for women and not for the very poor, which is the classic microfinance target group. Instead, this study showed an increase in profits among men and people in the higher income end of the spectrum – those who had more money to begin with.

PA: Any thoughts about why the impacts were greater for these populations?
NG: Usually research leads to more research. In order to unpack any of these curious findings we see here researchers will need to attack those specific questions to really get some solid answers about them. Sometimes we will speculate as best we can with the evidence we have, but I don’t think we have enough here to say too much about it. It is a finding that we’ve noted. It is surprising and it could lead to two types of research outputs in the future. One would be just answering something about why we think this is happening. The other would be, if we think women are an important target group to help, to ask if there may be a different type of product that would be more productive for women to use. So just to give an example, you could wonder if perhaps a smaller loan size would be more appropriate for women and test it.

PA: This result that men are seeing increases in profits but women are not is reinforcing some of the research that David McKenzie from the World Bank published last year – His study showed lower returns on capital for women than for men among micro-entrepreneurs in Sri Lanka.

NG: That is absolutely right. These studies are raising a small but growing alarm. Innovations for Poverty Action believes in replicating studies before we can be confident about any particular trend or result holding across all sorts of different settings. We aren’t nearly there yet, but this could be the beginning of a growing concern about the impact of credit on women in particular and it is certainly something we need to continue to pay attention to across more studies. In the case of the McKenzie study, there was an issue about traditional male types of enterprises and traditional female enterprises – for example, auto repair for men – and it is not clear to what extent the differences in returns are driven by male types of industries having higher returns. That kind of dynamic would be different from saying that women are not able to profit from credit or enterprise in general.

PA: Another unexpected result from the Karlan/Zinman study in the Philippines was that the profits seem to come at the expense of the size of the business.

NG: People have been wondering for years about the effects of microfinance on employment, and here we see results that actually show businesses shedding employees after borrowing. That is a pretty puzzling result when you wonder, why would they need to borrow to spend less money on staff?

So the first thing you might think is these businesses are perhaps shifting into a different activity. They might be investing in some kind of mechanized process and therefore losing employees. But in fact we don’t see that kind of investment in this study. So we can only speculate with what we have.

The best explanation seems to be some kind of a household risk management story, where the employees potentially had been part of a risk sharing network which the employers found they no longer needed when given access to a more formal type of credit.

PA: It seems hard to interpret whether these are good results or bad results. Are the people better off, or not?

NG: It is certainly not negative. We don’t see harmful effects here. Just as people take on too much credit in developed countries, becoming too indebted in a developing country setting would be possible as well. The way we would measure that would be to look for a decrease in welfare, which could be measured as a decrease in consumption. But the results from the Philippines study don’t show a decrease in the types of consumption we would care about, like school fees and nutrition and those kinds of things – That would be a clear signal that there is a problem here.

What we do see in the India study is some decrease in consumption of the types of goods you could imagine households giving up because they are actually investing in their businesses. So in India, think of going out and purchasing tea. In the Spandana study, those are the kinds of purchases we do see less of, but only in the households that are found to be likely to start a business. So the story seems to be that the people who are likely to start businesses and given access to credit are actually cutting back a little bit on some non-necessary expenditures, because they are going to ramp up that investment to enhance it and put even more money into the enterprise. And that is exactly what microfinance advocates would be hoping to see.

What we aren’t seeing yet in the case of the Spandana impact assessment is any marked increase in profits within this group that is starting a business. Nor are we seeing any positive impact on education and health spending or on women’s empowerment, which is measured here by women’s power to determine how money is spent in the home. The lack of impact in those areas could be a matter of the study’s timeframe. The researchers went back after 15-18 months, and we see an increase in productive investment and a small decrease consumption, which would set up what we think will be an increase in consumption once the profits start coming in. It would be great to have that longer term measurement, but we don’t have that now.

Some critics have said, well you’ve only come in after 15 months, how could you possibly expect to see a positive impact. I would say to that, there are lots of people who have been telling a story about an immediate increase in people’s welfare based on participating in microfinance programs. It was important as a first step to check for that, and we don’t see that in this particular case. As a next step we need to see whether microfinance has this big impact and how long it takes to achieve that.

Now remember, that decrease in consumption is only among likely entrepreneurs. Among the ones that seem to be not the type of households to undertake an enterprise we see an increase in consumption. With the same type of financial product it seems some people are starting an enterprise and other people are using it as more of a consumption loan. What may happen is the ones who invest will enjoy future profits and the ones who have taken it on as a consumption loan will have to repay that money eventually – they may have a future decrease. So the effect may be reversed next year. 

PA: Do the results from these studies mean anything from the perspective of the donor or the practitioner? How should the donor be donating his money? Or how should the practitioner be structuring loans or relationships with potential clients?

NG: These studies are the first of what needs to be a series of important randomized evaluations of different types of microfinance products and services. They are very important and they are very new, but I don’t think they are nearly enough to determine how investment decisions in microfinance should be made. They are the first round in exactly the kind of evidence we are going to need to see before we know if microfinance is a good investment in general and what types of microfinance services are the most effective at helping populations that we care about.

PA: Any particular studies coming out in the future that you are excited about?

NG: Some research that I am involved in is the Targeting the Ultra-Poor program. That research is focused on whether there are ways to start with a grant-based program to get the poorest families into entrepreneurship and food self-sufficiency. We have studies on different types of savings products, and we have studies on village savings and loan associations to see if savings-led community groups can raise their own investment funds to create impact within the community. It is going to take that kind of a range of studies to see all the different ways we can create impact and improve welfare.

Comments

Remember my personal information

Notify me of follow-up comments?

Comments may be edited for length. Inappropriate comments will not be published.