News & CommentaryArchive
Jul 16, 2009
Debating the Value of RCTs
I wondered back in January if randomized controlled trials (RCTs) in philanthropy were reaching a tipping point. Since then, the debate over the usefulness, value and pros and cons of RCTs has heated up. Today, Bill Easterly, author of The White Man’s Burden, writes about the growing interest in RCTs and their limitations such as the ethical quandaries of “experimenting” on human beings, the limited applicability and the limited questions they can address.
I think the post misattributes some of the enthusiasm and claims for RCTs from the researchers who do them to the non-researchers who use them to advocate for certain solutions. We write about RCTs a great deal here at Philanthropy Action and I’ve never read an RCT paper that didn’t: 1) exhaustively discuss how the experiment came to an equitable and ethical method for randomization, and 2) exhaustively discuss the limited application of the results and the need for replication in more contexts. In fact, if you have a policy agenda (which we can certainly be accused of), talking to Duflo, Kremer, Karlan et. al. can be maddening since it’s virtually impossible to get them to commit to a policy recommendation beyond the local context of an experiment they’ve run. See, for instance, our interview with Duflo and Bannerjee.
Easterly is treading ground that has been more thoroughly (and excellently) covered by David Roodman at the Center for Global Development. For anyone interested in the pros and cons of RCTs, I highly recommend reading this, this and this.
There are in fact plenty of limitations to RCTs but no more so, and mostly much less so, than other methods and approaches. I suspect a great deal of the questioning of RCTs is based in how quickly they are debunking existing development theories and many of the feel-good stories of philanthropy.