Philanthropy Action

News & Commentary

Archive

One of the reasons we started Philanthropy Action was the belief that there was too little good information available to people and institutions who wanted to make the world a better place. If good information is not available, then no one can make the best possible decisions about where to invest. That’s why we’re such outspoken supporters of high quality evaluations of any project—and such outspoken critics of poor or altogether lacking evaluation. Of course, in this messy world we live in, conclusive evaluations are hard and expensive to do. There are reasonable trade-offs to be made between time, cost, and absolute rigor.

In recent days we’ve been following the back and forth on just this issue over one of the highest profile aid/philanthropy projects in the world today: the Millennium Villages Project. Given the ambitions of the MVP, insiders and critics alike have agreed about the importance of evaluation. But that’s been the only point of agreement. The plans for evaluation have been the subject of much discussion, at least among development and evaluation wonks. For a quick primer, read this and this by Chris Blattman. There’s plenty of other discussion on this topic from a variety of sources.

The discussion has heated up this month because of a paper by Michael Clemens of CGD and Gabriel Demombynes of the World Bank. Clemens and Demombynes discuss why they believe the current evaluation plans for the MVP are inadequate and propose a way of adding significant rigor to the evaluation at low cost. A few weeks after the paper was released, the MVP responded on its blog defending its evaluation protocols and the goals of the project. Clemens and Demombynes responded on the World Bank’s Africa Can…End Poverty blog. Chris Blattman weighed in again as well with some thoughts on other ways of usefully evaluating the MVP. That’s when things started getting strange.

As Laura Freschi points out in a post on AidWatch that critiques MVP’s response, there was a seminar scheduled by the World Bank for Jeff Sachs and his team to engage in a public conversation (streamed live on the Web) about evaluation with Clemens and Demombynes on October 27th. A few days before the seminar though, it was “rescheduled.“ Given the demands on the time of the people involved, that’s not surprising.

But I’ve now heard from sources at the Bank that there was no scheduling conflict. My sources told me that the MVP team demanded that the public discussion be canceled, while a private meeting with the same participants, in the same time-slot was held instead. I’m also told that the cancellation of the public discussion was accomplished by going over the heads of the original planners, and while I have no way of verifying that this is true, it’s certainly disturbing. The fact that none of the people I spoke to were willing to go on the record for fear of falling afoul of the wrong people is also very concerning.

So why wasn’t the discussion public? I admit to being a little baffled. I can’t imagine why someone would make up the stories I’ve been told, nor can I imagine a good reason why the MVP would use its clout to close down public discussion on such an important issue.

I certainly hope the cancellation of the public discussion was truly a “rescheduling” and not because MVP pressured the Bank to keep such an important discussion out of public view. Regardless of the reasons for the original discussion not being public, there is no good reason that a follow-up public discussion should not happen. The entire development and philanthropy community, and the MVP, can only benefit from an open and honest conversation about how to best evaluate the MVP. This is not just an esoteric discussion. Tens of millions of dollars and hundreds of thousands of lives, at the very least, are going to be directly affected by the evaluation of the MVP. Ensuring that we have the best possible evaluation within reasonable constraints should be a priority for everyone. I don’t know whether Clemens or Demombynes have cracked the code or whether the MVP’s protocols are sufficient—that’s why I want to see and hear the discussion and debate.

To encourage all the parties to come to the table and have that public discussion, I’ve launched a petition on Tumblr here. It simply asks CGD, the World Bank and the MVP to have the open public discussion that was promised. If you agree that such a public discussion would benefit the whole sector, please sign on and encourage friends and colleagues to do the same. You can sign the petition by visiting the Tumblr site and leaving a comment. You can also submit a post to the Tumblr by clicking on the submit button if you’d like to elaborate on your thoughts.

UPDATE: People seem to be having trouble with the Tumblr comments and/or submissions. So feel free to add your name in the comments here (via the comment button below) and I’ll add them to the list of supporters. Thanks.

Comments

Tim: Good to read your call to action for more and better philanthropy (for why that is still rare go here: http://www.cgdev.org/section/initiatives/_active/evalgap).  At the Center for Global Development/CGD (@cgdev on Twitter) we will certainly hold a public discussion of the Millennium Village Project if the World Bank doesn’t reschedule the one they planned (in December I hear), and will invite Jeff Sachs and his MP colleagues.  I hope you and others will encourage them to participate.  Nancy

October 31, 2010

Tim: Nice post.  The paper by Michael Clemens and Gabriel Demombynes was tough but constructive.  And Chris Blattman’s comments helped set the stage for a real conversation about what we *could* learn from the MVP experiment.  None of these three have an agenda; they just want to help find what works.  So it was puzzling to hear that the MVP team did not want to have the public discussion, which promised to shed more light than heat.  Let me second Nancy’s suggestion to host the rescheduled event at CGD.  I can think of no better environment than CGD to hash out differences and come up with at least a partial understanding that moves the field forward: http://denniswhittle.blogspot.com/2010/10/cgd-as-social-capital.html

November 01, 2010

Please add me to the petition. For some reason I am blocked from accessing that page.

November 01, 2010

Great post Tim.  I strongly support your proposal to get this discussion to happen, and in public. CGD would be the perfect place.

November 01, 2010

MVP are essentially Potemkim Villages. The Sachian model is well intentioned but ineffective with shifting metrics. Glad to see some light cast here - a discussion is certainly needed.

November 02, 2010

I support attempts to improve the accountability and transparency of charities, particularly major charities such as the Millennium Villages Project.

I am not sure of the extent to which a debate would constitute accountability/transparency.  It appears to me that both Millennium Villages representatives and the project’s critics have said what they wish to say, online and in the public domain. 

If Tim (a GiveWell Board member) were pushing for the disclosure of important information, I would be wholly on board.  As it is, I support the spirit of this effort and feel that Millennium Villages should align its actions with its intentions (i.e., if it doesn’t plan on having a debate it should say so; if it says the debate will be rescheduled it should do so).

Holden Karnofsky
http://www.givewell.org

November 02, 2010

Tim, Please add my name to the list. Thanks, Laura
NYU Development Research Institute, Aid Watch blog

November 03, 2010

I’m in.

November 03, 2010

Tim, please add me, also having difficulty accessing the site.

November 03, 2010

Please add my name to the list. 
  When a large amount of outside money is spent in a small area it is obvious that there will be “effects”, but they are not and cannot be evidence of program effectiveness, a la Clemons and ensuing discussion. The MVP project has been set up to avoid ever being evaluated in a full and serious sense of the words.
  The evident pre-selection of villages for easy success, means that their elites are already, as in Sauri, skilled at telling the money source what it wants to hear.  The dependence on imported hybrid seed and fertilizer, absence of local funding mechanisms, exacerbation of local political rivalries and so on, set off by inclusion of a village in the project are negative dynamics that will only play out after the external financing and control stop.
  Robert McIntyre, Local Development in Transition.

November 03, 2010
Editor

Thanks to all of you for your support.

November 05, 2010

Remember my personal information

Notify me of follow-up comments?

Comments may be edited for length. Inappropriate comments will not be published.