In ‘Nature,’ Prof. Mushfiq Mobarak Says Aid Programs Must Be Evaluated as They Scale
Mobarak, the founder of the Yale Research Initiative on Innovation and Scale, writes that rigorous data collection is key to understanding whether the results of a pilot hold on a larger scale.
In a comment in the journal Nature published last week, Yale SOM development economist Ahmed Mushfiq Mobarak writes that rigorous evaluation of evidence-based aid programs should continue as they are scaled up, rather than ending after a successful pilot. Mobarak is the founder of the Yale Research Initiative on Innovation and Scale (Y-RISE), which focuses on the effects of policy interventions when delivered at scale.
“When programs enter a ‘scaling stage,’ the focus often immediately shifts to solving the practical issues of broader implementation of the program (such as how to teach government staff about an innovation, distribute subsidies to tens of thousands of people, instead of hundreds, or integrate a program across government systems),” he writes. “All that work, although essential, overlooks the crucial question of whether exciting pilot results still hold. Many—if not most—development programs encounter uncertainties and complexities that emerge only at scale. These are rarely observed—and therefore cannot be analyzed—during the initial pilots.”
Mobarak describes his experience studying a program in Bangladesh that provided bus tickets to rural laborers so that they could temporarily relocate to a city for work during seasons when less agricultural work is available. The intervention was effective during a pilot, but when it was expanded to hundreds of thousands of workers, the benefits diminished and it was discontinued. “Although this was disheartening, I remain proud of collecting that decision-aiding information: it prevented waste and meant that the limited money for anti-poverty programs was better spent.”