The high-profile failure of a Social Impact Bond (SIB) experiment at Riker’s Island begs the question: how can we de-risk investment in SIBs, and social programming generally? SIBs are largely predicated on private investors assuming the risk that programs will succeed at producing desired social benefits. In the Riker’s Island example, Goldman Sachs lost $1.2M and Bloomberg Philanthropies lost $6M because a juvenile recidivism intervention failed to work. Some backers hailed it a success nonetheless, insofar as the government didn’t have to pay for the program. But was it really a success? Will investors be more attracted to SIBs in the future?

Investors can only assume risks that they understand, and quantify. In the case of SIBs, it appears unlikely that Goldman, or other SIB investors were adequately able to evaluate the risk. And if they did try to assess the risk, they certainly didn’t guess right. While the program underlying the Riker’s SIB, Adolescent Behavioral Learning Experience (ABLE) intervention, was purported to be “evidence-based,” the designation remains fairly arbitrary. What does it take to deem a program “evidence-based”? Does the program need to be 20% effective? 50% effective? 80% effective? Proven effective once? And how reliable does the evidence have to be? Indeed, a comprehensive academic meta-analysis of the program’s core model, Moral Reconation Theory (MRT), suggests that the intervention was not very effective. In fact, the meta-analysis, based on 33 underlying studies, found an overall r-squared of .16 (a statistical measure of efficacy), indicating that the MRT intervention was barely effective in the past. The researchers even suggest that the MRT was “more successful with adult than juvenile defenders” in institutional settings, such as Rikers. Finally, the vast majority of the evidence on MRT was produced by Correctional Consulting, Inc. the for-profit company that invented the program.

So how can an investor, with no formal training in social science, be in a position to interpret fairly dense academic literature and make a judgment about the likelihood of success for a particular intervention? They can’t. No more than an investor can personally assess the underlying creditworthiness of an individual loan or mortgage. These financial investors typically rely on a qualified third party like a rating agency or credit bureau to appraise that credit risk. This is absent in the structure of SIBs. The focus of measurement is ex-post, not ex-ante. And therein lies a major challenge of SIBs preventing mass adoption. The rigor, the measurement, happens on the back-end, not the front-end.

“This can unlock new pieces of funding, private capital especially,” said Jim Anderson, who leads the government innovation program at Bloomberg Philanthropies. “It also brings a laser-like focus to measurable data. Everyone has an incentive. We’re not doing this to feel good. We want positive results.” The fact is, failures like Rikers won’t unlock new pieces of funding like private capital; it’ll scare them away. And that is the entire proposition of SIBs.

Here’s what we can do about it:

1. Bonds need ratings: Standardized, independent, predictive assessments of risk and return. There is no other reliable way for investors to assume capital markets risk. To date, SIB investors like Goldman Sachs have assumed philanthropic risk, and maybe some reputational risk. But the only way that SIBs or any form of social finance can attract true risk-capital is if investors are able to pre-determine the likelihood of success for an investment. With advances in meta-analysis and evaluation modeling, like the Impact Genome Project® investors can derive directionally-right estimations of effect sizes of a social program, before it runs its course.
2. Measurement needs to be done upfront, not after the fact: Most SIBs are designed based on a particular charity or social program that an intermediary has deemed to be “evidence-based.” This program is then brought forward to government supporters and financial underwriters by a SIB intermediary as the basis for a bond. Rigorous measurement happens after the program has been completed, to determine its effectiveness. This may not be the most efficient way to design a SIB. Witness Riker’s Island. Ideally, a government agency would first identify the outcome that they would like to advance via a SIB. Then the intermediary would partner with a third-party evaluator upfront to evaluate the best, most effective programs that can produce that outcome. A program would then be selected based on its efficacy score and its cost-per-outcome, based on the broader evidence base of what works. Governments should be purchasing outcomes, not programs that they hope will produce outcomes. See, e.g., our learnings from the State of Illinois Budgeting for Results Commission.
3. Be intentional about the purpose of the SIB: We’ve seen a variety of explanations of why SIBs are being issued: to teach governments how to be accountable and results-driven, to test innovations that wouldn’t otherwise be funded, to finance the most effective interventions, or to shift the risk of social programs and tap into private sector funding. The design of the SIB should be tied to its purpose. If the purpose is to finance the most effective interventions, then this merits a careful upfront analysis of which interventions are indeed the most effective of all interventions ever studied. Finding a program that happens to have been evaluated once or is deemed “evidence-based” is not the same. If the SIB is designed to test innovative approaches that are too risky for government to invest in, then programs should be evaluated for their highest impact potential regardless of track-record. The ABLE program selected for the Riker’s Island SIB clearly didn’t fit either of these criteria: it would be hard to justify that this was the most effective recidivism program ever evaluated, and it would be hard to say that (with a goal of 10% reduction in recidivism) this program held the most innovative, breakthrough potential.

We believe that the future of SIBs and social finance is brighter than ever. But if we are to create a true social capital market, we must be more rigorous, intentional and outcomes-driven than the current approaches.