We Fund the Wrong Things

Beth and John had been writing year-end checks to the same three nonprofits for six years. The organizations sent beautiful reports. Children pictured at wells. Families standing in front of rebuilt homes. Graduates holding diplomas.

Last December, Beth looked up from the stack and asked John a question neither of them could answer.

"Do we actually know if any of this is working?"

They wrote the checks anyway.

The Part of the Story We Leave Out

There is a story we tell ourselves about philanthropy. A problem exists. An organization forms. People give money. The organization gets to work. Lives improve.

It is a good story and it is incomplete. The part we leave out sits between the organization gets to work and lives improve, and it is the only part that matters. Most of the time, neither the funder nor the organization has a reliable way to know whether that part happened at all.

This is not about fraud or negligence. The people giving are overwhelmingly good people who care. The organizations receiving the resources are staffed by people who work extraordinarily hard and genuinely believe they are helping. The problem is not intent. The problem is the standard we have collectively agreed to accept, and how far it falls short of what the people being served deserve.

What Gets Reported, and What Nobody Knows

500 people completed our job training program. That sounds significant. It tells you nothing about how many got jobs, how long they kept them, whether those jobs were better than what they would have found anyway, whether a single family's stability changed, or if their quality of life improved.

We distributed food supplements to 10,000 children. Also significant-sounding. Did their nutritional status improve? Their cognitive development? Did families become more food secure over time, or simply dependent on the supplements, with nothing underneath having changed? We cared about children's health. Did it improve?

We disbursed $40 million in microloans with a 96 percent repayment rate. Those numbers suggest discipline and efficiency. They say nothing about whether the businesses grew, whether incomes rose, whether children stayed in school longer, or whether the families wellbeing improved.

Each of these is something that happened. None of them is evidence that it helped.

Why We Accept It

Time. Most funders are busy people who care about a cause. Reading a glossy report and writing a check is accessible. Genuine due diligence on outcome evidence is not, at least not without the right tools.

Emotion. Human beings are wired to respond to the specific and the personal, not the statistical and the abstract. There is nothing wrong with being moved by a story. The problem is when the story substitutes for evidence instead of illustrating it. A good story is a reason to look more carefully. It is not a reason to fund.

Prestige. If a major foundation funds an organization, smaller funders follow. If an organization has operated for twenty years with a long roster of institutional supporters, we assume someone, somewhere, must have checked. Often nobody has. Large funders make the same mistakes small ones do. Longevity is not evidence of effectiveness, and neither is a prestigious donor list.

We Helped Build This

Here is the uncomfortable part.

If funders reward activity and story, organizations will produce activity and story. That is not a character flaw. It is a rational response to the incentives we set. The philanthropic marketplace does not send resources to the organizations producing the best ultimate outcomes. It sends them to the organizations who are best at describing their activities in emotionally compelling ways.

We know this from the inside. Since 1997 we have founded or co-founded five social problem-solving organizations that have worked with more than 75 million people and raised and invested over $1 billion. We reported volumes. We reported completion rates. We reported repayment rates. Nobody was lying, and the numbers were real. But if you had asked us, then, whether the people we served were living better lives because of our work, we could not have told you.

That is not a criticism we are making of someone else.

What the Alternative Looks Like

It exists, and it is recognizable once you know what to look for.

Healthy Learners trains teachers in Zambian schools to assess and treat sick children. Rather than reporting how many children were seen, they ran a matched-control study: seven schools with the program against seven without. Children in the program schools had 38 percent lower odds of acute illness and 52 percent lower odds of stunting. Cost per child per year: $1.50.

Big Brothers Big Sisters of America submitted the same mentoring program to randomized trials thirty years apart. The second followed 1,353 young people across 17 affiliates for four years and found lower violence, lower substance use, better mental health, and higher hope for the future. It also found no effect on arrest, and published that too.

Neither organization is remarkable for working hard. Thousands of organizations work hard. They are remarkable because they asked whether their programs were working, in a way that could have produced an answer they did not want, and then told everyone what they found. Then they used that information to improve their programs.

That is the standard.