Start With The Wound

Most evaluations go wrong at the first step, because the funder accepts the organization's description of the problem and that description is often written to make the organization's solution look inevitable.

The Five-Word Discipline

Before you evaluate an organization, state the problem it exists to solve in five to seven words or fewer, including the population, and the location. Five words sounds like a constraint. It is a diagnostic. If you cannot compress a problem to five to seven words, you do not yet understand it well enough to judge whether anyone is solving it.

Think through it in three parts:

• Social Problem
• Population
• Location

A problem definition looks like these:

• Child mortality from waterborne illness in ______
• Chronic homelessness among addicted adults in ______
• Smallholder farmer food insecurity in rural ______

Then Name Twenty Consequences

Once the problem is stated, list the negative consequences of that problem, for that population, in that place. We use twenty.

Twenty is not magic. It is simply more than you can produce from the obvious. The first five consequences of any social problem come easily. The next fifteen require you to understand what the problem actually does to people.

Take chronic homelessness among adults with mental illness and addiction in a mid-size American city. The obvious consequences are people sleeping unsheltered and visible neighborhood disorder. The full accounting looks different.

Physical health. Untreated chronic illness, emergency room overuse, exposure injury, malnutrition, dental disease, foot and skin conditions, untreated mental illness, unmanaged addiction, trauma from street violence and sexual assault, and premature death.

Social and economic. Disconnection from family, unemployability, criminal justice involvement, lost identification documents, inability to access benefits.

Community. Neighborhood destabilization, children's school performance, demands on police and emergency services.

Long term. Children inheriting instability, lost years of productive life, permanent disability, death decades before expectation.

Why This Changes The Evaluation

Now look at any organization working on homelessness and ask how many items on that list it touches, and how many it measures.

An organization reporting shelter utilization is addressing two or three. One measuring housing stability, sobriety, employment, health, and quality of life is addressing eight or ten. Before you built the list, both looked like homelessness organizations. After, they look like different kinds of work.

The list also protects you from a specific failure. Organizations naturally describe the problem in terms their program solves. A pantry describes hunger as a food distribution problem. A clinic describes it as a nutrition problem. A cash transfer program describes it as an income problem. Build the consequence list yourself, from the problem rather than from the solution, and you can see what each one is leaving out.

Where Organizations Resist This

Expect pushback, and expect it to sound reasonable. The problem is complex. Everything is connected. Narrowing it is reductive.

All true, and none of it is a reason to skip the step. An organization that cannot say precisely who it serves and what specifically goes wrong in their lives cannot design a measurement system either, because it has nothing specific to measure.