Measuring impact without a research team

This guide is for small charities that are asked "what difference do you make?" by funders, donors or their own board, and aren't sure how to answer beyond "a lot." You don't need a statistician or expensive software to measure your impact. After reading, you will understand the difference between outputs and outcomes, be able to draw a simple logic model, run a short survey, and report your results honestly.

Why measure at all

Measuring isn't just for funders. Done well, it helps you:

• See whether your work is actually helping, and for whom
• Spot problems early and fix them
• Decide where to spend scarce time and money
• Tell a truthful, convincing story to supporters

Done badly, it eats volunteer time and produces numbers nobody uses. The aim is a little measurement, used well.

Outputs versus outcomes

These two words cause a lot of confusion. They mean different things.

• Outputs are what you do and produce: the number of meals served, workshops held, people tutored, hours of mentoring. They are easy to count.
• Outcomes are the changes that happen for people because of what you do: children read at a higher level, families have enough to eat through the month, people find work, older adults feel less lonely.

Hypothetical example: A tutoring program held 80 sessions with 25 children (outputs). By the end of the school year, most of those children said they felt more confident reading aloud, and their teachers reported improved reading levels (outcomes).

Outputs show you are busy. Outcomes show you are making a difference. You need both. Outputs are often the first step toward outcomes, and they are much easier to count.

A simple logic model

A logic model is a one-page picture of how your program is supposed to work. The CDC, which publishes a widely used framework for evaluating programs, describes it as a one-page graphic showing the connection between a program's activities and its intended outcomes.

The common parts are:

Hypothetical example for a reading program:

• Inputs: 10 volunteer tutors, donated books, a room at the library
• Activities: weekly one-to-one reading sessions
• Outputs: sessions held, children attending, books sent home
• Short-term outcomes: children enjoy reading more and read more often at home
• Long-term outcomes: children read at grade level

Draw it with your team, and with some of the people you serve if you can. The CDC encourages involving the people who have a stake in the program from the start. Drawing the model often reveals gaps, like an outcome that no activity is really aiming at.

Choose a few things to measure

For each outcome, ask: "How would we know if this was happening?" That answer is your indicator: a sign you can observe or ask about.

Good indicators are:

• Simple to collect with the time you have
• Meaningful to the people you serve, not only to funders
• Consistent, so you can compare over time

Mix numbers with stories. Numbers tell you how much. Stories and comments tell you why.

Simple ways to collect information

• Attendance and activity records for outputs. A shared spreadsheet is fine.
• Short surveys at the start and end of a program, or a few months later.
• Conversations: a few short interviews or a small group discussion.
• Observation: what volunteers notice, written down regularly.
• Information from partners, like teachers or caseworkers, with appropriate consent.

Writing a short survey

• Keep it short: five to eight questions. Respect people's time.
• Use plain language and offer translations where needed.
• Ask about change: "Compared to before the program, how confident do you feel about...?"
• Use the same questions each time so you can compare.
• Include one open question: "What is one thing we could do better?"
• Make it anonymous if you can, and say so. People are more honest when they can't be identified.
• Never make help depend on filling it in.

Look at the results together

Set aside an hour each quarter with your team:

• What do the numbers say?
• What do the comments say?
• What surprised us?
• What will we change?

Write down one or two decisions. Measurement only matters if it changes what you do.

Report honestly

Honest reporting builds trust. Funders and donors know that not everything works.

• Use real numbers, and say how you got them. "18 of the 25 parents who answered our survey said..." is better than "most parents."
• Separate outputs from outcomes. Don't present the number of meals served as proof that hunger went down.
• Don't claim credit for everything. Other things in people's lives also cause change. Say "contributed to" rather than "caused" unless you are sure.
• Share what didn't work and what you learned from it.
• Protect privacy. Report in groups, not about identifiable individuals, unless they have agreed.
• Put one or two stories alongside the numbers, with consent.

Your impact checklist

• List your main outputs and set up a simple way to count them
• Choose one or two outcomes that matter most
• Draw a one-page logic model with your team
• Pick one or two indicators for each outcome
• Write a short, plain-language survey and test it on two people
• Hold a quarterly review and record one decision
• Report numbers and stories honestly, including what you learned

Measuring impact is not about proving you are perfect. It is about learning, a little at a time, how to do more good.

If you want help drawing your logic model or writing a survey, ask a member charity for help through the Exchange.