5 Questions to Ask Before Starting a Program Evaluation
The program evaluation. Everyone has a love/hate relationship with them, but they’re so necessary for organizations who are doing good work to continue doing so.
It’s important that non-profits regularly evaluate their programs and apply the lessons learned to the program’s direction, and to future programming. They’re also a great tool to use when starting your strategic planning process (You know, that binder that you threw on your bookshelf after the planning process was complete. Take a look. It’s probably dusty). Evaluations are also great pieces of information to share with your funders and donors. Of course, you’ll have to edit the final report into an easily digestible summary for them.
Often though, program evaluations start off hazy, and the resulting report isn’t exactly what we were looking for. So, here are 5 questions to ask that will help guide your evaluation outline:
1. What are we evaluating?
This is a really obvious question but it’s good to think through exactly what you’re evaluating. Are you looking at a program, a project or your entire organization? What timeline are you working with? I’ve witnessed scenarios where consultants were asked to evaluate a program from its inception onward (a span of 20 years). For non-profits on limited budgets, it’s important to narrow down what’s actually important. You might also want to define which aspects of your program you want to evaluate: design, implementation, HR management, etc.
2. Internal or external evaluator?
Depending on the kind of evaluation you’re conducting, you may want to consider hiring an external consultant, especially if you don’t have anyone on staff with evaluation experience. Your budget will also be an important factor in making this decision. As a general rule, you should allocate 5-10% of the program’s cost to evaluating it. If you have a really small budget, you can still work with an external consultant but think about how you can save money; for example by getting a volunteer to assist.
3. Which stakeholders will be involved in the evaluation?
It’s a good idea to determine at the outset who you’ll be involving in the evaluation so that you can develop an action plan for working with them. You might want to ask your board, management, staff and volunteers (not just those working directly on the project). You may also consider asking for program beneficiaries to participate, as well as people from your target market (including those who didn’t participate in the program), and organizations similar to yours. And you might also want to include your program’s funders in the evaluation to get their unique perspectives. Most importantly, compensate people for their time: This can range from a simple thank you note for a staff member to an honorarium for beneficiaries.
4. What kind of evaluation will we be conducting?
There are several types of evaluations you can consider:
- Formative: This type of evaluation takes place before or during a project’s implementation. The goal is to improve the project’s design and performance.
- Process: Process evaluations will help you see how a program outcome or impact was achieved. This evaluation looks at the types and quantities of service provided, who the beneficiaries were, and what resources were needed to deliver the service. It also looks problems encountered by the program and how/if they were resolved.
- Summative: This type of evaluation is completed at the end of a project to determine whether the project was effective and if it achieved its goals.
- Outcome: These evaluations assess the whether a program was effective in producing change, especially in the long term. Outcome evaluations focus on the difference your program made on participants, or the impact it had on society.
5. What tools do I have available for the evaluation?
There are lots of evaluation tools out there, and you can tailor what you use to your own program, needs and budget. Some examples include:
- Surveys, interviews, focus groups
- Review of statistics, budgets, and analytics
- Comparison to internal and external benchmarks
Now you’re ready to start a program evaluation. Keep in mind, evaluations are a continuum. The evaluation you complete today will inform the questions you ask in the next one. And not all evaluations are the same. Every organization has a different mission, client base and staff structure, and these all impact how programs are reviewed.
Cheat Sheets are Charity Savant’s free guides to help you do more with less. They are quick-fix one pagers with easy to digest top-five tips to help you solve almost any problem. Check out our growing list of free Cheat Sheets.