Between 2019 and 2023, Evaluation Support Scotland set learning priorities. These were questions we wanted to explore to help us better support third sector evaluation practice. In the process of reflecting on our learning priorities, we realised that our learning from self-evaluation is your learning about self-evaluation. Here’s what we found.
1. What is “advanced” evaluation support?
ESS has learned there is no single answer. Everyone’s “advanced” is different and some interventions are inherently more complicated to evaluate than others. Nevertheless, adaptive challenges (eg getting colleagues on board) and tough topics (evaluating prevention or policy influence) are easier to solve if you already know the evaluation basics.
The key is for organisations to think through what you need so your evaluation is good enough for planning, improving and celebrating. Some tools to help:
2. How can organisations embed evaluation?
Embedding means evaluation is part of the everyday and everyone in the organisation is appropriately involved.
ESS has learned that what helps embedding are:
- senior staff and board saying evaluation is important and resourcing it
- internal champions who make sure it happens
- simple systems (including IT) to support it happening
- everyone (including service users) seeing that evaluation is actually used
Our learning and embedding pages include lots of relevant resources. We turned the findings of our research about leadership for evaluation into our practical evaluation leaders toolkit.
3. How can organisations determine if their evaluation and their evidence is good enough?
There is no absolute standard for evidence – it needs to be good enough for what you need it for and the decision you need to make. Our guide “how good is my evidence” pulls together our learning about how organisations can judge if their evidence is good enough for you.
How good is my evidence?: a guide to assessing the quality of self-evaluation evidence (2020)
Sometimes we are asked ‘How do I know my evidence is good enough?’ The answer is not straightforward, because it depends on a number of factors. This guide takes you through a series of steps that allow you to answer this broad question.
There is no absolute standard for self-evaluation. What is good enough depends on what an organisation needs evaluation for, service user needs and practical considerations like time and resources. The ESS principles for good evaluation can help organisations judge if they are happy with your evaluation.
4. What role does peer learning play in supporting evaluation?
Peer learning helps organisations work through their issue and find practical solutions; and it values experience and reassures.
These events need ESS input to kick things off and directed exercises to ensure networking turns into learning. Peer learning suits some learning styles better than others and ESS has learned to signpost clearly what peer learning events are (and are not) to ensure participants know what to expect.
5. How can funders and funded work together to share and use evidence and learning?
ESS’s shared learning programme model involves bringing projects in a similar area to share learning – often as an alternative to written impact reporting. Funders and funded say this can generate tangible impact evidence and actionable learning, and reduces a sense of competition between projects.
Workshops for funder and funded projects, or having a funder speak about how they use reports can also build shared understanding.
6. How can organisations feel able to share what doesn’t work?
Our 2022 stakeholder survey confirmed that organisations don’t always believe that funders value honesty! The way around this is relationship building to help project staff realise it’s OK to talk about difficult things in front of funders. ESS can play a brokerage role.
ESS Stakeholder Survey 2022
Summary of the ESS stakeholder survey results for 2022.
7. How to involve service users in evaluation?
User involvement means passing control of some aspect of evaluation to service users. However, paid staff can be uncomfortable with this even when there is a high level of user involvement in other aspects of the organisation. Sometimes this is because staff are not confident or skilled in evaluation themselves. It’s hard to hand over control of something you don’t understand yourself.
Solutions include building staff evaluation skills, asking service users how they want to give feedback; and not to do too much user involvement all at once.
This section on our website has resources, tips and case studies:
Involving users in evaluation
Over the last few years ESS has been supporting third sector organisations to involve people they work with in evaluation.
8. What is the role of trustees in evaluation?
Trustees have a role, in line with governance duties to champion evaluation for planning, improvement and sharing the charity’s work. We have a dedicated web page for trustees and a short resource:
Trustees and their role in evaluation
This resource is designed for trustees who wish to gain a better understanding of how they can support their organisations with evaluation. The resource contains questions and items of discussion to ensure all trustees and staff are on board with evaluation.
