Like many third sector organisations, at ESS we have had to adapt quickly to deliver our services in new ways during lockdown. After delivering our first two rounds of remote Let’s Evaluate! workshops, Training Officers Shona and Grace decided to spend some time reflecting on how they have gone so far. Evaluating our new way of working to see if we are still meeting our outcomes and to capture the learning about what works has helped us to keep on the right track and make our online services even better.
Evaluating our remote workshops
Online delivery has meant we have had to use different ways of collecting evaluation data than we would use in person – but the process we used to analyse and learn from evaluation has stayed the same.
We reflected on our sources of evaluation evidence – we used participant feedback forms (completed on Google docs instead of paper), participants comments from the ‘chat’ box in Zoom, screenshots of workshops, and ad hoc feedback received by email following workshops or on social media and collected in our Quote Bank.
We then compared our statistics from online delivery on our learning outcomes against last year’s self-evaluation of in-person workshops to see if there were any significant differences. We also coded our qualitative data to understand the most common themes that were emerging. Training officers Shona and Grace then met to compare and discuss what our evidence was telling us, and to agree next steps.
We had a further team development session with all colleagues from ESS to capture what we are learning about the different types of remote delivery we have done so far, and to share what works to overcome different challenges.
What we learned
On the whole we were pleased and reassured that Let’s Evaluate workshops continue to be useful and relevant, and that participants can learn equally well online.
“I was unsure as to how I would get on with doing the workshops online but other than a small blip today when my zoom crashed (I was able to get back in very quickly) it has been great. I understand this is the first time Grace and Shona have delivered the workshops online and I think you have both done a brilliant job, at times I almost forgot we were online!!” – Joanne, Home Start East Fife
We acted on our learning along the way
We made small adjustments (like extending the timings of workshops) as they came up in response to specific feedback.
Existing challenges are still relevant
Getting the balance right for everyone between re-capping the basics and introducing new material is difficult – this was a theme that emerged in our in-person delivery and continues to be relevant in remote sessions.
New challenges have emerged
Participants wanted more time to learn from and network with their peers, especially the informal learning that can often happen in person during breaks. We are now exploring how we can offer new opportunities for peer learning to meet this need (watch this space!).
Unexpected outcomes happened
Participants didn’t just learn about evaluation, they also picked up new ideas about how to deliver online for their own clients and felt more confident using different tools to do this. We felt delivering in shorter periods online would work best for our participants to avoid Zoom-fatigue, this worked well, and also helped participants fit time for training into their diaries more easily.
This ‘spot-check’ evaluation of the first two rounds of remote support has left us both reassured that we are making a difference for organisations, and inspired by helpful ideas for improving things in the future. Well worth the time spent!
