At ESS we often hear from organisations we work with that exploring digital tools for evaluation can be overwhelming. This feeling has only increased since the first Covid-19 lockdown as more and more digital tools emerged and became mainstream, giving organisations more opportunities to think about using new or different evaluation methods.
If you feel overwhelmed by all the digital evaluation tools out there, first of all, do not panic! You are probably already doing all the right things but just need to review the factors that influence how you choose your evaluation methods.
As always, we advise organisations to ensure their outcomes and indicators are clear before jumping into designing their evaluation methods. The principles we use to design digital evaluation methods are the same as we would use when designing any other evaluation method.
Remember to consider factors that will influence what methods will be most suitable for your participants and your team. Take into account your participants’ characteristics, such as their literacy levels, age or English fluency. Make sure to consider the most suitable time to gather feedback from them – who will collect the evidence (e.g. staff, volunteers), how often and when. A big part of designing evaluation methods involves looking at these factors first. You can find more details about designing evidence collection methods in our support guide: ESS Support Guide 2a Designing evidence collection methods.
Once you’ve clarified your outcomes and indicators, consider whether some of your evaluation methods could be implemented online using simple digital board tools. Some popular examples are Jamboard or Miro, but there are many others that have been made readily available in the past two years.
ESS does not endorse one single evaluation method, but these are examples of how you can use digital tools to design your evaluation methods:
- Let’s say you wanted to use choosing pictures or a sticky wall as evaluation methods. Where you would normally use a flipchart or a sticky wall, you could use a digital board instead:
- You can use digital boards for team discussions, focus groups with participants or even during volunteer inductions to get some before and after feedback. Below is an example of how we used Jamboard for a team discussion about reflecting on a time of change:
- Where you would usually use a survey or questionnaires, you could use online feedback forms or Zoom polls.
- When using a chat tool (for instance, on Teams or Zoom), you can ask participants to vote with emojis (image below) rather than writing down or discussing their feelings.
- There are also tools like Mentimeter that generate a word cloud from the answers that participants type in response to your questions.
Instead of feeling overwhelmed, we can try to embrace the many positive aspects of using digital tools for evaluation. These include providing more accessibility and making it easier to store information and analyse it later. We can also now call on a broader range of digital evidence, particularly from social media platforms.
While using digital tools for evaluation can be daunting, our overall advice is to apply the same principles of evaluation you usually would – set outcomes first, identify outcomes and indicators, and only then think about which evaluation methods are suitable. You will often find that there are ways to adapt offline tools to a digital format. And whether you are using digital tools or not – remember to use the feedback you are given to learn, to improve your services, and to celebrate success!
Check out our helpful tools and resources that may help you use digital tools for evaluation:
ESS Support Guide 2a Designing evidence collection methods
This guide covers things organisations should think about when designing a suitable evaluation method for the evidence they want to collect.
ESS Evaluation Method: Capturing casual moments
This method is a simple means for writing down service users’ actual words when staff or volunteers want to record comments in non-formal circumstances.
ESS Evaluation Method: Using case records
Using case records method sheet shows you how to make use of the information you collect during everyday practice to collect evidence of meeting your outcomes. Useful for: Organisations who
ESS Evaluation Method: Using contact forms to gather evidence during a call
ESS has developed this resource for those organisations who have recently adapted to remote delivery in light of COVID-19.
ESS Evaluation Method: Using social media to evaluate other activities
This method sheet can show you how to capture useful evidence from social media.
Case Study: Pain Concern Helpline
This case study is about the steps Pain Concern took in order to capture the difference their telephone helpline makes for people living in chronic pain.
Also check out our Learning From Covid page which lists all of our resources that can help you with evaluating remote activities:
Learning from Covid
ESS initially developed this page to provide evaluation support at the start of the COVID-19 crisis, but these resources can be useful to all organisations who are evaluating at a distance.
