A good evaluation needs experts. Obviously.
But wait .. who are the experts?
Evaluation Support Scotland (ESS) knows that expertise comes in many forms, including lived experience. We are passionate about the benefits of involving young people in evaluation and research.
So we were super excited when young people from the Scottish Youth Parliament (SYP) were commissioned by Scottish Government to carry out an evaluation of Scottish Government funded mental health and wellbeing community support and services for 5–24-year-olds, their parents and carers.
Nine young people formed a team and, through focus groups, interviews and online surveys investigated service user experiences and children and young people’s involvement in service design and delivery. They produced a report and presented their findings and recommendations to Scottish Government and COSLA.
ESS’s role involved (in the young people’s words) “helping us to understand different approaches to evaluation, supporting the development of our evaluation framework, and encouraging our love for colour coding and data analysis.” Children in Scotland provided top notch training and the young people were excellently supported throughout by SYP Manager Rosy Burgess.
But crucially the young people did the evaluation.
So what were the benefits?
The investigation and its findings were enriched by the young people’s own experiences. They brought unique insight and understanding.
Children and young people service users were comfortable being interviewed by who they saw as their peers. Service providers were equally comfortable; perhaps because they were used to working with young people and because it was clear how enthusiastic the young investigators were about the work.
The young investigators’ lived experience informed the questions they asked and enabled them to contextualise the findings. But they were careful not to be blinded by their own experiences. Indeed they were surprised and pleased to hear different views and perspectives from the people they interviewed.
As well as bringing skills and knowledge to the evaluation, the young investigators learned skills such as how to interview sensitively and how to make sense of data. The team valued being connected to others who shared their passion for the importance of quality mental health services.
It was not all plain sailing. The Scottish Government wanted the evaluation conducted in a very short timescale and this was particularly tough when the young investigators were also juggling exam and life commitments. The young people were frustrated by not being able to explore some issues further – such as who was dropping out of services or hearing service user journeys longer term. As one said “I feel like we have been left with more questions than answers.”
But they collected a huge amount of data and insights and they talked directly with policymakers to coproduce next steps. It’s too early to say what exactly will happen next but Scottish Government says they will act on the findings.
That sounds pretty expert to me!
But it’s time to put young people in the charge of this blog. We asked some of the team about their experience, and this is what they said…
“It’s important for young people to take the lead in evaluation like this as we are able to connect more effectively with young people, ask questions in ways that enable them to get their views across, and we create a safe space where we try our best to allow young people to feel heard.”
Millie
“Through this evaluation, I learned how to conduct interviews and all about data analysis. I also got to travel by myself and handle travelling to different parts of the country on my own. It was a great experience.”
Felix
“The biggest eye opener for me during this project was the variety of services. There are so many wonderful services that do not get enough recognition. Coming from a small rural area, it was nice to have a different perspective on what different services look like around Scotland. I was also surprised at how valued these services are by the people using them. Community mental health services are a place where people can come together and support one another, they bring a sense of community.”
Alannah
“I’m also amazed at how willing service users were to tell us about their experiences. The service providers weren’t scared to talk to us either and treated us like how they would another evaluator.”
Felix
