Steven Marwick, ESS’s director, talks about the challenges of evaluating collaborative work
I regularly come across words like collaboration and coproduction. Defining these words can be challenging and I won’t attempt to do so in this blog! But, in my simple way I mean “doing stuff with others to achieve change”.
Instead, I’m interested in an even greater challenge: how to evaluate collaboration?
This matters to ESS because almost everything we do is in collaboration – mostly with the third sector but also with funders and government. We want to know that our collaborative approaches are effective (process) and that they make a difference (outcome).
We also want to help the third sector evaluate their collaborative work. They might be collaborating with service users to deliver a service. They might be a partner in someone else’s collaboration – such as a local authority.
Here are four observations from me:
Collaboration should achieve greater benefits than doing something ourselves. That should go without saying! But sometimes collaboration is presented as an end in itself. How many collaborations genuinely ask at the start “what can we achieve together that we can’t achieve alone?”
We need to understand barriers and enablers to collaboration. If we understand ‘what works’ generally in enabling successful collaboration, we might be able to evaluate our collaboration on the extent to which common barriers are addressed and common enablers are present.
Two enablers in my experience are good facilitation and good methods. A facilitator might be external, or the person convening the collaboration. Whoever is facilitating or convening, there are issues about the effect of that person’s behaviour on the collaboration, balancing ownership with getting the job done well and managing power dynamics. In addition, activities, meetings and so on that happen as part of collaboration need appropriate and effective methods. So it can be useful to evaluate the effectiveness of facilitation and methods in enabling successful collaboration.
Finally, I have observed tensions between ‘top down’ and ‘bottom up’ in achieving change through collaboration. ‘Top down’ is when people are told to collaborate so the collaboration may start from a genuine place. But it can mean there are resources and drivers that help change to happen. ‘Bottom up’ means people are in the room because they want to be and the collaboration might be more genuine. But people outside the room might not care and there might be fewer resources to enable the change. What this means for evaluation is that the measurement of the outcomes of collaboration needs to take account of the power and nature of that collaboration.
So in summary, when evaluating collaboration you should think about evaluating:
- Process: How well the collaboration itself works for participants
- Outcome: What difference the collaboration makes as a result of the work it does
ESS and our partners have some existing relevant resources:
- Why Bother is a workbook for third sector organisations about involving the people they work with in evaluation. There are ideas and tools relevant to broader collaborative work.
- The Scottish Third Sector Research Forum Partnership Compass brings together evidence from third sector research about positive partnerships that could be helpful in understanding and measuring formal collaboration.
- The Scottish Third Sector Research Forum is also working on a paper that draws from third sector research about successful co-production in the third sector. Building on that ESS will be looking in the coming year at how to evaluate co-production.
- There is learning about collaboration on What Works Scotland’s website.
We would welcome feedback on this blog and what, if anything ESS could do to help you measure collaboration. So please get in touch!
