Breakthroughs in Shared Measurement and Social Impact takes a refreshing approach to investigating impact and knowledge sharing. Moving away from closed, single organisation evaluation models, the approach encourages organisations working toward the same or similar goals to look at impact on a field, issue or sector-wide basis.
Developed by the Foundation Strategy Group (FSG) the paper researches several shared, online platforms that dozens or even hundreds of non-profits submit data to, in order to assess their collective impact. All of the organisations the report looked at were linked through strong offline networks that focus on issue-based initiatives, mostly social change service-based organisations working in the areas of housing, health, education etc. The paper’s authors have also set up the Collective Impact Forum which includes practical tools and guides to help organisations apply a collective impact model.
Despite the different focus to Video for Change organisations (on services rather than communication/media), there remains much to learn from this research in terms of thinking about how we may adopt a collective model of impact.
Most importantly, the approach dispels the superhero model of social change, whereby a single actor is charged with (potentially) creating a monumental new advance that will massively disrupt the landscape. This individual approach has been predominant in the film/video impact measurement space, where single films are seen as having the ability to radically shift the dial on an issue on a national or even international level. Such success, whilst occasionally realised, represents only a small fraction of social change films, and these are most often films with large outreach budgets and very well connected directors/producers. The effect of the long tail of smaller-scale video production and consumption is cumulative and collective, yet these kinds of videos may ultimately have greater impact and have more important learnings.
The vast majority of social change initiatives, the FSG report notes, contribute to a collective pool and their individual impact can be in only a limited capacity: “No single non-profit can solve the problems they seek to address alone”. The same can be said for a film-maker or video-maker or indeed any communication initiative. These communication initiatives never exist in a vacuum and rarely, if ever, can claim to have achieved clear and measurable impact in complete isolation.
This is not rocket science however, it is social change. Perhaps the necessity of articulating the need for collective rather than individual impact models is a result of the often individualist NGO, film and funder cultures. FSG certainly believes this is the case, and that the foundation model of finding outstanding organisations to deliver breakthrough solutions misunderstands the complexity and scale of the issues at hand.
A Video-Based Collective Model for Measuring Impact?
FSG’s approach to measuring collective impact creates some challenges if we try and apply this kind of thinking to a medium, in this case video, rather than to a specific issue. The film impact space already has this challenge since existing models tend to attempt to measure the impact of a particular piece of media, rather than to look at impact across the issue (eg. climate change), due to the multiple and varied contributions of others.
However, an approach that usefully measures the impact of multiple, video-based initiatives is not unfeasible and this is, in fact, a core question we are exploring with our video for change research project. A collective impact model based on a medium rather than an issue has not been attempted before according to Srik Gopalakrishnan, FSG’s Director of Strategic evaluation when I spoke with him about this.
Our research has so far suggested that to develop such a shared impact Video for Change framework we need to first understand the commonalities within each group’s theory of change. From there a shared model for change can be developed. Srik pointed out that the initial indicators on the last page of the Video For Change research scoping report provide a good start for thinking about what might be shared. Once a shared framework is in place, more specific indicators can then be added. These may not cut across all projects but could be modularised and made use of as each organisation requires.
The process involved in developing such indicators for impact is a key goal of our current Video for Change impact research project. Bringing groups together to explore their needs and shared aspirations for change, so as to develop ways for sharing knowledge and producing a common language is quite a challenge, particularly for groups accustomed to working on their own.
Creating Shared Platforms
The recent ability to harvest and aggregate large amounts of data online appears to be one driver of the collective impact model as all the projects center around online platforms. These kinds of online data sharing platforms were not feasible 10 years ago.
Whenever new software tools are created, there is always a great risk of producing expensive shelfware! The challenge, in this case, is to design a lightweight, usable and secure system that is attractive and employed in a consistent manner. While partner organisations might often like the idea of a shared platform, in the end, few make use of those created. The platform examples cited in the Shared Impact report often cost millions of dollars and service hundreds of organisations. Ongoing support for these systems, both technical and content-focused, is critical, and this is why many of the successful platforms examined employ dozens of full-time staff.
For the Video for Change field there are two key audiences a Shared Impact Platform would need to cater to: internal members of the video4change network, and the more diverse plethora of video-producing groups that exist across a spectrum from formal production houses to informal collectives and solo creators.
The vast majority of producers in the Video for Change field have very limited capacity to design for and measure impact. Their films are often produced on a shoestring budget and they have even less money for impact evaluation if they even think about it at all. In these cases, any platform created needs to be intuitive, unburdensome and produce the maximum possible return. No easy task: but not one that is not necessarily out of reach.
The advantages a shared platform would bring to the Video for Change field include the ability to: identify sector-wide trends, provide successful case studies, document impact at the field level, allow funders and others to place achievements in context, drive collaborative problem solving, build shared learning environments, pool resources, and add credibility to the field as a whole.
Producing these benefits fits very well with the mission of the video4change network. How feasible this may be, in terms of available resources to create and sustain a shared platform is yet to be determined. More critical still is to ask whether there is a shared desire and need for such a platform in the first place. Within the video4change network, the desire for a shared model for measuring impact has already been clearly articulated; however, we are yet to examine whether a shared and collective impact model is a realistic proposition. Whether this desire for a collective impact model exists more broadly across the Video for Change field more broadly requires more investigation.
We’d love to hear your thoughts.