The Fledgling Fund is a private foundation 'driven by the passionate belief that film can inspire a better world'. The list of films it's supported is truly impressive, lots by and about women. Here's just a few– Brave Miss World, Budrus, Girl Rising, Leaving the Life, Miss Representation, Mothers of Bedford, Saving Face, The Invisible War, The Light in Her Eyes. Other significant projects include The Mask You Live In and Seed: The Untold Story. And there are many many more.
I love it that each project on the Fledgling Fund site has its own social impact page, powered by Sparkwise. On those pages I can read a synopsis and about the filmmakers, see the trailer, get a snapshot of the project's communities and social media reach, find out who its main supporters are. Learn about how I can become a supporter and/or viewer. There are graphs! There are maps even!
I can't show you a full screenshot of any of the pages, many of them rich and complex. But here's a partial example, from Seed: The Untold Story, chosen because I'm raising bee-loved plants from untreated seeds at the moment and, years ago, I was deeply affected by New Zealander Barry Barclay's The Neglected Miracle (1985), which covers similar themes.
The Fledgling Fund funds only docos, as far as I can see. But there are of course narrative films that aim for social engagement, Sophie Hyde's wonderful and multi-award-winning narrative feature 52 Tuesdays for instance. Its website has a front page not unlike a Sparkwise page and its 'My 52 Tuesdays' is–
...a worldwide participatory project where people build and share a unique portrait of a year in their lives. Every week, every Tuesday, a question is posed to everyone involved – you answer by writing down your response and taking your photo with it. See answers to the same questions from your closest friends and creative people all over the world. Share your photos or keep them private. It is a project about you, set in time, distinctly personal and lovingly communal – but only if you choose it to be so, because ultimately it is a project about choice. And only on Tuesdays. How much will you share?Branded entertainment does it too. Miranda July's contribution to Miu Miu's Women's Tales, Somebody, has an associated app which seems like a lot of fun. It allows users to send a message to a friend and have it delivered, in person, by a stranger who is geographically near the recipient. I was thinking that it might be problematic to use Somebody in Welly, with its small population but 'The artist advises that Somebody works best with a critical mass of users in a given area; any social gathering can become a Somebody hotspot'. In the Huffington Post, Naz Riahi reported on her experience–
I was one of the first handfuls of people to download and use the app, zealously running all over Brooklyn and Manhattan to act out and deliver messages to strangers. The experience was exhilarating...It was a whole new way of communication and connection.(Miranda July's website is content rich, too.)
To me, this thoughtful letter from the women at the Fledgling Fund, about whether and how to assess the impact of creative media, is both timely and useful, for all of us. I like their Crafting An Impact Assessment Plan: Some Questions To Get You Started, too, which follows the letter.
This is also a Happy Birthday post for Ruth Gerzon and for her life's work, making a social impact.
September 2, 2014
To Our Community,
Over the past several months, Fledgling has participated in many discussions with fellow funders, filmmakers, practitioners and others who are all wrestling with the question of whether and how to assess the social impact of creative media, and especially documentary film. Emerging tools and platforms, like The Participant Index (TPI), Harmony Institute’s Story Pilot, Sparkwise and ConTEXT, are attempting to capture social impact in different ways, many using techniques that rely in part on access to big data. We expect others will follow. Like many of you, we have reviewed these platforms and tools as they have evolved and have listened to the robust debate they have sparked, revealing the concerns and push back from many filmmakers and others about this increased focus on 'measurement'. This came through loud and clear in Aggregate’s survey of True/False filmmakers released in July in which 62% of respondents answered “no” to the question “Do you think there should be metrics to measure the social change created by a film?” In light of this, we have been thinking a lot about what all of this means for Fledgling, for our grantees, and for the field.
We recognize that not all documentaries have social change goals, rather they exist as works of art to be consumed and enjoyed, not to be assessed in the context of social impact. We are referring in this letter solely to films and filmmakers with social change as a key goal and especially those that have decided to build outreach and engagement campaigns around their films to deepen the connection that audiences have with the social issues presented. As a funder focused on supporting social impact strategies for documentaries, we have a special interest in how these new tools and platforms could help us and our grantees learn. We also are aware that as a funder we have a special responsibility to be thoughtful about our approach to assessing impact.
Overall, we believe that the proliferation of these tools and platforms and the debate they have spurred is healthy and important for the field. There are now more tools in our collective toolbox to document, track, and communicate the social impact of documentaries and their campaigns, and that is a good thing. And, when these insights are used in real-time throughout a campaign’s evolution, they can help shape and strengthen campaigns as they unfold. They also allow project teams to share information with key partners along the way, deepening those relationships. However, we know that not every tool is appropriate for every film and engagement campaign and it is highly unlikely that one tool will capture the complete scope of impact.
When we consider how the various tools apply to our work and the work of our grantees, there are a number of things that are important to keep in mind. First, each tool is developed with a specific purpose and definition of impact by its creators. For example, TPI focuses on the “relationship between social issues, emotional involvement in entertainment, and social actions an audience is inspired to take after consuming entertainment” and ultimately creates a numerical impact score for a project. To understand the value of this Index, it is important to understand the methodology behind it – how social action is defined, what assumptions are made, how information is gathered and over what timeframe. Importantly, we also need to decide whether this methodology captures the kind of social change envisioned for a project before deciding whether or not a project’s score is meaningful and if so how. At Fledgling, we are not interested in the overall numerical score. But we see value in understanding what specific parts of the Index reveal because we can map that against the project’s unique goals, its target audiences and its strategy for change.
Second, many of these emerging tools rely on big data sets that capture online activity. Many use complex algorithms and web crawling tools to determine how a piece of media or a film has influenced a conversation and public dialogue. For some projects, these tools can reveal incredibly interesting information, particularly for those that are seeking to influence a broad national conversation around a particular issue or affect culture change. Ultimately, the more information we have the better and these tools can add value to understanding the impact of film.
However, this style of analysis can capture only very specific aspects of a project’s social impact. It will inevitably miss much of the crucial long-term, off-line, grasstops and/or deeply personal impact that a project can have. For example, a new conversation around a dinner table about an important topic, a moment of personal healing, a quiet donation made to a cause after a viewing and discussion, or a closed-door influencer screening that could have policy impact in the longer term are all incredibly important ways in which documentaries have impact beyond what can be captured by big data. Likewise, a powerful community screening that leads to deep change at the community level or a project that does not have broad reach but has a lasting impact on specific target audiences might be missed. We, the social issue documentary field, often put great pressure on filmmakers to change the world, when changing someone’s world should be considered equally important. This does not mean that tools that rely on big data are not useful, only that we need to be clear about what they can and cannot capture. Indeed, for many outreach and engagement campaigns, these off-line and personal moments of change are the most valuable and most indicative of the deeper impact a film and its campaign can have.
We encourage our grantees to create an evaluation plan that is clearly linked to their own distinct impact goals and strategy and relies on different kinds of data that can help track key indicators of their progress over time. While quantitative or numerical data may be easier to come by and compare, qualitative data, which is more descriptive and observational, in many cases is more appropriate to capture the complexities of social change. With this quantitative and qualitative data, filmmakers can create a story (which we know they can do!) about the impact they have had. This 'impact story' allows for deep context that cannot be achieved with numerical data alone. One of the reasons we supported the development of Sparkwise as a platform for understanding and sharing social impact data and subsequently decided to use it for our grantees is its capability to show both quantitative data (number of Twitter followers, Facebook fans, web searches etc.), qualitative reports of impact as well as self-generated stories. Our grantees create Sparkwise boards that are customized to their specific social change goals and they can share those campaign goals as well as rich information about the issues and existing social movements that provide context to the numbers
We should also note here that it is important that any evaluation methodology distinguish between 'attribution' and 'contribution'. The vast majority of social issue documentaries and their engagement campaigns are entering into a community (however small or large) of activists, leaders, organizations and coalitions that have laid groundwork long before the films and campaigns were conceived and they will be there for many years continuing to build the movement. This critical work must be acknowledged - just another reason an element of 'story' is so important in this evaluation work - so that this relationship can be explained.
Think of the case studies that have been shared by BritdocMedia Impact Funders, Active Voice, Harmony Institute, Fledgling and others that rely on both quantitative and qualitative information to create a more complete picture of the impact that a film achieved, sometimes over a long period of time. Often these kinds of 'impact stories' offer a more complete and useful tool for understanding the power of social issue documentary. We recognize that creating these case studies can be time-consuming and resource intensive, particularly if done retroactively. However, we believe that filmmakers who are trying to achieve social change, impact producers and importantly funders need to make a commitment to supporting data collection and impact tracking from the moment that campaign planning begins. This front-end quantitative and qualitative data collection will not only inform the campaign but also make impact reporting more efficient and accurate. We are not calling for extensive and expensive evaluation strategies – simply for thoughtful collection of diverse data throughout the life of a campaign that will show broad sample evidence of the ways that the project is affecting the world, communities or individuals, according to stated goals.
We should also point out here that we believe 'measuring impact' is not a good description of what we are talking about, as it seems to imply a comparison between other film projects/campaigns or against arbitrary benchmarks. When we think about data collection and assessment we are seeking to understand the success a project has in meeting its stated goals. The importance of the link between a project's goals and strategy and its impact assessment plan cannot be understated.
Revisiting the Aggregate survey, we wonder how respondents interpreted the word 'metrics' in the question noted above. If they thought the question was about finding a set of numbers to measure and compare the social change of films, then Fledgling would stand strongly with the 62% that answered ”No”. But, if the question is about methods for understanding the social change of films and engagement campaigns and those methods include both quantitative and rich qualitative data - relying on big and small data and deep context, then Fledgling would say emphatically “Yes”!
Our goal with this letter is to share our thinking on the subject of if and how to assess the impact of documentaries and their associated outreach and engagement campaigns. In short, we are thrilled that this field is making solid progress in our collective understanding of how films and engagement campaigns make change in the world. The emerging tools increase the options for filmmakers and offer insights into how their creators define impact. Ultimately though, the utility of these tools for individual projects depends in large part on a project’s goals and its strategy for change. We will continue to encourage and support our grantees to tell the stories of the impact they are having – using a mix of data to provide deep context. We are deeply aware that the impact of this work is incredibly complex, often happens over long periods of time and can not all be captured. However, we have seen over and over again that evidence of impact is incredibly useful and continues to move this field forward.
The Fledgling Fund
Diana, Sheila and Emily
Crafting An Impact Assessment Plan: Some Questions To Get You Started
1) What is your outreach and engagement strategy? What short and long-term outcomes do you want to achieve?
2) Which target audiences to you want/need to reach? What role do you want them to play in your campaign or with the larger movement?
3) What kinds of data can you collect, analyze and share to demonstrate your project’s impact and inform your campaign strategy?
4) How have similar projects demonstrated and shared their impact?
5) Who are the internal and external audiences for this information? Over what time frame do you need to collect, analyze and share this data?
6) What tools, platforms and methodologies can you use to collect, analyze and share impact data? For each tool, platform or methodology, consider the following:
What kinds of quantitative and qualitative data can it capture?
Is this kind of data important to your project?
Can it help you demonstrate impact or provide insights into your strategy?
What will it NOT capture?
What kinds of questions does it ask and over what time frame?
Does the time frame used make sense for your project and the strategy you have outlined?
Has the platform or tool been vetted?
What are the pros and cons?7) What does your overall impact assessment plan look like? What mix of tools, platforms, and methodologies makes sense given your campaign goals and strategy? What are the costs associated?
website
52 Tuesdays
Miranda July
website
You're very welcome, Gujjar. As you can probably tell, I love the Fledgling Fund and want all the world to know about it!
ReplyDelete