Significant lessons can be learned from centering civilian and citizen concerns—including those of human rights defenders and journalists, globally—in the prioritization of risks, threats, and potential solutions in the areas of deepfakes, synthetic media, and audiovisual generative artificial intelligence (AI). Worldwide, these individuals and communities already confront harms similar to those emerging in the 'age of synthetic media'. However, despite being most at risk, these individuals and communities are marginalized from decisions on these new technologies.
Launched in 2018, WITNESS's 'Prepare, Don't Panic: Deepfakes and Synthetic Media' initiative has aimed to intervene early in the synthetic media ecosystem, focusing on technical infrastructure, emergent tools, digital literacy efforts, and policy and legislative aspects. The work is based on extensive research, industry consultation (including workshops in Europe, South AfricaFootnote 146Footnote 147Footnote 148Footnote 149Footnote 150
Threats and Risks from a Civil Society Perspective
Civil society actors, through the past five years of WITNESS consultations, consistently identify a set of existing harms and potential risks from synthetic media. Women are consistently identified as being particularly vulnerable to threats from synthetic media because of how the technology has enabled new forms of gender-based violence.
Synthetic media, or the existence of it, is used to exercise plausible deniability and dismiss demonstrably true evidence by claiming it is false (also known as the ‘Liar’s Dividend’), or to claim that all content cannot be trusted—often to discredit journalists, activists, and civil society organizations more broadly, along with the trustworthy content they put out to the world. Research participants expressed concern on the use of such claims (or fears) to justify the enactment of laws that limit speech more broadly.
The threat of synthetic media to spur misinformation and incite violence is consistently noted —particularly within existing vectors of rapid spread such as messaging apps—as well as how this can be used by foreign and domestic actors to target groups and communities who are already vulnerable based on ethnicity, religion, political identity, professional role(s) and/or other characteristics.
All of these threats are consistently connected to the existing challenges of media literacy, under-resourced journalistic capacity, and limited access to detection and authenticity tools for both critical civil society actors and individual citizens.
Workshop participants identified that these threat vectors combine with existing threat patterns directed towards civil society and citizens by their own governments in the context of closing civil society space. For example, spreading disinformation targeting civil society, surveilling and harassing of journalists and human rights defenders, and attempting the criminalization of their activities.
In the past year, as generative-AI based synthetic media tools have become more accessible, easier to use and more personalizable, more people have had the ability to engage with them. They have been able to imagine—or experience—how the tools could impact their lives. This shift has resulted in an emerging reevaluation of the risks and potential harms.
As people experimented with synthetic media tools and realized how easy they were to use to create individual content items, produce variants on content, and produce images of real-life events (with limited input data), the challenge of an information ecosystem flooded with synthetic content came up more regularly linked to the inadequacy of the volume of, for example, fact-checking responses. Participants placed these in critical contexts such as elections and public health crisesFootnote 151
Principles for Building Better Civilian Resiliency
In terms of building civilian resiliency to AI-based manipulation and synthesis, a number of core principles emerged through this research.
Prioritize: 1) people globally facing similar harms; and 2) journalists and civil society who support a reliable, trustworthy information ecosystem
A response to deepfakes and synthetic media, as well as the expansion in availability and ease of creation of audiovisual generative AI and deepfake technology requires attention to who is most at risk from both targeted attacks as well as a broader undermining of trust in critical content. In many cases, these same communities and stakeholder groups have already experienced related harms from previous technologies. For example, globally women journalists and public figures are targeted with non-consensual simulated sexual images, while human rights defenders and investigative journalists consistently face claims that their documentation and investigations have been falsified.
Similarly, these same constituencies already face real-world constraints on their capacity to respond. For instance, local journalists, local elections officials, and female and LGBTQI-identified community-level political figures often find themselves targeted, under-resourced, and over-burdened.
Avoid de-historicizing or decontextualizing synthetic media
Although synthetic media is an emerging technology, the threats it poses are not new. As noted above, existing experiences, particularly of vulnerable populations, critical civil society, and media intermediaries, should inform responses to the threats and opportunities of synthetic media. Marginalized populations know the ways in which they are targeted with disinformation, and community-based response strategies and fact-checkers have experience addressing more traditional methods of video and audio manipulation, or ‘shallowfakes’; while social media platforms are already grappling with how to handle satire (a common deepfake usage) on a global level.
Place firm responsibility across the pipeline of foundational model and tech builders, tech deployers, content creators and content distributors (media and social media)
Any solutions require careful attention across the pipeline of how synthetic media is made, from foundational model and tech builders, to the deployers of technology and content distributors. Responses should not place the burden or pressure on end-users to identify synthetic media or disclose their personal usage, in the absence of broader responsibility throughout the pipeline of how synthetic media is created.
For instance, it is not viable to double-down on a media literacy strategy focused on civilian forensic analysis of video. An example of this is the reliance on or promotion of tips that encourage someone encountering an image in their timeline to attempt to spot a potential visual glitch such as a distorted hand, created through the generative process. These tips rely on the current algorithmic ‘Achilles Heel’ and are often quickly remedied by technical progress.
Governments, social media platforms, technology companies, and news organizations all have a role in the development of mechanisms (such as regulation, policies, functionalities, processes, etc.) that proactively tackle threats without placing the responsibility on content creators or consumers alone, and locating responsibility (where relevant) with upstream stakeholders.
Developing a human rights-based technical infrastructure, norms, consistent global platform policies, and laws and regulations is key.
Goverments and regulators can support a range of options at the technical and policy levels that help to establish clear human rights-based guardrails, mandate rights protections, and also pay close attention to critical rights issues around privacy and freedom of expression.
Actions to Support an Informed Digital Citizenry
These prioritized solutions reflect outcomes from WITNESS’s research and require implementation that takes into account the factors identified above.
Media and digital literacy will not be enough, but are still as necessary as ever
Supporting media literacy more broadly is a critical component of societal and governmental responses. This is especially true considering that society is in the early stages of the use of synthetic media and deepfakes, and that so-called “shallowfakes”—where media is decontextualized, lightly edited or miscaptioned—are currently far more prevalent than synthetic content.
Techniques and approaches to synthetic media should not be developed in isolation from broader media literacy approaches or from approaches relevant to shallowfakes. For example, the SIFT approachFootnote 152
One key approach to consider in media literacy campaigns is to avoid adding to the existing hype around generative AI and synthetic media, and mitigating their impact in reducing the ability to trust content. Particularly for vulnerable communities and critical civil society voices, claims of ‘it’s a deepfake’ and the broader ‘nothing can be trusted because anything can be falsified’ are already growing in prominence, and media literacy campaigns should be calibrated to the scale of the synthetic media problem and avoid alarmist rhetoric that compounds this issue.
In tandem with media literacy approaches aimed at the general population it is important to focus on ‘the media’s’ literacy, and journalism’s contribution to neither providing a simplistic educative response focused on visual tips, nor contributing to the panic cycle that is weaponized in some contexts against critical societal voices.
Detection tools should be accessible to those that need it most
Detection tools are part of a solution. In general, current detection tools are not reliable at scale, and require expert input to assess their results. In a number of global cases, the use by the general public of detection tools available online has contributed to confusion and increased doubt around real footage rather than contributing to clarityFootnote 153
Accessibility in these conditions extends beyond technical access to support in how to effectively use the detection tool(s), and to effectively communicate results to stakeholders.
While platforms can play a role in moderating synthetic media content, this should not be an automated process of unnuanced synthesis detection given the unreliability of existing detection models. The reality must be acknowledged that not only is most synthetic content personal communication or non-malicious/harmful, but that content will increasingly be a complex mix of synthetic and non-synthetic, and that detection efforts will have to accommodate for this. However, there is the potential for platforms to provide signals that could support both civil society and media entities doing analysis, as well as the broader media literacy noted above.
Verifiable provenance and watermarks can provide signals to support informed digital participation, but human rights and accessibility concerns should shape the infrastructure and tools
A range of initiatives are exploring how to provide authenticity and provenance signals to consumers of media (for example, initiatives such as the Coalition for Content Provenance and Authenticity, C2PA.org and the Content Authenticity Initiative, contentauthenticity.org). While there are a number of proposals for incorporating watermarking technologies into AI-generated technologies, all of these approaches rely on participation and integration across the pipeline of responsibility (outlined above) from foundational model developers, tool developers, social media platforms, and major news media outlets. They all hold a key responsibility to enable disclosure of how media encountered by citizens is created. Buy-in from social media platforms, major news media outlets and AI model and tool developers is essential as they play a key role in guaranteeing that the verifiable provenance or watermarks are part of, or attached to, the audiovisual content from early on in its lifecycle, but also in making sure that their users and viewers are effectively informed about the nature of the content they are consuming.
Within this context, a core responsibility for democratic government is to ensure that these technologies are deployed with privacy and access centered, and where necessary legislated. Work by WITNESS has identified a range of human rights concerns that are related to authenticity, provenance, watermarking and disclosure approaches—among the most prominent are ensuring that media provenance is not intrinsically or de facto connected to personal identity, and ensuring global stakeholder inputFootnote 154Footnote 155
Synthetic media content moderation at scale still needs contextualized policies and local expertise to deal with threats from synthetic media and protect speech
Synthetic media will increase the amount of content created and shared online. Automating moderation, including by leveraging AI (e.g., detection tools, or provenance and watermarks), will be a facet of responses, however, these policies should be designed alongside impacted groups and based on human rights principles. In addition, there should be clear processes for local experts and experience to be involved in the moderation loop.
One key area of concern is to preserve options for the frequent satirical and parodic uses of synthetic mediaFootnote 156
