Synthesizing realistic audio, images, and videos using algorithms has always been essential in Signal Processing, Computer Graphics, and Computer Vision. When using pre-artificial intelligence (AI) tools, this process is usually lengthy, costly, and technically demanding for ordinary users. However, the rapid developments in AI technology in recent years have significantly lowered the resources, time, and technical expertise required to create compelling fakes. Such developments first caught the public’s attention in late 2017 when a Reddit account called ‘DeepFake’, a portmanteau of deep learning and fake media, began spreading pornographic videos with transplanted celebrity faces created using a Deep Neural Network (DNN)-based algorithm. Since then, more sophisticated algorithms that synthesize realistic audio, images, and videos have emerged, along with a plethora of open-source software tools and commercial services. ‘Deepfake’ is also used more broadly as a term that refers to any impersonating media created or edited by deep learning algorithms.
Deepfakes are just the tip of the iceberg of this troubling trend. By creating illusions of an individual’s presence and activities that did not occur in reality, deepfakes can cause real harm when they are weaponized. For instance, a fake video showing a politician engaged in an inappropriate activity may be enough to sway an election if released close to voting day. A falsified audio recording of a high-level executive commenting on her company’s financial situation could send the company’s stock into freefall. Using a synthesized realistic human face as the profile photo for a fake social platform account can significantly increase the impact of deception. An online predator can masquerade as a family member or friend in a video chat in order to lure unaware victims. Left unchecked, deepfakes can escalate the scale and danger of online disinformation and fundamentally erode society’s trust in digital media.
Recent developments in generative AI are significantly boosted by the availability of large-scale language and image generation models, such as the OpenAI Generative Pre-trained Transformer (GPT) family, DALL-E, and Midjourney. Catching the public’s imagination of the superpower of AI technology and hinting at the prospect of Artificial General Intelligence, these developments have also opened up new opportunities and challenges in the making of deepfakes. These developments have focused on three main directions: i) making the models more powerful, and capable; ii) making them more accessible; and iii) giving users more control over the style and content of the generated media through detailed text prompts.
One of the most significant advancements in generative AI is the increased power and capability of the models. This has been made possible by the availability of vast volumes of training data that enable the models to learn complex patterns and generate high-quality output. These models can generate realistic and complex images, videos, and audio that are almost indistinguishable from those created by humans. The applications of these models are vast, ranging from generating realistic images for virtual environments to creating realistic voices for virtual assistants.
Another important direction in generative AI is the accessibility of the models. Many tools now provide web-based interfaces that require little to no coding and/or installation effort, making it easier for non-experts to use and benefit from these models.
Finally, developing AI tools that give users more control over the style and content of the generated media through detailed text prompts is another important direction. This enables users to specify the desired output style and content by providing text prompts that the AI model can use as input. This can be useful in generating customized content for marketing campaigns, creating personalized content for social media, or generating realistic simulations for training purposes.
The main forms of current deepfake making methods are summarized in three categories: images, video, and audio/voice.
Images
A quintessential example of deepfakes is the highly realistic images created from the generative adversarial network (GAN) models. A GAN model consists of two DNNs trained in tandem. The ‘generator’ synthesizes images, and the ‘discriminator’ differentiates synthesized images from real ones. In training, the two DNNs compete: the generator tries to create more realistic images to defeat the discriminator, while the discriminator attempts to improve the classification accuracy. The training ends when the two DNNs reach an equilibrium. The generator is then used to create realistic images from input white noises.
Recent works, known as StyleGANs, have demonstrated the superior capacity of GAN models in generating high-resolution and realistic human faces. GAN models can also be used to edit or transfer the attributes and expressions of faces. A more recent variant of the image generation model is known as the diffusion model. Like the GAN model, the diffusion model creates realistic images from input noise. However, the training mechanism of the diffusion model is different. It uses a Deep Neural Network to simulate the physical process of diffusion, in which a structured signal is slowly dissolved into thermal-dynamical equilibrium through the stochastic process of diffusion—imagine a drop of ink dissolving in a cup of water. The deep neural network is then used as the reverse model to transform input noise into a structured image. Diffusion models have led to the state-of-the-art generation of realistic human faces with software systems such as Stable Diffusion being widely used.
Videos
The original namesake of ‘DeepFake’ is face-swap videos generated using an image-to-image translation framework. Specifically, the faces of a target are replaced by the faces of a donor synthesized using the auto-encoder (AE) model. The AE model consists of two DNNs, encoder and decoder, trained using the target and the donor's faces. The encoder retains the target's facial expressions and head poses while the decoder combines these with the target's identity. This approach of synthesizing face-swap videos has been mainstreamed through open-source software implementations on GitHub (github.com).
There are also techniques to create videos of upper-body reenactment, and whole-body motions. Other variants of this method are those which animate a single face image from a driving video of another person. Examples of such methods are Reenact GAN and First Order Motion. These methods use Deep Neural Network models to transfer the facial movement from the driving video to the input face image to create a video sequence of the subject in the image with the same facial movement. Several start-ups have commercialized the making of face-swaps or reenact videos (Synthesia and Canny AI, for example).
Audio/Voices
DNN models have also been used to create realistic, synthetic human voices. Two types of deepfake audio differ in their input modality. The text-to-speech models (e.g., Parrotron and Spectron) convert an input text to the target’s voice, while the voice conversion models use a source person’s voice as input. The underlying speaker-adaptive neural speech synthesis system usually includes i) acoustic modeling models, ranging from simple spectrograms to the more sophisticated neural speaker and style embedding (e.g., Tacotron and its variations); ii) vocoders such as WaveNet or WaveRNN for speech waveform generation; and iii) conversion algorithms based on auto-encoder or GAN models. Several commercial companies, such as Lyrebird, Respeecher, Murf.ai, ElevenLabs, and Dessa, provide voice imitation as a service.
Multimodal Generation
Text-to-image generation has significantly improved in the past two years with recent advancements in attention-based transformer and diffusion models. Several large-scale language-image models have been developed, including the DALL-E model proposed by OpenAI in 2021, which uses an autoregressive transformer to generate high-quality images on the MS-COCO dataset without any training labels. Other models such as CogView, Parti, Make-A-Scene, and most recently, MidJourney, have also used autoregressive transformer models for text-to-image generation. In 2022, an updated version of DALL-E, DALL-E2, was developed using a diffusion model with CLIP image embeddings, enabling it to produce higher quality and more diverse samples more efficiently. Other models, such as GLIDE, Stable-Diffusion, and Imagen, have also used diffusion models to improve text-to-image synthesis.
These powerful text-to-image synthesis models have inspired several studies focused on developing text-guided image editing models, including DiffEdit, Prompt-to-prompt, Null-text Inversion, Imagic, and Muse. These models perform local semantic editing to an image based on text input with the desired edit and an optional scene layout (segmentation map). However, their optimization often maximizes similarity to the original image while maintaining the ability to perform meaningful editing on local regions. This type of entire synthesis can be easily identified if seen in the training data.
AI methods that create lip-synch videos with input voices for any video have become increasingly popular in recent years. These methods aim to generate realistic mouth movements that synchronize with the audio of a person speaking in a given video, allowing the video to be dubbed or re-voiced in a different language. One common approach to lip-synch video generation is deep learning-based models that can learn the relationship between audio and mouth movements. These models usually involve training on large datasets of audio-visual pairs to learn the mapping between the audio and visual domains. Other approaches involve using facial landmark detection techniques to predict the movements of the lips based on the audio input. Recent advancements include using neural machine translation techniques to enable lip-synch generation in different languages and integrating natural language processing techniques for more accurate and contextually relevant lip-synch generation.
Conclusion
Although the future of deepfakes is hard to predict, one thing is certain, the technology will continue accelerating towards producing more realistic content more efficiently and more cost-effectively. Various stakeholders will need to take action to control the potential misuse of such tools for disinformation. The most direct measure is for the service/tool providers to regulate the uses and watermark the generated contents so that they can be traced and exposed more easily when spread on social media. Platform companies are also responsible for filtering and limiting the viral spread of synthetic content, and the associated orchestrated disinformation campaigns. Public media can help users to expose disinformation through fast response times in fact checking and debunking. Users also need to increase their own awareness and knowledge of synthetic media and be encouraged not to spread unreliable information. Lastly, government agencies can play a critical role in guiding national research strategies to invest more into researching countermeasures to deepfakes while focusing legislative efforts to control the problem.
