You may have seen a video of Nicholas Cage’s face transposed onto another character doing the rounds on the internet recently. No, we don’t mean a clip from the movie Face/Off. These are scenes from various movies in which Nicholas Cage did definitely not play, yet they look very convincing indeed. How did they pull this off?
Well, the answer isn’t “witchcraft”, but a
technology that has been dubbed “deepfake” and it’s causing quite a ruckus in
various industries and communities.
What is a “Deepfake”?
The word “deepfake” comes from the term “deep
learning” and of course “fake”. Deep learning is a specialized branch of
machine learning, which is again part of
the overall area of Artificial Intelligence.
With the dramatic rise in computing power and
newly-discovered ways for computers to process and analyze massive amount of
data from the real world, computers can now suddenly do things most of us could
never imagine. Deepfakes apply this technology to synthesize human images,
creating photos or videos of things those people never did and never said.
How Does Deepfake Technology
Work?
Deep learning, which underpins deepfake
methods, describes the modern application of neural net simulation to massive
data sets. Neural nets are not a new concept or technology, but until modern
times they have been pretty rudimentary.
An artificial neural net simulates the learning process that happens in biological brains, at least to some extent. When you learn or otherwise deal with the outside world, the connections between your brain cells change.
They form circuits and logical structures, strengthening some connections and weakening others. As you master something, like learning to drive or play tennis, those brain circuits become fast and efficient. Eventually you are so good at something that you don’t even have to think about doing it.
That’s essentially the same thing that happens with a deep learning system. It looks at heaps of examples of something and then becomes progressively better at “understanding it”.
In the case of deepfakes the software looks at examples of the face you want to transpose as well as the video you want to transpose it to. With enough training it can eventually synthesize a face that matches the training data and then seamlessly overlay it on any other face.
What Software is Used to Make
Deepfakes?
There are a number of applications that allow
people to make deepfakes. FakeApp was the first app we know of aimed at giving
normal people a shot at making deepfake content. The website it now defunct and
finding a copy is not easy at all.
Makers of deepfakes have now largely moved on to an application called DeepFaceLab, which is hosted on GitHub and has spawned an endless stream of tutorials on places like Reddit.
Making a Deepfake
This article is not meant to be a tutorial, so
we are going to provide an overview of how deepfakes are made in practice, but
not provide exact steps on how to make one yourself.
There are a few reasons for this, but the main
one is that the legality of making deepfakes is highly-contentious. Just as we
wouldn’t provide exact steps to help you pirate software or do other illegal
activities, we won’t give step-by-step instructions for making deepfakes.
Besides, the actual creator of DeepFaceLab has released a step-by-step video tutorial for the software which anyone can follow along, should they wish to take responsibility for such a risk.
To get a gist of how good this stuff has become, check out this website where a new photograph of a person who does not exist is generated when you refresh the page.
Our goal is to help you understand the
technology, since you are bound to encounter it more and more as time goes by.
With that being said, these are the broad phases of creating a deepfake with
DeepFaceLab.
After downloading and unzipping DeepFaceLab
you are faced with a folder containing many other folders and a series of batch
files.
There’s a folder called “workspace” which
contains the training models, the source videos and the output. DeepFaceLab
works with specific file names and locations, so that the batch files can work.
For example, the source file is always named “data_src” and the destination is
named “data_dst”.
There is no software interface as most people know it. Just a list of numbered batch files representing the steps of the process. First the source and destination videos have their frames extracted as picture files.
Then several analysis steps are run, followed by GPU-based training, where the neural net learns what it needs about the faces in the two videos to build a model. Finally a new composite video is produced.
Misuse of Deepfakes
As we mentioned above, deepfakes are very
controversial. It may violate existing laws in some countries and new laws are
in the works to deal with the technology and its applications.
Deepfakes can be used to create hoaxes, such as a video of a president saying crazy things. It can be used to insert people into pornographic films with the purpose or harassing or otherwise harming them.
As you can probably imagine, you could do a lot of damage with a convincing deepfake and the consequences if you are caught may become much more severe in the near future.
Question Everything
Now that this technology exists and works as
well as it does, it means we have to look at media like videos in a whole new
light. If someone circulates a video on social media of a famous person or
politician saying or doing something controversial, you’ll first have to ask if
the video is even real.
Most poorly-done deepfakes are obviously fake,
once you know what to look for and have seen a few. However, in some cases even
a trained eye may have trouble telling that some sort of CG manipulation is
happening and as the tech improves it will eventually become impossible.