Detect AI-generated Images & Deepfakes (Part 1)

Source
Source
Source (The fake one is generated by StyleGAN)
Source

Design & Implementation Flaws

Design and implementation usually come with shortcomings and mistakes. For example, the instance normalization method used in StyleGAN often triggers blob artifacts and color bleed in generated images. This reveals the fake images easily.

Deepfakes Review

In Deepfakes, step ① below builds a common encoder to encode the latent factors of pictures for two different persons. In steps ② and ③, it builds two separate decoders to reconstruct the first and second photo respectively. To reconstruct the image correctly, the encoder must capture all the variants in a person’s photos, i.e. the latent factors that apprehend information like the pose, the expression, illumination, etc…

Source: Reddit
OpenCV (left), Right

Deepfakes flaws

Blurry

Source (left: the face is swapped, right: a real person in the video)
Created with Adobe After Effects & FakeApp by from Jordan Peele and BuzzFeed
Modified from source
Source
Source (right: the original impersonator)
Generated from Deepfakes
Paul Rudd & Jimmy Fallon
Source
Steve Buscemi on Jennifer Lawrence video
Source
Source
Source
the right one is fake
the left one is fake
Source

Politician & celebrities

Face shape and aspect

High-budget productions

The term “high-budget” production in this article does not necessarily mean projects spending tons of money. In this article, we actually refer to projects that have the right know-how people, decent computer graphics card, and a reasonable amount (days) of time to train the model. Collecting, selecting and cleanup of the training dataset is critical to the quality of the project. It is not hard to gain professional knowledge either. There are tons of online tutorials and free tools. You may need some trials and errors but no AI knowledge is needed. (Even AI knowledge may help, many guides will give you enough suggestions.) And post-production manual manipulations are often applied to produce the top-quality videos. Many people with video editing experience can learn the whole process quickly.

Breaking Bad

Let’s check out another video.

Deepfakes in Politics

With the media attention on Deepfakes, the abuse of Deepfakes in politics is still relatively small in 2020. More likely, it will be used as a last-minute surprise rather than a daily attacking mechanism. Many existing political Deepfakes videos come with a disclosure that they are created with Deepfakes (like the ones below). But this can change when software like Reface, FaceSwap, and DeepFaceLab, are getting more popular by the general public.

  • If you pay close attention to the teeth, you will find the rendering is wrong once a while.
  • Some area of the face is more bury compared with other parts.
  • The movement in the chin and the edge area stood out compared with its background.

Low-budget productions

We tend to believe what we want to believe. A fake video on Nancy Pelosi was circulated around the Internet with slurred or drunken speech. This low-quality reproduction is not created by Deepfakes. Instead, the view is slowed down by 25% and the pitch is altered to make like she is slurring her words. The lesson learned here is low-quality fake videos can widely distribute also. Contents are pushed in the social platform by engagement algorithm. None of them passes through any journalist standard. So do check the source carefully. Information from social media is usually a bad source of information.

Source

More

Deepfakes is just one of the many approaches to generate fake videos. Part 2 looks into more academic approaches in this area first.

Credits & References

Top 10 Deepfake Videos

Photos Credits

Head scarf

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store