publication . Report . Preprint . 2018

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

Korshunov, Pavel; Marcel, Sébastien;
Open Access English
  • Published: 20 Dec 2018
  • Publisher: Idiap
Abstract
It is becoming increasingly easy to automatically replace a face of one person in a video with the face of another person by using a pre-trained generative adversarial network (GAN). Recent public scandals, e.g., the faces of celebrities being swapped onto pornographic videos, call for automated ways to detect these Deepfake videos. To help developing such methods, in this paper, we present the first publicly available set of Deepfake videos generated from videos of VidTIMIT database. We used open source software based on GANs to create the Deepfakes, and we emphasize that training and blending parameters can significantly impact the quality of the resulted vide...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition
25 references, page 1 of 2

[1] H. Allcott and M. Gentzkow, “Social media and fake news in the 2016 election,” Journal of Economic Perspectives, vol. 31, no. 2, pp. 211-236, 2017.

[2] P. Isola, J. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017, pp. 5967- 5976.

[3] I. Korshunova, W. Shi, J. Dambre, and L. Theis, “Fast face-swap using convolutional neural networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017, pp. 3697-3705.

[4] Y. Nirkin, I. Masi, A. T. Tuan, T. Hassner, and G. Medioni, “On face segmentation, face swapping, and face perception,” in 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), May 2018, pp. 98-105.

[5] H. X. Pham, Y. Wang, and V. Pavlovic, “Generative adversarial talking head: Bringing portraits to life with a weakly supervised neural network,” CoRR, vol. abs/1803.07716, 2018. [Online]. Available: http://arxiv.org/abs/1803.07716

[6] A. Ro¨ssler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics: A large-scale video dataset for forgery detection in human faces,” CoRR, vol. abs/1803.09179, 2018. [Online]. Available: http://arxiv.org/abs/1803.09179 [OpenAIRE]

[7] J. Thies, M. Zollhfer, M. Stamminger, C. Theobalt, and M. Niener, “Face2face: Real-time face capture and reenactment of RGB videos,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 2387-2395.

[8] A. Agarwal, R. Singh, M. Vatsa, and A. Noore, “Swapped! digital face presentation attack detection via weighted local magnitude pattern,” in 2017 IEEE International Joint Conference on Biometrics (IJCB), Oct 2017, pp. 659-665. [OpenAIRE]

[9] O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference, 2015. [OpenAIRE]

[10] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 815-823. [OpenAIRE]

[11] P. Korshunov and S. Marcel, “Speaker inconsistency detection in tampered video,” in European Signal Processing Conference (EUSIPCO), Sep. 2018. [OpenAIRE]

[12] J. Galbally and S. Marcel, “Face anti-spoofing based on general image quality assessment,” in 2014 22nd International Conference on Pattern Recognition, Aug 2014, pp. 1173-1178.

[13] D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,” IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746-761, April 2015. [OpenAIRE]

[14] D. Bitouk, N. Kumar, S. Dhillon, P. Belhumeur, and S. K. Nayar, “Face swapping: Automatically replacing faces in photographs,” ACM Trans. Graph., vol. 27, no. 3, pp. 39:1-39:8, Aug. 2008. [Online]. Available: http://doi.acm.org/10.1145/1360612.1360638

[15] N. M. Arar, N. K. Bekmezci, F. Gney, and H. K. Ekenel, “Real-time face swapping in video sequences: Magic mirror,” in 2011 IEEE 19th Signal Processing and Communications Applications Conference (SIU), April 2011, pp. 825-828.

25 references, page 1 of 2
Abstract
It is becoming increasingly easy to automatically replace a face of one person in a video with the face of another person by using a pre-trained generative adversarial network (GAN). Recent public scandals, e.g., the faces of celebrities being swapped onto pornographic videos, call for automated ways to detect these Deepfake videos. To help developing such methods, in this paper, we present the first publicly available set of Deepfake videos generated from videos of VidTIMIT database. We used open source software based on GANs to create the Deepfakes, and we emphasize that training and blending parameters can significantly impact the quality of the resulted vide...
Subjects
free text keywords: Computer Science - Computer Vision and Pattern Recognition
25 references, page 1 of 2

[1] H. Allcott and M. Gentzkow, “Social media and fake news in the 2016 election,” Journal of Economic Perspectives, vol. 31, no. 2, pp. 211-236, 2017.

[2] P. Isola, J. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017, pp. 5967- 5976.

[3] I. Korshunova, W. Shi, J. Dambre, and L. Theis, “Fast face-swap using convolutional neural networks,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017, pp. 3697-3705.

[4] Y. Nirkin, I. Masi, A. T. Tuan, T. Hassner, and G. Medioni, “On face segmentation, face swapping, and face perception,” in 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), May 2018, pp. 98-105.

[5] H. X. Pham, Y. Wang, and V. Pavlovic, “Generative adversarial talking head: Bringing portraits to life with a weakly supervised neural network,” CoRR, vol. abs/1803.07716, 2018. [Online]. Available: http://arxiv.org/abs/1803.07716

[6] A. Ro¨ssler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, “Faceforensics: A large-scale video dataset for forgery detection in human faces,” CoRR, vol. abs/1803.09179, 2018. [Online]. Available: http://arxiv.org/abs/1803.09179 [OpenAIRE]

[7] J. Thies, M. Zollhfer, M. Stamminger, C. Theobalt, and M. Niener, “Face2face: Real-time face capture and reenactment of RGB videos,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp. 2387-2395.

[8] A. Agarwal, R. Singh, M. Vatsa, and A. Noore, “Swapped! digital face presentation attack detection via weighted local magnitude pattern,” in 2017 IEEE International Joint Conference on Biometrics (IJCB), Oct 2017, pp. 659-665. [OpenAIRE]

[9] O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference, 2015. [OpenAIRE]

[10] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 815-823. [OpenAIRE]

[11] P. Korshunov and S. Marcel, “Speaker inconsistency detection in tampered video,” in European Signal Processing Conference (EUSIPCO), Sep. 2018. [OpenAIRE]

[12] J. Galbally and S. Marcel, “Face anti-spoofing based on general image quality assessment,” in 2014 22nd International Conference on Pattern Recognition, Aug 2014, pp. 1173-1178.

[13] D. Wen, H. Han, and A. K. Jain, “Face spoof detection with image distortion analysis,” IEEE Transactions on Information Forensics and Security, vol. 10, no. 4, pp. 746-761, April 2015. [OpenAIRE]

[14] D. Bitouk, N. Kumar, S. Dhillon, P. Belhumeur, and S. K. Nayar, “Face swapping: Automatically replacing faces in photographs,” ACM Trans. Graph., vol. 27, no. 3, pp. 39:1-39:8, Aug. 2008. [Online]. Available: http://doi.acm.org/10.1145/1360612.1360638

[15] N. M. Arar, N. K. Bekmezci, F. Gney, and H. K. Ekenel, “Real-time face swapping in video sequences: Magic mirror,” in 2011 IEEE 19th Signal Processing and Communications Applications Conference (SIU), April 2011, pp. 825-828.

25 references, page 1 of 2
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Report . Preprint . 2018

DeepFakes: a New Threat to Face Recognition? Assessment and Detection

Korshunov, Pavel; Marcel, Sébastien;