Exposing Manipulated Photos and Videos in Digital Forensics Analysis

Sara Ferreira; Mário Antunes; Manuel E. Correia

doi:10.3390/jimaging7070102

journal article Open Access Jun 24, 2021

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

Sara Ferreira

Mário Antunes

Manuel E. Correia

Journal of Imaging Vol. 7 No. 7 pp. 102 · MDPI AG

View at Publisher Save 10.3390/jimaging7070102

Abstract

Tampered multimedia content is being increasingly used in a broad range of cybercrime activities. The spread of fake news, misinformation, digital kidnapping, and ransomware-related crimes are amongst the most recurrent crimes in which manipulated digital photos and videos are the perpetrating and disseminating medium. Criminal investigation has been challenged in applying machine learning techniques to automatically distinguish between fake and genuine seized photos and videos. Despite the pertinent need for manual validation, easy-to-use platforms for digital forensics are essential to automate and facilitate the detection of tampered content and to help criminal investigators with their work. This paper presents a machine learning Support Vector Machines (SVM) based method to distinguish between genuine and fake multimedia files, namely digital photos and videos, which may indicate the presence of deepfake content. The method was implemented in Python and integrated as new modules in the widely used digital forensics application Autopsy. The implemented approach extracts a set of simple features resulting from the application of a Discrete Fourier Transform (DFT) to digital photos and video frames. The model was evaluated with a large dataset of classified multimedia files containing both legitimate and fake photos and frames extracted from videos. Regarding deepfake detection in videos, the Celeb-DFv1 dataset was used, featuring 590 original videos collected from YouTube, and covering different subjects. The results obtained with the 5-fold cross-validation outperformed those SVM-based methods documented in the literature, by achieving an average F1-score of 99.53%, 79.55%, and 89.10%, respectively for photos, videos, and a mixture of both types of content. A benchmark with state-of-the-art methods was also done, by comparing the proposed SVM method with deep learning approaches, namely Convolutional Neural Networks (CNN). Despite CNN having outperformed the proposed DFT-SVM compound method, the competitiveness of the results attained by DFT-SVM and the substantially reduced processing time make it appropriate to be implemented and embedded into Autopsy modules, by predicting the level of fakeness calculated for each analyzed multimedia file.

Topics

No keywords indexed for this article. Browse by subject →

References

59

[1]

(2019). Accenture/Ponemon Institute: The Cost of Cybercrime. Netw. Secur., 2019, 4. 10.1016/s1353-4858(19)30032-7

[2]

Roškot, M., Wanasika, I., and Kroupova, Z.K. (2020). Cybercrime in Europe: Surprising results of an expensive lapse. J. Bus. Strategy. 10.1108/jbs-12-2019-0235

[3]

Kertysova, K., Frinking, E., van den Dool, K., Maričić, A., and Bhattacharyya, K. (2018). Cybersecurity: Ensuring Awareness and Resilience of the Private Sector Across Europe in Face of Mounting Cyber Risks-Study, European Economic and Social Committee. Available online: https://www.eesc.europa.eu/en/our-work/publications-other-work/publications/cybersecurity-ensuring-awareness-and-resilience-private-sector-across-europe-face-mounting-cyber-risks-study.

[4]

(2021, March 16). ENISA Threat Landscape—2020. Available online: https://www.enisa.europa.eu/topics/threat-risk-management/threats-and-trends/.

[5]

Anderson "The economics of information security" Science (2006) 10.1126/science.1130992

[6]

Bada, M., and Nurse, J.R. (2020). The social and psychological impact of cyberattacks. Emerging Cyber Threats and Cognitive Vulnerabilities, Academic Press. 10.1016/b978-0-12-816203-3.00004-6

[7]

Lallie, H.S., Shepherd, L.A., Nurse, J.R., Erola, A., Epiphaniou, G., Maple, C., and Bellekens, X. (2021). Cyber Security in the Age of COVID-19: A Timeline and Analysis of Cyber-Crime and Cyber-Attacks during the Pandemic. Comput. Secur., 102248. 10.1016/j.cose.2021.102248

[8]

Alheneidi, H., AlSumait, L., AlSumait, D., and Smith, A.P. (2021). Loneliness and Problematic Internet Use during COVID-19 Lock-Down. Behav. Sci., 11. 10.3390/bs11010005

[9]

The Emergence of Deepfake Technology: A Review

Mika Westerlund

Technology Innovation Management Review 2019 10.22215/timreview/1282

[10]

Botha, J., and Pieterse, H. Fake News and Deepfakes: A Dangerous Threat for 21st Century Information Security. Proceedings of the International Conference on CyberWarfare and Security, Norfolk, VA, USA, 12–13 March 2020.

[11]

Harris "Deepfakes: False pornography is here and the law cannot protect you" Duke L. Tech. Rev. (2018)

[12]

Spivak "Deepfakes: The Newest Way to Commit One of the Oldest Crimes" Geo. L. Tech. Rev. (2019)

[13]

Soltani, S., and Seno, S.A.H. (2017, January 26–27). A survey on digital evidence collection and analysis. Proceedings of the 2017 7th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Iran. 10.1109/iccke.2017.8167885

[14]

Garfinkel "Digital forensics research: The next 10 years" Digit. Investig. (2010) 10.1016/j.diin.2010.05.009

[15]

Casey "The chequered past and risky future of digital forensics" Aust. J. Forensic Sci. (2019) 10.1080/00450618.2018.1554090

[16]

Horsman "Tool testing and reliability issues in the field of digital forensics" Digit. Investig. (2019) 10.1016/j.diin.2019.01.009

[17]

Raghavan "Digital forensic research: Current state of the art" CSI Trans. ICT (2013) 10.1007/s40012-012-0008-7

[18]

Deepfakes and beyond: A Survey of face manipulation and fake detection

Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez et al.

Information Fusion 2020 10.1016/j.inffus.2020.06.014

[19]

Bhatt "Machine learning forensics: A new branch of digital forensics" Int. J. Adv. Res. Comput. Sci. (2017) 10.26483/ijarcs.v8i8.4613

[20]

Durall, R., Keuper, M., Pfreundt, F.J., and Keuper, J. (2019). Unmasking deepfakes with simple features. arXiv.

[21]

Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 13–19). Celeb-df: A large-scale challenging dataset for deepfake forensics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA. 10.1109/cvpr42600.2020.00327

[22]

Hadhazy, A. (2021, March 11). Is That Iranian Missile Photo a Fake?. 2008., Available online: https://www.scientificamerican.com/article/is-that-iranian-missile/.

[23]

Tait, A. (2021, March 11). How a Badly Faked Photo of Vladimir Putin Took Over Twitter. Available online: https://www.newstatesman.com/science-tech/social-media/2017/07/how-badly-faked-photo-vladimir-putin-took-over-twitter.

[24]

(2021, June 22). Iran ’Faked Missile Test Image’. Available online: http://news.bbc.co.uk/2/hi/middle_east/7500917.stm.

[25]

(2021, June 22). In an Iranian Image, a Missile Too Many. Available online: https://thelede.blogs.nytimes.com/2008/07/10/in-an-iranian-image-a-missile-too-many/.

[26]

Fridrich, A.J., Soukal, B.D., and Lukáš, A.J. Detection of copy–move forgery in digital images. Proceedings of the Digital Forensic Research Workshop, Cleveland, Ohio, USA, 6–8 August 2003.

[27]

Xu, B., Liu, G., and Dai, Y. (2014). Detecting image splicing using merged features in chroma space. Sci. World J., 2014. 10.1155/2014/262356

[28]

Kietzmann "Deepfakes: Trick or treat?" Bus. Horizons (2020) 10.1016/j.bushor.2019.11.006

[29]

Nguyen, T.T., Nguyen, C.M., Nguyen, D.T., Nguyen, D.T., and Nahavandi, S. (2019). Deep learning for deepfakes creation and detection. arXiv.

[30]

Christian, J. (2016, June 22). Experts Fear Face Swapping Tech Could Start an International Showdown. Available online: https://theoutline.com/post/3179/deepfake-videos-are-freaking-experts-out.

[31]

Roose, K. (2016, June 22). Here, Come the Fake Videos, Too, 2018. Available online: https://www.nytimes.com/2018/03/04/technology/fake-videos-deepfakes.html.

[32]

Niyishaka, P., and Bhagvati, C. Digital image forensics technique for copy–move forgery detection using dog and orb. Proceedings of the International Conference on Computer Vision and Graphics, Madrid, Spain, 17–19 July 2018. 10.1007/978-3-030-00692-1_41

[33]

Vincent "A descriptive algorithm for sobel image edge detection" Proceedings of the Informing Science & IT Education Conference (InSITE), Macon, GA, USA, 12–15 June 2009 (2009)

[34]

ORB: An efficient alternative to SIFT or SURF

Ethan Rublee, Vincent Rabaud, Kurt Konolige et al.

2011 International Conference on Computer Vision 10.1109/iccv.2011.6126544

[35]

Castillo Camacho, I., and Wang, K. (2021). A Comprehensive Review of Deep-Learning-Based Methods for Image Forensics. J. Imaging, 7. 10.3390/jimaging7040069

[36]

Diallo "Robust forgery detection for compressed images using CNN supervision" Forensic Sci. Int. Rep. (2020) 10.1016/j.fsir.2020.100112

[37]

O’Shea, K., and Nash, R. (2015). An introduction to convolutional neural networks. arXiv.

[38]

Jafar, M.T., Ababneh, M., Al-Zoube, M., and Elhassan, A. Forensics and Analysis of Deepfake Videos. Proceedings of the 2020 11th International Conference on Information and Communication Systems (ICICS), Copenhagen, Denmark 24–27 August 2020. 10.1109/icics49469.2020.239493

[39]

Amidi, A., and Amidi, S. (2021, June 14). CS 230—Recurrent Neural Networks Cheatsheet. Available online: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks.

[40]

Yang, P., Baracchi, D., Ni, R., Zhao, Y., Argenti, F., and Piva, A. (2020). A survey of deep learning-based source image forensics. J. Imaging, 6. 10.3390/jimaging6030009

[41]

Martinez "Automatic analysis of facial actions: A survey" IEEE Trans. Affect. Comput. (2017) 10.1109/taffc.2017.2731763

[42]

He, J., Li, D., Yang, B., Cao, S., Sun, B., and Yu, L. Multi view facial action unit detection based on CNN and BLSTM-RNN. Proceedings of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017),Washington, DC, USA, 30 May–3 June 2017. 10.1109/fg.2017.108

[43]

Zhi "A comprehensive survey on automatic facial action unit analysis" Vis. Comput. (2020) 10.1007/s00371-019-01707-5

[44]

McCloskey, S., and Albright, M. (2018). Detecting gan-generated imagery using color cues. arXiv. 10.1109/icip.2019.8803661

[45]

Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., and Manocha, D. (2020, June 22). Emotions Don’t Lie: A Deepfake Detection Method Using Audio-Visual Affective Cues. Available online: https://arxiv.org/abs/2003.06711. 10.1145/3394171.3413570

[46]

Dolhansky, B., Howes, R., Pflaum, B., Baram, N., and Ferrer, C.C. (2020). The deepfake detection challenge (dfdc) dataset. arXiv.

[47]

Vapnik, V. (2013). The Nature of Statistical Learning Theory, Springer Science & Business Media.

[48]

Support vector machines

M.A. Hearst, S.T. Dumais, E. Osuna et al.

IEEE Intelligent Systems and their Applications 1998 10.1109/5254.708428

[49]

A training algorithm for optimal margin classifiers

Bernhard E. Boser, Isabelle M. Guyon, Vladimir N. Vapnik

Proceedings of the fifth annual workshop on Comput... 10.1145/130385.130401

[50]

Feng, X., Cox, I.J., and Doërr, G. An energy-based method for the forensic detection of re-sampled images. Proceedings of the 2011 IEEE International Conference on Multimedia and Expo, Barcelona, Spain, 11–15 July 2011.

Showing 50 of 59 references

Metrics

34

Citations

59

References

Details

Published: Jun 24, 2021
Vol/Issue: 7(7)
Pages: 102
License: View

Authors

S

Sara Ferreira

Department of Computer Science, Faculty of Sciences, University of Porto, 4169-007 Porto, Portugal

M

Mário Antunes

Computer Science and Communication Research Centre (CIIC), School of Technology and Management, Polytechnic of Leiria, 2411-901 Leiria, Portugal; INESC TEC, CRACS, 4200-465 Porto, Portugal

M

Manuel E. Correia

Department of Computer Science, Faculty of Sciences, University of Porto, 4169-007 Porto, Portugal; INESC TEC, CRACS, 4200-465 Porto, Portugal

Cite This Article

Sara Ferreira, Mário Antunes, Manuel E. Correia (2021). Exposing Manipulated Photos and Videos in Digital Forensics Analysis. Journal of Imaging, 7(7), 102. https://doi.org/10.3390/jimaging7070102

Exposing Manipulated Photos and Videos in Digital Forensics Analysis

You May Also Like