Understanding and Creating Art with AI: Review and Outlook

Eva Cetinic; James She

doi:10.1145/3475799

journal article Feb 16, 2022

Understanding and Creating Art with AI: Review and Outlook

ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 18 No. 2 pp. 1-22 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3475799

Abstract

Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts. The growing number of research initiatives and creative applications that emerge in the intersection of AI and art motivates us to examine and discuss the creative and explorative potentials of AI technologies in the context of art. This article provides an integrated review of two facets of AI and art: (1) AI is used for art analysis and employed on digitized artwork collections, or (2) AI is used for creative purposes and generating novel artworks. In the context of AI-related research for art understanding, we present a comprehensive overview of artwork datasets and recent works that address a variety of tasks such as classification, object detection, similarity retrieval, multimodal representations, and computational aesthetics, among others. In relation to the role of AI in creating art, we address various practical and theoretical aspects of AI Art and consolidate related works that deal with those topics in detail. Finally, we provide a concise outlook on the future progression and potential impact of AI technologies on our understanding and creation of art.

Topics

No keywords indexed for this article. Browse by subject →

References

141

[1]

G. Goh A. Ramesh M. Pavlov and S. Gray. 2021. DALL \cdot E: Creating Images from Text. Retrieved January 25 2021 from https://openai.com/blog/dall-e/.

[2]

10.1016/j.sigpro.2012.01.016

[3]

Panos Achlioptas, Maks Ovsjanikov, Kilichbek Haydarov, Mohamed Elhoseiny, and Leonidas Guibas. 2021. ArtEmis: Affective language for visual art. arXiv preprint arXiv:2101.07396 (2021).

[4]

10.1109/wacv.2015.84

[5]

10.1109/cvpr.2016.566

[6]

Seyed Ali Amirshahi, Gregor Uwe Hayn-Leichsenring, Joachim Denzler, and Christoph Redies. 2014. Jenaesthetics subjective dataset: Analyzing paintings by subjective scores. In Proceedings of the European Conference on Computer Vision. 3–19.

[7]

Yaniv Bar Noga Levy and Lior Wolf. 2014. Classification of artistic styles using binarized features derived from a deep neural network. In Computer Vision—ECCV 2014 Workshops . Lecture Notes in Computer Science Vol. 8925. Springer 71–84. https://doi.org/10.1007/978-3-319-16178-5_5 10.1007/978-3-319-16178-5_5

[8]

10.1109/icpr.2018.8545064

[9]

10.1515/mial-2019-0004

[10]

10.1016/j.eswa.2019.05.036

[11]

Margaret A. Boden. 2010. Creativity and Art: Three Roads to Surprise. Oxford University Press.

[12]

10.1080/14626260902867915

[13]

Pietro Bongini, Federico Becattini, Andrew D. Bagdanov, and Alberto Del Bimbo. 2020. Visual question answering for cultural heritage. arXiv preprint arXiv:2003.09853 (2020).

[14]

Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018).

[15]

Sheila Bsteh and Filip Vermeylen. 2021. From Painting to Pixel: Understanding NFT Artworks. Retrieved June 15 2021 from https://www.researchgate.net/publication/351346278_From_Painting_to_Pixel_Understanding_NFT_artworks.

[16]

10.1007/978-3-642-33765-9_11

[17]

10.1007/s11042-020-09995-z

[18]

Giovanna Castellano and Gennaro Vessio. 2020. Towards a tool for visual link retrieval and knowledge discovery in painting datasets. In Digital Libraries: The Era of Big Data and Data Science . Communications in Computer and Information Science Vol. 1177. Springer 105–110. 10.1007/978-3-030-39905-4_11

[19]

Eva Cetinic. 2020. Iconographic image captioning for artworks. In Pattern Recognition. ICPR International Workshops and Challenges . Lecture Notes in Computer Science Vol. 12663. Springer 502–516. 10.1007/978-3-030-68796-0_36

[20]

Eva Cetinic and Sonja Grgic. 2013. Automated painter recognition based on image feature extraction. In Proceedings of the 2013 55th International Symposium (ELMAR’13). IEEE, Los Alamitos, CA, 19–22.

[21]

10.1109/elmar.2016.7731786

[22]

10.1016/j.eswa.2018.07.026

[23]

10.1109/access.2019.2921101

[24]

10.1016/j.patrec.2019.11.008

[25]

Putting the art in artificial: Aesthetic responses to computer-generated art.

Rebecca Chamberlain, Caitlin Mullin, Bram Scheerlinck et al.

Psychology of Aesthetics, Creativity, and the Arts 10.1037/aca0000136

[26]

10.1145/3326338

[27]

Christie’s. 2018. Is Artificial Intelligence Set to Become Art’s Next Medium? Retrieved December 2 2020 from https://www.christies.com/features/A-collaboration-between-two-artists-one-human-one-a-machine-9332-1.aspx.

[28]

Christie’s. 2021. Monumental Collage by Beeple Is First Purely Digital Artwork NFT to Come to Auction. Retrieved June 15 2021 from https://www.christies.com/features/Monumental-collage-by-Beeple-is-first-purely-digital-artwork-NFT-to-come-to-auction-11510-7.aspx.

[29]

10.1007/s13347-016-0231-5

[30]

10.5555/2424046

[31]

Simon Colton, Alison Pease, and Rob Saunders. 2018. Issues of authenticity in autonomously creative systems. In Proceedings of the 9th International Conference on Computational Creativity.

[32]

10.5244/c.29.65

[33]

Elliot J. Crowley and Andrew Zisserman. 2014. In search of art. In Computer Vision—ECCV 2014 Workshops . Lecture Notes in Computer Science Vol. 8925. Springer 54–70. 10.1007/978-3-319-16178-5_4

[34]

10.5244/c.28.38

[35]

Elliot J. Crowley and Andrew Zisserman. 2016. The art of detection. In Computer Vision—ECCV 2016 Workshops . Lecture Notes in Computer Science Vol. 9913. Springer 721–737. 10.1007/978-3-319-46604-0_50

[36]

10.1145/3306618.3314233

[37]

Omid E. David and Nathan S. Netanyahu. 2016. DeepPainter: Painter classification using deep convolutional autoencoders. In Artificial Neural Networks and Machine Learning . Lecture Notes in Computer Science Vol. 9887. Springer 20–28. 10.1007/978-3-319-44781-0_3

[38]

ImageNet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher et al.

2009 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2009.5206848

[39]

Yingying Deng, Fan Tang, Weiming Dong, Chongyang Ma, Feiyue Huang, Oliver Deussen, and Changsheng Xu. 2020. Exploring the representativity of art paintings. IEEE Transactions on Multimedia 23 (2020), 2794–2805.

[40]

10.1145/2466816.2466837

[41]

10.1080/14626268.2012.709940

[42]

Image quilting for texture synthesis and transfer

Alexei A. Efros, William T. Freeman

Proceedings of the 28th annual conference on Compu... 10.1145/383259.383296

[43]

10.1145/3474085.3475586

[44]

10.1511/2019.107.1.18

[45]

Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. 2017. CAN: Creative adversarial networks, generating “art” by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068 (2017).

[46]

10.5555/3504035.3504301

[47]

10.1016/j.isci.2020.101515

[48]

10.1038/s42256-020-0161-x

[49]

10.1109/eusipco.2016.7760382

[50]

Massimo Franceschet, Giovanni Colavizza, T’ai Smith, Blake Finucane, Martin Lukas Ostachowski, Sergio Scalet, Jonathan Perkins, James Morgan, and Sebástian Hernández. 2020. Crypto art: A decentralized view. Leonardo 54, 4 (2020), 1–8.

Showing 50 of 141 references

Cited By

385

GENERATIVE AI AS CREATIVE COLLABORATORS IN VISUAL ART AND FILM: A POSTHUMANISM PERSPECTIVE

Shopita Khurana, Kamaljeet Kaur · 2026

ShodhKosh: Journal of Visual and Pe...

Control of flow behavior in complex fluids using automatic differentiation

Mohammed G. Alhashim, Kaylie Hausknecht · 2025

Proceedings of the National Academy...

If ChatGPT can do it, where is my creativity? generative AI boosts performance but diminishes experience in creative writing

Peidong Mei, Deborah N. Brewis · 2025

Computers in Human Behavior: Artifi...

Emerging human-technology relationships in a co-design process with generative AI

Henriikka Vartiainen, Päivikki Liukkonen · 2025

Thinking Skills and Creativity

Unleashing Creativity in the Metaverse: Generative AI and Multimodal Content

Abdulmotaleb El Saddik, Jamil Ahmad · 2025

ACM Transactions on Multimedia Comp...

A Comparative Study on the Influence of Artificial Intelligence ( AI ) on Student Creativity, Learning Outcomes and Attitudes in Product Design Education ( PDE )

Wei Lu, Gurpinder Singh Lalli · 2025

European Journal of Education

Technology Acceptance and Innovation Diffusion: Are Users More Inclined Toward AIGC-Assisted Design?

Qianling Jiang, Jie Zhang · 2024

International Journal of Human–Comp...

A Study on Factors Influencing Designers’ Behavioral Intention in Using AI-Generated Content for Assisted Design: Perceived Anxiety, Perceived Risk, and UTAUT

Weiyi Li · 2024

International Journal of Human–Comp...

Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services

Minrui Xu, Hongyang Du · 2024

IEEE Communications Surveys & T...

Integration of AI Training in the Field of Higher Education in the Republic of Bulgaria: An Overview

Svilen Simeonov, Firgan Feradov · 2024

Education Sciences

Unravelling the Impact of Generative Artificial Intelligence (GAI) in Industrial Applications: A Review of Scientific and Grey Literature

Arpan Kumar Kar, P. S. Varsha · 2023

Global Journal of Flexible Systems...

Factors Influencing the Adoption of Generative AI for Art Designing Among Chinese Generation Z: A Structural Equation Modeling Approach

Yiyang Wang, Weining Zhang · 2023

IEEE Access

AI as an Artist? A Two-Wave Survey Study on Attitudes Toward Using Artificial Intelligence in Art

Rita Latikka, Jenna Bergdahl · 2023

Poetics

From Turing to Transformers: A Comprehensive Review and Tutorial on the Evolution and Applications of Generative Transformer Models

Emma Yann Zhang, Adrian David Cheok · 2023

Sci

Does human–AI collaboration lead to more creative art? Aesthetic evaluation of human-made and AI-generated haiku poetry

Jimpei Hitsuwari, Yoshiyuki Ueda · 2023

Computers in Human Behavior

Metrics

385

Citations

141

References

Details

Published: Feb 16, 2022
Vol/Issue: 18(2)
Pages: 1-22
License: View

Authors

E

Eva Cetinic

Rudjer Boskovic Institute and Durham University, Zagreb, Croatia UK

J

James She

Hamad Bin Khalifa University (HBKU) and Hong Kong University of Science and Technology, Kowloon, Hong Kong

Cite This Article

Eva Cetinic, James She (2022). Understanding and Creating Art with AI: Review and Outlook. ACM Transactions on Multimedia Computing, Communications, and Applications, 18(2), 1-22. https://doi.org/10.1145/3475799

Understanding and Creating Art with AI: Review and Outlook

You May Also Like