journal article
Open Access
Jan 24, 2024
Semantic Communication: A Survey of Its Theoretical Development
Abstract
In recent years, semantic communication has received significant attention from both academia and industry, driven by the growing demands for ultra-low latency and high-throughput capabilities in emerging intelligent services. Nonetheless, a comprehensive and effective theoretical framework for semantic communication has yet to be established. In particular, finding the fundamental limits of semantic communication, exploring the capabilities of semantic-aware networks, or utilizing theoretical guidance for deep learning in semantic communication are very important yet still unresolved issues. In general, the mathematical theory of semantic communication and the mathematical representation of semantics are referred to as semantic information theory. In this paper, we introduce the pertinent advancements in semantic information theory. Grounded in the foundational work of Claude Shannon, we present the latest developments in semantic entropy, semantic rate-distortion, and semantic channel capacity. Additionally, we analyze some open problems in semantic information measurement and semantic coding, providing a theoretical basis for the design of a semantic communication system. Furthermore, we carefully review several mathematical theories and tools and evaluate their applicability in the context of semantic communication. Finally, we shed light on the challenges encountered in both semantic communication and semantic information theory.
Topics
No keywords indexed for this article. Browse by subject →
References
99
[1]
The Roadmap to 6G: AI Empowered Wireless Networks
Khaled B. Letaief, Wei Chen, Yuanming Shi et al.
IEEE Communications Magazine
2019
10.1109/mcom.2019.1900271
[2]
Qin "Beyond transmitting bits: Context, semantics, and task-oriented communications" IEEE J. Sel. Areas Commun. (2022)
[3]
Yang "Semantic communications for future internet: Fundamentals, applications, and challenges" IEEE Commun. Surv. Tutor. (2022) 10.1109/comst.2022.3223224
[4]
Shannon, C.E., and Weaver, W. (1949). The Mathematical Theory of Communication, The University of Illinois Press.
[5]
Qin, Z., Tao, X., Lu, J., Tong, W., and Li, G.Y. (2021). Semantic communications: Principles and challenges. arXiv.
[6]
Strinati "6G networks: Beyond Shannon towards semantic and goal-oriented communications" Comput. Netw. (2021) 10.1016/j.comnet.2021.107930
[7]
Shi "From semantic communication to semantic-aware networking: Model, architecture, and open problems" IEEE Commun. Mag. (2021) 10.1109/mcom.001.2001239
[8]
Kountouris "Semantics-empowered communication for networked intelligent systems" IEEE Commun. Mag. (2021) 10.1109/mcom.001.2000604
[9]
Kalfa "Towards goal-oriented semantic signal processing: Applications and future challenges" Digit. Signal Process. (2021) 10.1016/j.dsp.2021.103134
[10]
Lan "What is semantic communication? A view on conveying meaning in the era of machine intelligence" J. Commun. Inf. Netw. (2021) 10.23919/jcin.2021.9663101
[11]
Uysal "Semantic communications in networked systems: A data significance perspective" IEEE Netw. (2022) 10.1109/mnet.106.2100636
[12]
Zhang "Toward wisdom-evolutionary and primitive-concise 6G: A new paradigm of semantic communication networks" Engineering (2022) 10.1016/j.eng.2021.11.003
[13]
Shi, Y., Zhou, Y., Wen, D., Wu, Y., Jiang, C., and Letaief, K.B. (2023). Task-oriented communications for 6g: Vision, principles, and technologies. arXiv.
10.1109/mwc.002.2200468
[14]
A Mathematical Theory of Communication
C. E. Shannon
Bell System Technical Journal
1948
10.1002/j.1538-7305.1948.tb01338.x
[15]
Bao, J., Basu, P., Dean, M., Partridge, C., Swami, A., Leland, W., and Hendler, J.A. (2011, January 22–24). Towards a theory of semantic communication. Proceedings of the 2011 IEEE Network Science Workshop, West Point, NY, USA.
10.1109/nsw.2011.6004632
[16]
Iyer "A survey on semantic communications for intelligent wireless networks" Wirel. Pers. Commun. (2023) 10.1007/s11277-022-10111-7
[17]
Carnap, R. (1950). Logical Foundations of Probability, University of Chicago Press.
[18]
Carnap, R., and Bar-Hillel, Y. (1952). An Outline of a Theory of Semantic Information, Massachusetts Institute of Technology.
[19]
Floridi "Outline of a theory of strongly semantic information" Minds Mach. (2004) 10.1023/b:mind.0000021684.50925.c9
[20]
"On quantifying semantic information" Information (2011) 10.3390/info2010061
[21]
Basu "Preserving quality of information by using semantic relationships" Pervasive Mob. Comput. (2014) 10.1016/j.pmcj.2013.07.013
[22]
Chattopadhyay, A., Haeffele, B.D., Geman, D., and Vidal, R. (2024, January 19). Quantifying Task Complexity through Generalized Information Measures. Available online: https://openreview.net/forum?id=vcKVhY7AZqK.
[23]
Melamed, I.D. (1997, January 4–5). Measuring semantic entropy. Proceedings of the Tagging Text with Lexical Semantics: Why, What, and How?, Washington, DC, USA. Available online: https://aclanthology.org/W97-0207.pdf.
[24]
Liu "AFSSE: An interpretable classifier with axiomatic fuzzy set and semantic entropy" IEEE Trans. Fuzzy Syst. (2019) 10.1109/tfuzz.2019.2945239
[25]
De Luca, A., and Termini, S. (1993). Readings in Fuzzy Sets for Intelligent Systems, Elsevier.
[26]
Choi, J., Loke, S.W., and Park, J. (2022, January 16–20). A unified view on semantic information and communication: A probabilistic logic approach. Proceedings of the 2022 IEEE International Conference on Communications Workshops (ICC Workshops), Seoul, Republic of Korea.
10.1109/iccworkshops53468.2022.9814642
[27]
Xin, G., and Fan, P. (2022). EXK-SC: A semantic communication model based on information framework expansion and knowledge collision. Entropy, 24.
10.20944/preprints202210.0399.v1
[28]
Xin, G., Zhu, Z., and Fan, P. (June, January 28). Information Framework Expansion Meets Knowledge Collision for Semantic Communications. Proceedings of the ICC 2023-IEEE International Conference on Communications, Rome, Italy.
10.1109/icc45041.2023.10278568
[29]
Kolchinsky "Semantic information, autonomous agency and non-equilibrium statistical physics" Interface Focus (2018) 10.1098/rsfs.2018.0041
[30]
Venhuizen, N.J., Crocker, M.W., and Brouwer, H. (2019). Semantic entropy in language comprehension. Entropy, 21.
10.3390/e21121159
[31]
Lu, C. (2018, January 2–5). From Bayesian inference to logical Bayesian inference: A new mathematical frame for semantic communication and machine learning. Proceedings of the Intelligence Science II: Third IFIP TC 12 International Conference, ICIS 2018, Beijing, China. Proceedings 2.
10.1007/978-3-030-01313-4_2
[32]
Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley.
[33]
Johnson, J., Alahi, A., and Fei-Fei, L. (2016, January 11–14). Perceptual losses for real-time style transfer and super-resolution. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part II 14.
10.1007/978-3-319-46475-6_43
[34]
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
10.1109/cvpr.2018.00068
[35]
Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., and Wu, Y. (2014, January 23–28). Learning fine-grained image similarity with deep ranking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.
10.1109/cvpr.2014.180
[36]
Zhu, T., Peng, B., Liang, J., Han, T., Wan, H., Fu, J., and Chen, J. (2023). How to Evaluate Semantic Communications for Images with ViTScore Metric?. arXiv.
[37]
Farsad, N., Rao, M., and Goldsmith, A. (2018, January 15–20). Deep learning for joint source-channel coding of text. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.
10.1109/icassp.2018.8461983
[38]
Yener "The semantic communication game" IEEE Trans. Cogn. Commun. Netw. (2018) 10.1109/tccn.2018.2872596
[39]
Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6–12). Bleu: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.
10.3115/1073083.1073135
[40]
Xie "Deep learning enabled semantic communication systems" IEEE Trans. Signal Process. (2021) 10.1109/tsp.2021.3071210
[41]
Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
[42]
Rix "Perceptual evaluation of speech quality (PESQ)—A new method for speech quality assessment of telephone networks and codecs" Proceedings of the 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing (2001) 10.1109/icassp.2001.941023
[43]
Taal "An algorithm for intelligibility prediction of time–frequency weighted noisy speech" IEEE Trans. Audio Speech Lang. Process. (2011) 10.1109/tasl.2011.2114881
[44]
Bińkowski, M., Donahue, J., Dieleman, S., Clark, A., Elsen, E., Casagrande, N., Cobo, L.C., and Simonyan, K. (2019). High fidelity speech synthesis with adversarial networks. arXiv.
[45]
Liu, J., Zhang, W., and Poor, H.V. (2021, January 12–20). A rate-distortion framework for characterizing semantic information. Proceedings of the 2021 IEEE International Symposium on Information Theory (ISIT), Virtual Event.
10.1109/isit45174.2021.9518240
[46]
Guo, T., Wang, Y., Han, J., Wu, H., Bai, B., and Han, W. (2022). Semantic compression with side information: A rate-distortion perspective. arXiv.
[47]
Stavrou, P.A., and Kountouris, M. (July, January 26). A rate distortion approach to goal-oriented communication. Proceedings of the 2022 IEEE International Symposium on Information Theory (ISIT), Espoo, Finland.
10.36227/techrxiv.19128026
[48]
Shao, Y., Cao, Q., and Gunduz, D. (2022). A theory of semantic communication. arXiv.
[49]
Agheli, P., Pappas, N., and Kountouris, M. (2022, January 4–8). Semantic Source Coding for Two Users with Heterogeneous Goals. Proceedings of the GLOBECOM 2022–2022 IEEE Global Communications Conference, Rio de Janeiro, Brazil.
10.1109/globecom48099.2022.10001230
[50]
Xiao, Y., Zhang, X., Li, Y., Shi, G., and Başar, T. (2022, January 6–9). Rate-distortion theory for strategic semantic communication. Proceedings of the 2022 IEEE Information Theory Workshop (ITW), Mumbai, India.
10.1109/itw54588.2022.9965825
Showing 50 of 99 references
Metrics
28
Citations
99
References
Details
- Published
- Jan 24, 2024
- Vol/Issue
- 26(2)
- Pages
- 102
- License
- View
Authors
Funding
National Key Research and Development Program of China
Award: 2021.YFA1000504
Cite This Article
Gangtao Xin, Pingyi Fan, Khaled B. Letaief (2024). Semantic Communication: A Survey of Its Theoretical Development. Entropy, 26(2), 102. https://doi.org/10.3390/e26020102
Related
You May Also Like
Explainable AI: A Review of Machine Learning Interpretability Methods
Pantelis Linardatos, Vasilis Papastefanopoulos · 2020
2,260 citations
Approximate Entropy and Sample Entropy: A Comprehensive Tutorial
Alfonso Delgado-Bonal, Alexander Marshak · 2019
587 citations
Enhanced Sampling in Molecular Dynamics Using Metadynamics, Replica-Exchange, and Temperature-Acceleration
Cameron Abrams, Giovanni Bussi · 2013
387 citations