A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks

Zhigang Jin; Yingying Ma; Yishan Su; Shuo Li; Xiaomei Fu

doi:10.3390/s17071660

journal article Open Access Jul 19, 2017

A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks

Zhigang Jin

Yingying Ma Yishan Su

Shuo Li

Xiaomei Fu

Sensors Vol. 17 No. 7 pp. 1660 · MDPI AG

View at Publisher Save 10.3390/s17071660

Abstract

Underwater sensor networks (UWSNs) have become a hot research topic because of their various aquatic applications. As the underwater sensor nodes are powered by built-in batteries which are difficult to replace, extending the network lifetime is a most urgent need. Due to the low and variable transmission speed of sound, the design of reliable routing algorithms for UWSNs is challenging. In this paper, we propose a Q-learning based delay-aware routing (QDAR) algorithm to extend the lifetime of underwater sensor networks. In QDAR, a data collection phase is designed to adapt to the dynamic environment. With the application of the Q-learning technique, QDAR can determine a global optimal next hop rather than a greedy one. We define an action-utility function in which residual energy and propagation delay are both considered for adequate routing decisions. Thus, the QDAR algorithm can extend the network lifetime by uniformly distributing the residual energy and provide lower end-to-end delay. The simulation results show that our protocol can yield nearly the same network lifetime, and can reduce the end-to-end delay by 20–25% compared with a classic lifetime-extended routing protocol (QELAR).

Topics

No keywords indexed for this article. Browse by subject →

References

26

[1]

Felemban "Underwater sensor network applications: A comprehensive survey" Int. J. Distrib. Sens. Netw. (2015) 10.1155/2015/896832

[2]

Sheikh, A.A., Felemban, E., Felemban, M., and Qaisar, S.B. (2016, January 28–30). Challenges and opportunities for underwater sensor networks. Proceedings of the 12th IEEE International Conference on Innovations in Information Technology (IIT), Al Ain, United Arab Emirates. 10.1109/innovations.2016.7880021

[3]

Zhang "Energy-aware routing for delay-sensitive underwater wireless sensor networks" Sci. China Inf. Sci. (2014)

[4]

Li, N., Martínez, J.F., Meneses Chaus, J.M., and Eckert, M. (2016). A survey on underwater acoustic sensor network routing protocols. Sensors, 16. 10.3390/s16030414

[5]

Qian, L., Zhang, S., Liu, M., and Zhang, Q. (2016, January 9–11). A MACA-Based Power Control MAC Protocol for Underwater Wireless Sensor Networks. Proceedings of the IEEE/OES Ocean Acoustics (COA), Harbin, China. 10.1109/coa.2016.7535810

[6]

Kacimi "Load balancing techniques for lifetime maximizing in wireless sensor networks" Ad Hoc Netw. (2013) 10.1016/j.adhoc.2013.04.009

[7]

Darehshoorzadeh "Underwater sensor networks: A new challenge for opportunistic routing protocols" IEEE Commun. Mag. (2015) 10.1109/mcom.2015.7321977

[8]

Han "Routing protocols for underwater wireless sensor networks" IEEE Commun. Mag. (2015) 10.1109/mcom.2015.7321974

[9]

Ahmadi "An efficient routing algorithm to preserve k-coverage in wireless sensor networks" J. Supercomput. (2014) 10.1007/s11227-013-1054-0

[10]

Naranjo "P-SEP: A prolong stable election routing algorithm for energy-limited heterogeneous fog-supported wireless sensor networks" J. Supercomput. (2016)

[11]

Shojafar, M., Pooranian, Z., Naranjo, P.G.V., and Baccarelli, E. (2017). FLAPS: Bandwidth and Delay-Efficient Distributed Data Searching in Fog-Supported P2P Content Delivery Networks. J. Supercomput., 1–22. 10.1007/s11227-017-2082-y

[12]

Bai, W., Wang, H., Shen, X., Zhao, R., and Zhang, Y. (2016). Minimum delay multipath routing based on TDMA for underwater acoustic sensor network. Int. J. Distrib. Sens. Netw., 2016. 10.1155/2016/1394340

[13]

Alzeidi "EMGGR: An energy-efficient multipath grid-based geographic routing protocol for underwater wireless sensor networks" Wirel. Netw. (2016)

[14]

Ali "End-to-end delay and energy efficient routing protocol for underwater wireless sensor networks" Wirel. Pers. Commun. (2014) 10.1007/s11277-014-1859-z

[15]

Pooranian "Queen-bee algorithm for energy efficient clusters in wireless sensor networks" World Acad. Sci. Eng. Technol. (2011)

[16]

Wei, B., Luo, Y.M., Jin, Z., Wei, J., and Su, Y. (2012, January 8−10). ES-VBF: An energy saving routing protocol. Proceedings of the 2012 International Conference on Information Technology and Software Engineering, Beijing, China. 10.1007/978-3-642-34528-9_10

[17]

Al-Bzoor, M., Zhu, Y., Liu, J., Reda, A., Cui, J.H., and Rajasekaran, S. (2012, January 8–10). Adaptive power controlled routing for underwater sensor networks. Proceedings of the International Conference on Wireless Algorithms, Systems, and Applications, Huangshan, China. 10.1007/978-3-642-31869-6_48

[18]

Hu "QELAR: A machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks" IEEE Trans. Mob. Comput. (2010) 10.1109/tmc.2010.28

[19]

Pompili "Distributed routing algorithms for underwater acoustic sensor networks" IEEE Trans. Wirel. Commun. (2010) 10.1109/twc.2010.070910.100145

[20]

Hsu "Delay-sensitive opportunistic routing for underwater sensor networks" IEEE Sens. J. (2015) 10.1109/jsen.2015.2461652

[21]

Nowé, A., and Brys, T. (2016). A Gentle Introduction to Reinforcement Learning. Scalable Uncertainty Management, Springer International Publishing. 10.1007/978-3-319-45856-4_2

[22]

Zhang "Dynamic node cooperation in an underwater data collection network" IEEE Sens. J. (2016) 10.1109/jsen.2015.2453552

[23]

Xie, P., Cui, J.H., and Lao, L. (2006, January 15–19). VBF: Vector-Based Forwarding Protocol for Underwater Sensor Networks. Proceedings of the International Conference on Research in Networking, Coimbra, Portugal. 10.1007/11753810_111

[24]

Su, Y., Zhu, Y., Mo, H., Cui, J.H., and Jin, Z. (2013, January 7–10). UPC-MAC: A Power Control MAC Protocol for Underwater Sensor Networks. Proceedings of the International Conference on Wireless Algorithms, Systems, and Applications, Zhangjiajie, China. 10.1007/978-3-642-39701-1_31

[25]

Yan, H., Zhou, S., Shi, Z.J., and Li, B. (2007, January 14). A DSP implementation of OFDM acoustic modem. Proceedings of the Second Workshop on Underwater Networks, Montreal, QC, Canada. 10.1145/1287812.1287831

[26]

Molins, M., and Stojanovic, M. (2007, January 16–19). Slotted FAMA: A MAC protocol for underwater acoustic networks. Proceedings of the IEEE Oceans, Singapore. 10.1109/oceansap.2006.4393832

Metrics

69

Citations

26

References

Details

Published: Jul 19, 2017
Vol/Issue: 17(7)
Pages: 1660
License: View

Authors

Z

Zhigang Jin

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

Y

Yingying Ma

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

Y

Yishan Su

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

S

Shuo Li

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

X

Xiaomei Fu

School of Marine Science and Technology, Tianjin University, Tianjin 300072, China

Funding

National Natural Science Foundation of China Award: 61571318

Guangxi Science and Technology Project Award: AC16380094

Cite This Article

Zhigang Jin, Yingying Ma, Yishan Su, et al. (2017). A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks. Sensors, 17(7), 1660. https://doi.org/10.3390/s17071660

A Q-Learning-Based Delay-Aware Routing Algorithm to Extend the Lifetime of Underwater Sensor Networks

You May Also Like