DRL-Based Hybrid Task Offloading and Resource Allocation in Vehicular Networks

Ziang Liu; Zongpu Jia; Xiaoyan Pang

doi:10.3390/electronics12214392

journal article Open Access Oct 24, 2023

DRL-Based Hybrid Task Offloading and Resource Allocation in Vehicular Networks

Ziang Liu

Zongpu Jia Xiaoyan Pang

Electronics Vol. 12 No. 21 pp. 4392 · MDPI AG

View at Publisher Save 10.3390/electronics12214392

Abstract

With the explosion of delay-sensitive and computation-intensive vehicular applications, traditional cloud computing has encountered enormous challenges. Vehicular edge computing, as an emerging computing paradigm, has provided powerful support for vehicular networks. However, vehicle mobility and time-varying characteristics of communication channels have further complicated the design and implementation of vehicular network systems, leading to increased delays and energy consumption. To address this problem, this article proposes a hybrid task offloading algorithm that combines deep reinforcement learning with convex optimization algorithms to improve the performance of the algorithm. The vehicle’s mobility and common signal-blocking problems in the vehicular edge computing environment are taken into account; to minimize system overhead, firstly, the twin delayed deep deterministic policy gradient algorithm (TD3) is used for offloading decision-making, with a normalized state space as the input to improve convergence efficiency. Then, the Lagrange multiplier method allocates server bandwidth to multiple users. The simulation results demonstrate that the proposed algorithm surpasses other solutions in terms of delay and energy consumption.

Topics

No keywords indexed for this article. Browse by subject →

References

32

[1]

Storck "A Survey of 5G Technology Evolution, Standards, and Infrastructure Associated with Vehicle-to-Everything Communications by Internet of Vehicles" IEEE Access (2020) 10.1109/access.2020.3004779

[2]

Feng "Computation offloading in mobile edge computing networks: A survey" J. Netw. Comput. Appl. (2022) 10.1016/j.jnca.2022.103366

[3]

Yan "Offloading and resource allocation with general task graph in mobile edge computing: A deep reinforcement learning approach" IEEE Trans. Wirel. Commun. (2020) 10.1109/twc.2020.2993071

[4]

Arena "When Edge Computing Meets IoT Systems: Analysis of Case Studies" China Commun. (2020) 10.23919/jcc.2020.10.004

[5]

Li "Optimized Content Caching and User Association for Edge Computing in Densely Deployed Heterogeneous Networks" IEEE Trans. Mob. Comput. (2020) 10.1109/tmc.2020.3033563

[6]

Zhang "Mobile Edge Intelligence and Computing for the Internet of Vehicles" Proc. IEEE (2019) 10.1109/jproc.2019.2947490

[7]

Khan "Learning-based resource allocation for backscatter-aided vehicular networks" IEEE Trans. Intell. Transp. Syst. (2021) 10.1109/tits.2021.3126766

[8]

Fatemidokht "Efficient and Secure Routing Protocol Based on Artificial Intelligence Algorithms With UAV-Assisted for Vehicular Ad Hoc Networks in Intelligent Transportation Systems" IEEE Trans. Intell. Transp. Syst. (2021) 10.1109/tits.2020.3041746

[9]

Yue "Software Defined Radio and Wireless Acoustic Networking for Amateur Drone Surveillance" IEEE Commun. Mag. (2018) 10.1109/mcom.2018.1700423

[10]

Mach "Mobile Edge Computing: A Survey on Architecture and Computation Offloading" IEEE Commun. Surv. Tutor. (2017) 10.1109/comst.2017.2682318

[11]

Zhang "A new algorithm of clustering AODV based on edge computing strategy in IOV" Wirel. Netw. (2021) 10.1007/s11276-021-02624-z

[12]

Zhang "New algorithm of multi-strategy channel allocation for edge computing" Int. J. Electron. Commun. (2020) 10.1016/j.aeue.2020.153372

[13]

Wang, S., Hu, L., Cao, L., Huang, X., Lian, D., and Liu, W. (2018, January 2–7). Attention-based transactional context embedding for next-item recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA. 10.1609/aaai.v32i1.11851

[14]

Feng "Latency Minimization of Reverse Offloading in Vehicular Edge Computing" IEEE Trans. Veh. Technol. (2022) 10.1109/tvt.2022.3151806

[15]

Zhang "Task Offloading in Vehicular Edge Computing Networks: A Load-Balancing Solution" IEEE Trans. Veh. Technol. (2020) 10.1109/tvt.2019.2959410

[16]

Xu "Energy efficient multi-resource computation offloading strategy in mobile edge computing" Jisuanji Jicheng Zhizao Xitong/Comput. Integr. Manuf. Syst. (2019)

[17]

Yang "Mobile edge computing empowered energy efficient task offloading in 5G" IEEE Trans. Veh. Technol. (2018) 10.1109/tvt.2018.2799620

[18]

Chen, H., Todd, T.D., Zhao, D., and Karakostas, G.J. (2023). Wireless and Service Allocation for Mobile Computation Offloading with Task Deadlines. arXiv. 10.1109/tmc.2023.3301577

[19]

Deep Reinforcement Learning: A Brief Survey

Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage et al.

IEEE Signal Processing Magazine 2017 10.1109/msp.2017.2743240

[20]

Liao "Intelligent dynamic service pricing strategy for multi-user vehicle-aided MEC networks" Futur. Gener. Comput. Syst. (2021) 10.1016/j.future.2020.07.051

[21]

A Hybrid DQN and Optimization Approach for Strategy and Resource Allocation in MEC Networks

Yi-Chen Wu, Thinh Quang Dinh, Yaru Fu et al.

IEEE Transactions on Wireless Communications 2021 10.1109/twc.2021.3057882

[22]

Chen "Efficiency and Fairness Oriented Dynamic Task Offloading in Internet of Vehicles" IEEE Trans. Green Commun. Netw. (2022) 10.1109/tgcn.2022.3167643

[23]

DQN-based mobile edge computing for smart Internet of vehicle

Lianhong Zhang, Wenqi Zhou, Junjuan Xia et al.

EURASIP Journal on Advances in Signal Processing 2022 10.1186/s13634-022-00876-1

[24]

Elgendy "Joint computation offloading and task caching for multi-user and multi-task MEC systems: Reinforcement learning-based algorithms" Wirel. Netw. (2021) 10.1007/s11276-021-02554-w

[25]

Hazarika "DRL-Based Resource Allocation for Computation Offloading in IoV Networks" IEEE Trans. Ind. Inform. (2022) 10.1109/tii.2022.3168292

[26]

Wu "Delay Constrained Hybrid Task Offloading of Internet of Vehicle: A Deep Reinforcement Learning Method" IEEE Access (2022) 10.1109/access.2022.3206359

[27]

Joint Service Caching and Computation Offloading Scheme Based on Deep Reinforcement Learning in Vehicular Edge Computing Systems

Zheng Xue, Chang Liu, Canliang Liao et al.

IEEE Transactions on Vehicular Technology 2023 10.1109/tvt.2023.3234336

[28]

Hazarika, B., Singh, K., Biswas, S., Mumtaz, S., and Li, C.P. (2023). Multi-Agent DRL-Based Task Offloading in Multiple RIS-Aided IoV Networks. IEEE Trans. Veh. Technol., 1–15. 10.1109/tvt.2023.3302010

[29]

Chen "Decentralized computation offloading for multi-user mobile edge computing: A deep reinforcement learning approach" J. Wireless Commn. Netw. (2020) 10.1186/s13638-020-01801-6

[30]

Deep Reinforcement Learning with Double Q-Learning

Hado Van Hasselt, Arthur Guez, David Silver

Proceedings of the AAAI Conference on Artificial I... 10.1609/aaai.v30i1.10295

[31]

Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10–15). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.

[32]

Hu "Deep reinforcement learning based offloading decision algorithm for vehicular edge computing" PeerJ Comput. Sci. (2022) 10.7717/peerj-cs.1126

Metrics

9

Citations

32

References

Details

Published: Oct 24, 2023
Vol/Issue: 12(21)
Pages: 4392
License: View

Authors

Z

Ziang Liu

School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China

Z

Zongpu Jia

School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China

X

Xiaoyan Pang

School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China

Funding

National Natural Science Foundation of China Youth Fund Award: 62202145

Cite This Article

Ziang Liu, Zongpu Jia, Xiaoyan Pang (2023). DRL-Based Hybrid Task Offloading and Resource Allocation in Vehicular Networks. Electronics, 12(21), 4392. https://doi.org/10.3390/electronics12214392

DRL-Based Hybrid Task Offloading and Resource Allocation in Vehicular Networks

You May Also Like