Deep Reinforcement Learning Assisted Beam Tracking and Data Transmission for 5G V2X Networks

Published

February 6, 2023

Author(s)

Junliang Ye, Hamid Gharavi

Abstract

Beam tracking is a core issue in 5G vehicle-to-everything (V2X) networks. Specifically, higher beamforming gain is required to compensate for the path loss at higher frequencies, e.g., 5G FR2, to realize high data rate vehicle-to-infrastructure (V2I) communications. Bear in mind that shorter time slots at higher frequencies, high velocity of vehicles, and unpredictable localization errors make this problem more challenging. Under these circumstances, wider beams can lead to higher beam tracking accuracy. However, wider beams mean lower beamforming gain, which cannot compensate for high path loss at high frequencies and would further influence the data rate of V2I communications. Thus, there exists a trade-off between tracking accuracy and data rate in V2I communications. Furthermore, this problem needs to be solved within an extremely short time slot according to the high transmission frequency. To solve this problem, we propose a reinforcement learning (RL) assisted, high-resolution codebook-based beam tracking method. By comparing several different RL frameworks, we found that the twin delayed deep deterministic policy gradient (TD3) framework can help the roadside infrastructure (RSI) determine a proper beam pattern within a short duration. Moreover, according to the Hurst exponent analysis, recurrent neural networks (RNNs) are selected to improve the performance of the RL framework. The simulation results show that the proposed method performs well in tracking accuracy, data rate, and temporal efficiency.

Citation

IEEE Transactions on Intelligent Transportation Systems

Volume

71

Pub Type

Journals

Download Paper

https://doi.org/10.1109/TSP.2023.3236166

Local Download

Keywords

5G, Machin learning, vehicular communications, V2I, beam tracking, millimeter wave communications.

Citation

Ye, J. and Gharavi, H. (2023), Deep Reinforcement Learning Assisted Beam Tracking and Data Transmission for 5G V2X Networks, IEEE Transactions on Intelligent Transportation Systems, [online], https://doi.org/10.1109/TSP.2023.3236166, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=935638 (Accessed April 29, 2024)