Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Deep Reinforcement Learning Assisted Beam Tracking and Data Transmission for 5G V2X Networks



Junliang Ye, Hamid Gharavi


Beam tracking is a core issue in 5G vehicle-to-everything (V2X) networks. Specifically, higher beamforming gain is required to compensate for the path loss at higher frequencies, e.g., 5G FR2, to realize high data rate vehicle-to-infrastructure (V2I) communications. Bear in mind that shorter time slots at higher frequencies, high velocity of vehicles, and unpredictable localization errors make this problem more challenging. Under these circumstances, wider beams can lead to higher beam tracking accuracy. However, wider beams mean lower beamforming gain, which cannot compensate for high path loss at high frequencies and would further influence the data rate of V2I communications. Thus, there exists a trade-off between tracking accuracy and data rate in V2I communications. Furthermore, this problem needs to be solved within an extremely short time slot according to the high transmission frequency. To solve this problem, we propose a reinforcement learning (RL) assisted, high-resolution codebook-based beam tracking method. By comparing several different RL frameworks, we found that the twin delayed deep deterministic policy gradient (TD3) framework can help the roadside infrastructure (RSI) determine a proper beam pattern within a short duration. Moreover, according to the Hurst exponent analysis, recurrent neural networks (RNNs) are selected to improve the performance of the RL framework. The simulation results show that the proposed method performs well in tracking accuracy, data rate, and temporal efficiency.
IEEE Transactions on Intelligent Transportation Systems


5G, Machin learning, vehicular communications, V2I, beam tracking, millimeter wave communications.


Ye, J. and Gharavi, H. (2023), Deep Reinforcement Learning Assisted Beam Tracking and Data Transmission for 5G V2X Networks, IEEE Transactions on Intelligent Transportation Systems, [online],, (Accessed April 18, 2024)
Created February 6, 2023, Updated August 10, 2023