Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Scheduling Real-time Traffic in Wireless Networks: A Network-aided Offline Reinforcement Learning Approach

Published

Author(s)

Tao Zhang, Jialin Wan, Sen Lin, Zhaofeng Zhang, Junshan Zhang

Abstract

5G and Beyond (B5G) technology promises to offer ultra-reliable low-latency communications (URLLC) services, which opens the door for a wide variety of new real-time applications. Real-time traffic has stringent requirements in terms of latency, and low latency and deadline guarantees on packet delivery play a vital role in these real-time applications. Deadline-aware wireless scheduling of real-time traffic has been a long-standing open problem, despite significant efforts using analytical methods. Departing from the conventional approaches, this work studies deadline-aware traffic scheduling by taking an offline reinforcement learning (RL) approach to train scheduling algorithms, ready to be used for online scheduling. To address the challenges therein, we propose a Network-Aided Offline RL (NA-ORL) framework for deadline-aware scheduling, by making use of the fact that the network dynamics follows a well-defined physics model. Specifically, in NA-ORL the initialization of the scheduling policy is obtained through behavior cloning with a good model-based scheduling algorithm (Adaptive Mixing over Non-Dominated links (AMIX-ND) in this paper), and the network-aided actor-critic (A-C) method is utilized to train a better scheduling policy with carefully designed states and reward function, thanks to its nature of policy improvement. Building on NA-ORL, we further devise a Network-Aided Offline Meta-RL (NAMRL) algorithm to deal with the non-stationary network dynamics. Extensive experiments are conducted to evaluate the performance of both NA-ORL and NA-MRL. The experimental results clearly demonstrate that the proposed NA-ORL and NA-MRL algorithms can achieve better performance over AMIX-ND, in various scenarios for the deadline-aware wireless scheduling. Permission
Citation
IEEE Internet of Things Journal

Keywords

end-to-end quality of services, resource allocation

Citation

Zhang, T. , Wan, J. , Lin, S. , Zhang, Z. and Zhang, J. (2023), Scheduling Real-time Traffic in Wireless Networks: A Network-aided Offline Reinforcement Learning Approach, IEEE Internet of Things Journal, [online], https://doi.org/10.1109/JIOT.2023.3304969, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934470 (Accessed May 23, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created August 14, 2023, Updated February 9, 2024