Scheduling Real-time Traffic in Wireless Networks: A Network-aided Offline Reinforcement Learning Approach

Published

August 14, 2023

Author(s)

Tao Zhang, Jialin Wan, Sen Lin, Zhaofeng Zhang, Junshan Zhang

Abstract

5G and Beyond (B5G) technology promises to offer ultra-reliable low-latency communications (URLLC) services, which opens the door for a wide variety of new real-time applications. Real-time traffic has stringent requirements in terms of latency, and low latency and deadline guarantees on packet delivery play a vital role in these real-time applications. Deadline-aware wireless scheduling of real-time traffic has been a long-standing open problem, despite significant efforts using analytical methods. Departing from the conventional approaches, this work studies deadline-aware traffic scheduling by taking an offline reinforcement learning (RL) approach to train scheduling algorithms, ready to be used for online scheduling. To address the challenges therein, we propose a Network-Aided Offline RL (NA-ORL) framework for deadline-aware scheduling, by making use of the fact that the network dynamics follows a well-defined physics model. Specifically, in NA-ORL the initialization of the scheduling policy is obtained through behavior cloning with a good model-based scheduling algorithm (Adaptive Mixing over Non-Dominated links (AMIX-ND) in this paper), and the network-aided actor-critic (A-C) method is utilized to train a better scheduling policy with carefully designed states and reward function, thanks to its nature of policy improvement. Building on NA-ORL, we further devise a Network-Aided Offline Meta-RL (NAMRL) algorithm to deal with the non-stationary network dynamics. Extensive experiments are conducted to evaluate the performance of both NA-ORL and NA-MRL. The experimental results clearly demonstrate that the proposed NA-ORL and NA-MRL algorithms can achieve better performance over AMIX-ND, in various scenarios for the deadline-aware wireless scheduling. Permission

Citation

IEEE Internet of Things Journal

Pub Type

Journals

Download Paper

https://doi.org/10.1109/JIOT.2023.3304969

Local Download

Keywords

end-to-end quality of services, resource allocation

Citation

Zhang, T. , Wan, J. , Lin, S. , Zhang, Z. and Zhang, J. (2023), Scheduling Real-time Traffic in Wireless Networks: A Network-aided Offline Reinforcement Learning Approach, IEEE Internet of Things Journal, [online], https://doi.org/10.1109/JIOT.2023.3304969, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934470 (Accessed April 28, 2024)