Towards Deep Q-Network Based Resource Allocation in Industrial Internet of Things

Published

June 29, 2021

Author(s)

Fan Liang, Wei Yu, Xing Liu, David W. Griffith, Nada T. Golmie

Abstract

With the increasing adoption of Industrial Internet of Things (IIoT) devices, infrastructures, and supporting applications, it is critical to design schemes to effectively allocate resources (e.g., networking, computing, and energy) in IIoT systems, generally formalized as optimization problems. Nonetheless, because the system is highly complex, operation environments are time-varying, and required information may not be available, it is difficult to leverage traditional optimization techniques to solve the optimal resource allocation problem. To this end, in this paper we propose a Deep Q-Network (DQN) based scheme to address both bandwidth utilization and energy efficiency in an IIoT system. In detail, we design a DQN model that consists of two deep neural networks (DNN) and a Q-learning model. The DNN network abstracts the features from the highly dimensional inputs and obtains the approximate Q-function for the Q-learning model. Based on the Q-function, the Q-learning model can generate the Q-table and reward function. After the training process, the DQN model can select appropriate actions for the agents (i.e., robots in smart warehouse in this study) to improve bandwidth utilization and energy efficiency. To evaluate our proposed scheme, we design a simulation environment from which to investigate a typical IIoT scenario: the actuation of robotics in a smart warehouse. We then implement the DQN model and conduct extensive experiments to validate the efficacy of our scheme. Our experimental results confirm that our scheme can improve both bandwidth utilization and energy efficiency, as compared to other representative schemes.

Citation

IEEE Internet of Things Journal

Pub Type

Journals

Download Paper

https://doi.org/10.1109/JIOT.2021.3093346

Local Download

Keywords

Industrial IoT, Deep Q-Network, Non-Orthogonal Multiple Access, Resource Management

Citation

Liang, F. , Yu, W. , Liu, X. , Griffith, D. and Golmie, N. (2021), Towards Deep Q-Network Based Resource Allocation in Industrial Internet of Things, IEEE Internet of Things Journal, [online], https://doi.org/10.1109/JIOT.2021.3093346, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931907 (Accessed April 23, 2024)