Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Towards Deep Q-Network Based Resource Allocation in Industrial Internet of Things

Published

Author(s)

Fan Liang, Wei Yu, Xing Liu, David W. Griffith, Nada T. Golmie

Abstract

With the increasing adoption of Industrial Internet of Things (IIoT) devices, infrastructures, and supporting applications, it is critical to design schemes to effectively allocate resources (e.g., networking, computing, and energy) in IIoT systems, generally formalized as optimization problems. Nonetheless, because the system is highly complex, operation environments are time-varying, and required information may not be available, it is difficult to leverage traditional optimization techniques to solve the optimal resource allocation problem. To this end, in this paper we propose a Deep Q-Network (DQN) based scheme to address both bandwidth utilization and energy efficiency in an IIoT system. In detail, we design a DQN model that consists of two deep neural networks (DNN) and a Q-learning model. The DNN network abstracts the features from the highly dimensional inputs and obtains the approximate Q-function for the Q-learning model. Based on the Q-function, the Q-learning model can generate the Q-table and reward function. After the training process, the DQN model can select appropriate actions for the agents (i.e., robots in smart warehouse in this study) to improve bandwidth utilization and energy efficiency. To evaluate our proposed scheme, we design a simulation environment from which to investigate a typical IIoT scenario: the actuation of robotics in a smart warehouse. We then implement the DQN model and conduct extensive experiments to validate the efficacy of our scheme. Our experimental results confirm that our scheme can improve both bandwidth utilization and energy efficiency, as compared to other representative schemes.
Citation
IEEE Internet of Things Journal

Keywords

Industrial IoT, Deep Q-Network, Non-Orthogonal Multiple Access, Resource Management

Citation

Liang, F. , Yu, W. , Liu, X. , Griffith, D. and Golmie, N. (2021), Towards Deep Q-Network Based Resource Allocation in Industrial Internet of Things, IEEE Internet of Things Journal, [online], https://doi.org/10.1109/JIOT.2021.3093346, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931907 (Accessed April 17, 2024)
Created June 29, 2021, Updated November 29, 2022