Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Towards Online Continuous Reinforcement Learning on Industrial Internet of Things

Published

Author(s)

Cheng Qian, Wei Yu, Xing Liu, David W. Griffith, Nada T. Golmie

Abstract

In Industrial Internet of Things (IIoT), the information and communication technologies powered by IoT can greatly improve the efficiency and timeliness of information exchange between different industrial components. Likewise, machine learning techniques such as reinforcement learning can benefit from the massive amounts of data collected via IoT to successfully automate industrial monitoring and control processes. Nonetheless, training machine learning models, such as reinforcement learning models, require a significant investment of time, and a trained model can only work on a specific system in a specific environment. When the application scenario of reinforcement learning changes, or the application environment changes, the reinforcement learning model needs to be retrained. Thus, it is critical to design techniques that can reduce the overhead of retraining reinforcement learning models, enabling them adapt to constantly changing environments. In this paper, toward improving the performance of learning models in dynamic IIoT, we propose an online continuous reinforcement learning strategy. In our process, when the retraining condition is triggered, our online continuous learning strategy will re-engage the training process and update the well-trained model. To evaluate the performance of our proposed approach, we categorize the entire application space for applying reinforcement learning to IIoT systems into four scenarios, namely, non-continuous learning without learning model sharing, non-continuous learning with learning model sharing, continuous learning without learning model sharing, and continuous learning without learning model sharing. For each scenario, we design a Q-learning based reinforcement learning algorithm. Via extensive evaluation, our results show that the online continuous reinforcement learning approach that we propose can significantly reduce the overhead of retraining the learning model, enabling the learning algorithm to quickly adapt to a changing environment.
Conference Dates
October 18-21, 2021
Conference Location
Atlanta, GA, US
Conference Title
2021 IEEE Smart World Congress

Keywords

Industrial Internet of Things, Online continuous learning, Reinforcement learning

Citation

Qian, C. , Yu, W. , Liu, X. , Griffith, D. and Golmie, N. (2021), Towards Online Continuous Reinforcement Learning on Industrial Internet of Things, 2021 IEEE Smart World Congress, Atlanta, GA, US, [online], https://doi.org/10.1109/SWC50871.2021.00046, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931905 (Accessed August 16, 2022)
Created October 19, 2021, Updated February 8, 2022