Towards Online Continuous Reinforcement Learning on Industrial Internet of Things

Published

October 19, 2021

Author(s)

Cheng Qian, Wei Yu, Xing Liu, David W. Griffith, Nada T. Golmie

Abstract

In Industrial Internet of Things (IIoT), the information and communication technologies powered by IoT can greatly improve the efficiency and timeliness of information exchange between different industrial components. Likewise, machine learning techniques such as reinforcement learning can benefit from the massive amounts of data collected via IoT to successfully automate industrial monitoring and control processes. Nonetheless, training machine learning models, such as reinforcement learning models, require a significant investment of time, and a trained model can only work on a specific system in a specific environment. When the application scenario of reinforcement learning changes, or the application environment changes, the reinforcement learning model needs to be retrained. Thus, it is critical to design techniques that can reduce the overhead of retraining reinforcement learning models, enabling them adapt to constantly changing environments. In this paper, toward improving the performance of learning models in dynamic IIoT, we propose an online continuous reinforcement learning strategy. In our process, when the retraining condition is triggered, our online continuous learning strategy will re-engage the training process and update the well-trained model. To evaluate the performance of our proposed approach, we categorize the entire application space for applying reinforcement learning to IIoT systems into four scenarios, namely, non-continuous learning without learning model sharing, non-continuous learning with learning model sharing, continuous learning without learning model sharing, and continuous learning without learning model sharing. For each scenario, we design a Q-learning based reinforcement learning algorithm. Via extensive evaluation, our results show that the online continuous reinforcement learning approach that we propose can significantly reduce the overhead of retraining the learning model, enabling the learning algorithm to quickly adapt to a changing environment.

Conference Dates

October 18-21, 2021

Conference Location

Atlanta, GA, US

Conference Title

2021 IEEE Smart World Congress

Pub Type

Conferences

Download Paper

https://doi.org/10.1109/SWC50871.2021.00046

Local Download

Keywords

Industrial Internet of Things, Online continuous learning, Reinforcement learning

Citation

Qian, C. , Yu, W. , Liu, X. , Griffith, D. and Golmie, N. (2021), Towards Online Continuous Reinforcement Learning on Industrial Internet of Things, 2021 IEEE Smart World Congress, Atlanta, GA, US, [online], https://doi.org/10.1109/SWC50871.2021.00046, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=931905 (Accessed April 26, 2024)