Towards Online Continuous Reinforcement Learning on Industrial Internet of Things

Abstract

Training machine learning models, such as reinforcement learning models, require a significant investment of time, and a trained model can only work on a specific system in a specific environment. When the application scenario of reinforcement learning changes, or the application environment changes, the reinforcement learning model needs to be retrained. Thus, it is critical to design techniques that can reduce the overhead of retraining reinforcement learning models, enabling them adapt to constantly changing environments. In this paper, toward improving the performance of learning models in dynamic Industrial Internet of Things (IIoT), we propose an online continuous reinforcement learning strategy. In our process, when the retraining condition is triggered, our online continuous learning strategy will re-engage the training process and update the well-trained model. To evaluate the performance of our proposed approach, we categorize the entire application space for applying reinforcement learning to IIoT systems into four scenarios, namely, non-continuous learning without learning model sharing, non-continuous learning with learning model sharing, continuous learning without learning model sharing, and continuous learning without learning model sharing. For each scenario, we design a Q-learning based reinforcement learning algorithm. Via extensive evaluation, our results show that the online continuous reinforcement learning approach that we propose can significantly reduce the overhead of retraining the learning model, enabling the learning algorithm to quickly adapt to a changing environment.