Condition-based maintenance with reinforcement learning for refrigeration systems with selected monitored features

Abstract

Worldwide, buildings are responsible for almost 30% of energy consumption, and those buildings that intensively use refrigeration systems, such as supermarkets and grocery stores, are also among the most energy-intensive consumers. Refrigeration devices, either commercial or residential, are responsible for a significant part of net emissions. Based on careful measurements, it is possible to reduce energy consumption in these devices by up to 15% only by improving the fault detection and diagnosis techniques. Thus, improving maintenance programs has become a crucial area in energy management in recent years. Nowadays, the market has experienced a hike after smart systems and new network interfaces applied to smart buildings that have allowed previously isolated devices to become smart devices, interacting with control algorithms smartly and, to some extent, autonomously. Here, we propose a reinforcement learning framework to develop a maintenance policy for mechanical compression refrigeration devices. Firstly, a test bench is built in which each component is assigned to be individually repairable and individually degradable in parallel and interconnected processes. Then, the degradation of the components is combined to formulate the system degradation, and the optimal maintenance policy is modeled via Markov decision processes and solved by a reinforcement learning algorithm. The agent-proposed maintenance program if compared to corrective maintenance, managed to reduce energy use and emissions by around 6% while avoiding shortfalls, as well as about the preventive program, where the agent managed to accomplish the same level of energy efficiency while reducing the maintenance costs by 31% and the time under maintenance in 10%. It was found that the reinforcement learning frameworks applied to maintenance have a series of challenges but are innovative and can show promising results compared to traditional maintenance techniques, such as preventive and corrective ones.