Optimal continuous control of refrigerator for electricity cost minimization—Hierarchical reinforcement learning approach

Abstract

A refrigerator is a commonly used household appliance; however, limited research has focused on optimizing temperature control policy with the consideration of Time-of-Use (ToU) electricity price. This paper introduces a novel framework that utilizes hierarchical reinforcement learning (HRL) to control the intensity of refrigerator motors. The objective is to achieve both temperature regulation and cost savings under ToU and stochastic usage patterns. The problem is tackled by two HRL agents. The high-level agent is responsible for determining temperature reference based on ToU pricing, while the low-level agent adjusts the motor intensity to meet this temperature reference. To tackle non-stationarity in HRL, the high-level agent employs hindsight action transition and reward function approximation, while the low-level agent employs hindsight goal transition. Through the experimental evaluation, we found that the proposed method outperforms the conventional control methods and standard reinforcement learning approaches. It achieves the lowest total costs, resulting in a significant cost reduction of 5%–24%.