Sample-efficient reinforcement learning with knowledge-embedded hybrid model for optimal control of mining industry

Abstract

The froth flotation process is a cost-effective and widely employed method for mineral separation. A core challenge is to maximize mineral recovery while maintaining a specified minimum concentrate grade. This requires precise control of aeration and slurry level and overcoming disturbances. The rapid advancements in reinforcement learning present a promising approach for controlling froth flotation. However, reinforcement learning suffers from sample inefficiency, which significantly hampers its application to real-world problems. In this research, we proposed an innovative sample-efficient model-based reinforcement learning algorithm to enhance flotation performance by directly leveraging the dynamics of the slurry phase. Our proposed algorithm involves the construction of a hybrid model, comprising a physical model and a data residual model, which effectively learns the underlying dynamics of the flotation process during the interaction between the reinforcement learning agent and process. The physical model integrates domain knowledge into the hybrid model to expedite the training process and enhance the interpretability and generalizability of the hybrid model. To capture uncertainty and address unmodeled aspects of the physical model, we employ an ensemble data residual model. Actor and critic are augmented with fuzzy representation modules based on fuzzy logic inference to improve the learning capacity of the networks and the multi-head critic is applied to reduce overestimation biases and enhance training stability. Case studies demonstrate that, compared to the baseline algorithm, our proposed algorithm requires merely 86.8% of the samples employed by conventional model-based reinforcement learning algorithm and improves the long-term return by at least 12.0%, underscoring the role of domain knowledge in facilitating efficient reinforcement learning.