A latent space method with maximum entropy deep reinforcement learning for data assimilation

Abstract

Data assimilation aims to calibrate the uncertain state or parameter vectors of a system by matching simulation results with observations, which is crucial for uncertainty quantification and optimization. Although traditional methods of data assimilation have shown promising results in practical applications, they need well-designed iteration rules (i.e., gradient, covariance, search strategies). Deep reinforcement learning (DRL) can solve data assimilation problems by trial and error, which does not require convoluted iteration rules. However, previous DRL methods cannot tackle high-dimensional data assimilation problems efficiently. In this work, we propose a latent space method with maximum entropy DRL for data assimilation to extend DRL to complicated systems with lots of parameters. Solutions can be found in a reduced space, through the interaction between the agent represented by artificial neural networks and the environment of numerical simulation. The proposed method contains two key points. First, to make the method applicable to high-dimensional problems, we construct a latent space method by integrating a dimensionality reduction method with the state, action, and reward settings for the agent. Second, a maximum entropy DRL algorithm, Soft Actor–Critic (SAC), was employed to efficiently explore the parameter space. The proposed method overcomes the limitation that previous DRL algorithms are only suitable for small-scale cases, and enables DRL to deal with medium-scale or even large-scale data assimilation problems. The performance of the proposed method was validated by comparison with a deep reinforcement learning algorithm and traditional data assimilation algorithms on 2D synthetic and 3D reservoir cases.