An approach for automatic parameters evaluation in unconventional oil reservoirs with deep reinforcement learning

Abstract

Accurate estimation of unconventional reservoir parameters is of great significance to improve the development effect and prolong the life cycle of production wells. Reservoir parameter estimation based on pressure transient analysis (PTA) is a mainstream method due to its ease of use. However, the non-unique solution and human bias make the reliability of the method less than ideal. Therefore, a robust automatic interpretation method is urgently needed to alleviate these problems. In this work, we propose an automatic parameter evaluation method of unconventional reservoirs based on a combination of deep reinforcement learning (DRL) and PTA. Our key insight is to treat the PTA process, namely the pressure derivative curve matching process, as Markov decision process (MDP) and solve the optimal matching policy through DRL algorithm. Based on this idea, we trained an agent to automatically adjust the parameters of the trilinear flow model, a classic PTA model, and finally complete the pressure derivative curve matching to evaluate the unconventional oil reservoirs parameters. To make the training converge, branch deep Q-network with independent rewards strategy (IR-BDQ) was proposed to train the agent. Results show that IR-BDQ algorithm can effectively improve the convergence speed and parameter evaluation accuracy compared with other DRL algorithms. The results of 1000 curve matching tests showed that the mean average relative errors of parameters is 13.1%. In addition, comparison with the supervised learning algorithm reveals that the proposed method has the smallest variance of parameter inversion errors, indicating that the method has good robustness. Finally, the case study shows that the proposed method can effectively alleviate the non-unique solution problem, which is of great significance to improve the repeatability of parameter evaluation results in unconventional oil reservoirs.