A deep reinforcement learning (DRL) based approach for well-testing interpretation to evaluate reservoir parameters

Abstract

Parameter inversions in oil/gas reservoirs based on well test interpretations are of great significance in oil/gas industry. Automatic well test interpretations based on artificial intelligence are the most promising to solve the problem of non-unique solution. In this work, a new deep reinforcement learning (DRL) based approach is proposed for automatic curve matching for well test interpretation, by using the double deep Q-network (DDQN). The DDQN algorithms are applied to train agents for automatic parameter tuning in three conventional well-testing models. In addition, to alleviate the dimensional disaster problem of parameter space, an asynchronous parameter adjustment strategy is used to train the agent. Finally, field applications are carried out by using the new DRL approaches. Results show that step number required for the DDQN to complete the curve matching is the least among, when comparing the naive deep Q-network (naive DQN) and deep Q-network (DQN). We also show that DDQN can improve the robustness of curve matching in comparison with supervised machine learning algorithms. Using DDQN algorithm to perform 100 curve matching tests on three traditional well test models, the results show that the mean relative error of the parameters is 7.58% for the homogeneous model, 10.66% for the radial composite model, and 12.79% for the dual porosity model. In the actual field application, it is found that a good curve fitting can be obtained with only 30 steps of parameter adjustment.