Towards a Machine Learning Based Dynamic Surrogate Modeling and Optimization of Steam Injection Policy in SAGD

Abstract

Abstract This paper presents a methodology for the identification of a dynamic-surrogate model and the optimization of steam injection rates of a multi-well heterogeneous SAGD process. The optimization refers to finding the steam injection rates at every time step (steam injection policy) that will maximize cumulative net present value at the end of the production horizon. The solution methodology consists of identifying one-step prediction non-linear models and then using these models in a recursive scheme to predict the established production horizon. These models are identified offline and then used as a substitute for the reservoir simulation model, considered computationally expensive, in the optimization process. This approach makes use of the reinforcement learning agent-environment interaction: based on the current state St, the agent takes an action At, and the environment transitions into a new state St+1 and offers a scalar reward Rt. Additionally, the well-known genetic algorithm is used for optimization purposes. The approach is applied to a multi-well reservoir simulation model, built using publicly available data that includes data from northern Alberta SAGD operations considering two (2) time step lengths: daily (Case 1) and weekly (Case 2). Furthermore, the performance of the approach is evaluated in terms of: i) Mean Absolute Error (MAE) between the predicted time-series and the true values (effectivity), ii) the effect of randomness of the design of experiments over the MAE (robustness regarding the design of experiments) and iii) changes in the variance of the errors over the prediction time frame (performance as number of time step prediction increases). Results show that for a daily time step (Case 1) the proposed approach was able to predict significantly well the selected output as opposed to Case 2 which exhibit much higher MAE values. Also, there is a small but important effect of the randomness of the design of experiments over the MAE values in both cases. Furthermore, Case 1 showed a significant higher level of robustness over the prediction than Case 2. In particular, the changes in variance of the error in Case 1 was much less that for Case 2.