Stochastic control of geological carbon storage operations using geophysical monitoring and deep reinforcement learning

Abstract

Geological carbon storage (GCS) is the process of injecting and storing carbon dioxide (CO2) in the subsurface to reduce greenhouse gas emissions. Safe and profitable GCS operations require effective decision-making in the presence of uncertain geological models, a process which can often be facilitated with geophysical monitoring. In this study, we examine how sequential decision-making algorithms can be combined with geophysical measurements for the optimal control of GCS operations. Specifically, we develop an artificial intelligence model using deep reinforcement learning (DRL) that takes geophysical time-lapse gravity and well pressure monitoring data as input and delivers an optimal CO2 injection policy. The objective of the problem at hand is to maximize the profit of a hypothetical GCS operation while mitigating the potential for induced seismicity, by training DRL agents using combined geostatistical, reservoir and geophysical simulation. Comparisons against two benchmarks – a constant injection strategy and an injection schedule optimized using a commercial reservoir simulator toolbox – show that the stochastic control of such operations from subsurface monitoring data using deep reinforcement learning is feasible. Evaluation results show that DRL agent behavior generates profits which are on average 1 to 8 percent higher than what is possible through a constant injection approach. Furthermore, we show that DRL can generate optimal injection policies applicable to the true (yet previously unseen) subsurface given carefully managed levels of uncertainty.