Characterization of groundwater contamination: A transformer-based deep learning model

Abstract

Identification of groundwater contaminant sources in a highly-heterogenous geosystems results in a high-dimensional inverse problem and is often solved based on a surrogate model to alleviate the computational burden. Surrogate modeling through deep learning has a great potential for learning complex nonlinear relationships between model inputs and outputs. Most of the developed surrogates, however, can only estimate the contaminant concentration fields at a limit number of time-steps with a relatively large lag. In this paper, a transformer-based surrogate model is developed to provide a detailed release history of contaminant, which allows more accurate analyzing the distributions and planning. As such, a Koopman-operator-based convolutional autoencoder is trained and fixed prior to the training of transformer. Here, the encoder converts the concentration fields into a one-dimensional embedding space and the transformer is trained on this space to learn the system dynamics and predict the embedding feature at next time step, which is reconstructed back to the original space with the decoder. The proposed surrogate model is tested on a complex problem and the results demonstrate that the proposed transformer-based surrogate can efficiently provide an accurate estimation of the evolution of contaminant concentration field at a greater number of time-steps compared to the previous works.