Automating hyperparameter optimization in geophysics with Optuna: A comparative study

Abstract

AbstractDeep learning has gained attraction amongst geophysicists for solving complex longstanding problems. Nevertheless, proper hyperparameter optimization methodologies remain critically underexplored in geophysical deep learning research. This paper attempts to first highlight the importance of hyperparameter optimization and then showcase two geophysics‐related deep learning examples where a grid search and Optuna framework (an automated optimization approach) were used for hyperparameter optimization. We consider two geophysical problems related to denoising seismic traces and the inversion of seismic traces for velocity information. In both cases, models created based on Optuna hyperparameter optimization were able to perform better than those created through grid search. The most significant advantage of Optuna, however, is having quantifiable results to justify the choice of a neural network architecture, depth and other hyperparameters rather than relying on inefficient methods of exploring the hyperparameter space such as a trial‐and‐error or grid search. This study aims to stimulate further exploration and adoption of these frameworks, pushing the boundaries of current deep learning based geophysical problem‐solving methodologies towards full automation.