GeomorPM: a geomorphic pretrained model integrating convolution and Transformer architectures based on DEM data

Abstract

As the domain of artificial intelligence has advanced, the integration of deep learning techniques into terrain and landform analysis has become more prevalent. Nevertheless, many existing methods are fully supervised and designed for specific tasks; thus, their transferability is limited and massive annotated samples are required. This study introduces a geomorphic pretrained model (GeomorPM) capable of performing multiple tasks. First, an architecture was designed that combined a convolution-based Vector Quantised-Variational Autoencoder (VQVAE) with a Transformer-based masked autoencoder (MAE) framework, allowing it to autonomously learn local details and global patterns from large-scale digital elevation model (DEM) data. Subsequently, GeomorPM, based on the VQMAE architecture, was pretrained on massive DEM data and fine-tuned for three specific tasks: DEM void filling, DEM superresolution, and landform classification. GeomorPM outperformed the traditional and other deep learning methods in all three tasks, demonstrating the superior learning ability and transferability of the model. This study provides a practical framework for developing pretrained models based on DEMs that can be expanded to other continuous geoscientific data.