A two-stage spatial prediction modeling approach based on graph neural networks and neural processes

Abstract

Spatial prediction models hold significant application value in fields such as environmental science, economic development, and geological exploration. With advancements in deep learning technology, graph neural networks (GNNs) offer a powerful and scalable solution for spatial data modeling. In these models, each spatial location is represented as a vertex, with the explanatory variables at each point converted into vertex features, and the vertex label corresponding to the target value at that point. GNNs utilize this representation to predict vertex labels, thereby performing spatial prediction. However, the residuals of GNN predictions are found to still exhibit spatial autocorrelation, indicating that traditional GNNs only achieve suboptimal results in terms of capturing complex spatial relationships. In this paper, we propose a two-stage spatial prediction method called Location Embedded Graph Neural Networks-Residual Neural Processes (LEGNN-RNP) to address this challenge. In the first stage, we employ LEGNNs, a new framework that integrates location features into GNNs through an attention mechanism, enhancing the learning of complex spatial models by considering both spatial context and attribute features. In the second stage, we model the residuals from LEGNN predictions to further extract spatial patterns from the data. Specifically, we introduce RNP, a neural process-based approach to model the Gaussian Process (GP) distribution of residuals. The distribution parameters (i.e., mean and covariance) are parameterized by a neural network, where the mean estimates the residual and the variance quantifies the prediction uncertainty. Experiments on four datasets demonstrate that our proposed two-stage method achieves state-of-the-art results by effectively extracting spatial relationships and significantly improving spatial prediction accuracy.