CNN-Enhanced Heterogeneous Graph Convolutional Network: Inferring Land Use from Land Cover with a Case Study of Park Segmentation

Abstract

Land use segmentation is a fundamental yet challenging task in remote sensing. Most current methods mainly take images as input and sometimes cannot achieve satisfactory results due to limited information. Inspired by the inherent relations between land cover and land use, we investigate land use segmentation using additional land cover data. The topological relations among land cover objects are beneficial for bridging the semantic gap between land cover and land use. Specifically, these relations are usually depicted by a geo-object-based graph structure. Deep convolutional neural networks (CNNs) are capable of extracting local patterns but fail to efficiently explore topological relations. In contrast, contextual relations among objects can be easily captured by graph convolutional networks (GCNs). In this study, we integrated CNNs and GCNs and proposed the CNN-enhanced HEterogeneous Graph Convolutional Network (CHeGCN) to incorporate local spectral-spatial features and long-range dependencies. We represent topological relations by heterogeneous graphs which are constructed with images and land cover data. Afterwards, we employed GCNs to build topological relations by graph reasoning. Finally, we fused CNN and GCN features to accomplish the inference from land cover to land use. Compared with other homogeneous graph-based models, the land cover data provide more sufficient information for graph reasoning. The proposed method can achieve the transformation from land cover to land use. Extensive experiments showed the competitive performance of CHeGCN and demonstrated the positive effects of land cover data. On the IoU metric over two datasets, CHeGCN outperforms CNNs and GCNs by nearly 3.5% and 5%, respectively. In contrast to homogeneous graphs, heterogeneous graphs have an IoU improvement of approximately 2.5% in the ablation experiments. Furthermore, the generated visualizations help explore the underlying mechanism of CHeGCN. It is worth noting that CHeGCN can be easily degenerated to scenarios where no land cover information is available and achieves satisfactory performance.