Fusion of Hyperspectral and Lidar Data Based On Dual-Branch Convolutional Neural Network

Abstract

With to the development of sensors, the fusion of features from multisource data becomes an interesting but challenging problem. In this paper, the fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data is investigated with a novel and simplified deep learning architecture, named the dual-branch convolutional neural network (DB-CNN). More specifically, a 3D CNN framework as one of the two branches is used to extract spectral-spatial features simultaneously from HSI, which can keep three-dimensional structural characteristics of HSI. Another one is 2D CNN with cascade blocks, which is developed to extract elevation feature from LiDAR data, and it can exploit the multiscale features. Finally, the features of two branches will be flattened and stacked, and then sent to the fully connected layers. The experiments show that the proposed DB-CNN method can effectively fuse the HSI and LiDAR data, and yield higher classification performance than some existing methods.