Synergistic 2D/3D Convolutional Neural Network for Hyperspectral Image Classification

Xiaofei Yang,Xiaohui Huang,Xiaofeng Zhang,Raymond Y K Lau,Shijian Lu,Yunming Ye,Xutao Li

doi:10.3390/rs12122033

Abstract

Accurate hyperspectral image classification has been an important yet challenging task for years. With the recent success of deep learning in various tasks, 2-dimensional (2D)/3-dimensional (3D) convolutional neural networks (CNNs) have been exploited to capture spectral or spatial information in hyperspectral images. On the other hand, few approaches make use of both spectral and spatial information simultaneously, which is critical to accurate hyperspectral image classification. This paper presents a novel Synergistic Convolutional Neural Network (SyCNN) for accurate hyperspectral image classification. The SyCNN consists of a hybrid module that combines 2D and 3D CNNs in feature learning and a data interaction module that fuses spectral and spatial hyperspectral information. Additionally, it introduces a 3D attention mechanism before the fully-connected layer which helps filter out interfering features and information effectively. Extensive experiments over three public benchmarking datasets show that our proposed SyCNNs clearly outperform state-of-the-art techniques that use 2D/3D CNNs.

Highlights

With the rapid development of optics and photonics, hyperspctral sensors have been installed in many satellites
We first propose a simple synergistic trained deep learning model, which is constructed by mixing 2D convolutional neural networks (CNNs) and 3D CNNs to extract deeper spatial–spectral features with fewer 2D/3D convolutions
We further present a deep Synergistic Convolutional Neural Network (SyCNN) network for hyperspectral image classification, which introduces a data interaction module into the simple synergistic trained 2D/3D model

Summary

Introduction

With the rapid development of optics and photonics, hyperspctral sensors have been installed in many satellites. Based on the rich spatial–spectral information preserved in hyperspectral images, it enables us to distinguish different objects of interest in the scene. They have been widely used in a variety of fields such as precise agriculture, environmental surveillance, and astronomy [1]. Wang et al [6] proposed a dimensionality reduction method for hyperspectral image classification by utilizing the manifold ranking algorithm to perform band selection. Yuan et al [7] proposed a novel dual clustering-based band selection approach for hyperspectral image classification. While these methods have demonstrated superior classification performance, they are not effective to classify hyperspectral images under complex scenarios

Methods

Results

Conclusion