HSI Classification Based on Multimodal CNN and Shadow Enhance by DSR Spatial-Spectral Fusion

Abstract

Dynamic stochastic resonance (DSR) has been proved to be able to enhance the shadow region of HSIs. However, DSR can only be used for the enhancement on the spatial or spectral dimensions respectively, which destroy the correlation and integrality of HSI. Therefore, in this paper, to maintain the 3D characteristic of HSI, the spatial and spectral enhanced HSI has been fused. Furthermore, to make full use of two-modal information of HSI, a multimodal convolutional neural net-work () has been proposed for classification. In both 2D spatial modal and 3D tensor information can be integrated to improve the classification. Firstly, the shadow areas of HSI could be enhanced by DSR from spatial and spectral dimensions to get spatial enhanced HIS and spectral enhanced HSI respectively. Secondly, the enhanced HIS can be obtained by fuzing and in different proportions. Finally, can be classified by Two scenes containing shadow areas of a real-world HSI have been used in the experiment to evaluate the performance of the proposed method. The results have shown that the proposed method has a promising prospect in the classification of HSI with shadow.