Fully Contextual Network for Hyperspectral Scene Parsing

Abstract

In this article, we propose fully contextual networks (FullyContNets) for hyperspectral scene parsing. Different from the previous approaches that leveraging the local information, the proposed methods can effectively capture the more generic nonlocal contexts. To this end, we first propose the scale attention module (SAM) that can adaptively aggregate the multiple features through obtaining the interfeature dependencies of multiscale with self-attention mechanism, where the weights are determined by measuring the similarity between features. What is more, two fully contextual modules (FCMs) called pyramid fully contextual module (Pyramid-FCM) and atrous spatial pyramid fully contextual module (ASP-FCM) are separately developed to obtain the contextual information that simultaneously lying across positions, channels, and features when combining the intrafeature information aggregation algorithms with SAM on the foundation of existing multiscale modules, such as pyramid pooling (PP) in PSPNet and atrous spatial pyramid pooling (ASPP) in DeeplabV3. We design four schemes for FCMs to obtain more effective contexts. The corresponding FullyContNet-Pyramid and FullyContNet-ASP are separately constructed based on the Pyramid-FCM or ASP-FCM. There are extensive quantitative and qualitative experiments are conducted, depicting the capability of SAM and FCMs and demonstrating the competitiveness of proposed networks on four public hyperspectral scenes when comparing with the current state-of-the-art approaches.