Hyperspectral images typically have large-sized features and fuse a large amount of spatial and spectral data, increasing the complexity of feature selection and effective mining. In addition, its dimensional redundancy and feature intersection further exacerbate the interpretability problem of the model, limiting the overall improvement of classification performance. Therefore, this paper proposes an HSI classification model built on U-Nets and a graph convolutional network. This model utilizes multi-scale superpixel segmentation to enhance the flexibility of spatial structure modeling and achieves synchronous extraction of spatial topological relationships between land features from multiple scales through a multi-scale graph convolution architecture. The experiment showed that the proposed model achieved an F1 value of 96.7% on the comprehensive datasets (Indian Pines, Pavia University, Salinas, and GRSS 2013), demonstrating good robustness and generalization ability. Regardless of whether under interference conditions, the average classification entropy and average mutual information between categories of the proposed model were significantly lower than those of comparative models. Under the condition of random loss of some bands, the average classification entropy and average mutual information value between categories of the research model were 0.28 and 0.79, and 0.31 and 0.77 under Gaussian noise interference. The research model has strong discriminative ability in hyperspectral image classification tasks and effectively deals with complex scenes such as noise interference and data loss.

Scribble-supervised medical image segmentation based on dynamically generated pseudo labels via multi-scale superpixels

Shadow in remote sensing images can obscure important details of land features, making shadow detection crucial for enhancing the accuracy of subsequent analyses and applications. Current shadow detection methods primarily rely on the spectral information of images, which can often result in shadow misdetection due to the phenomenon of spectral confusion of different objects. To mitigate this issue, we propose a method that combines topography and spectra (CTS). Firstly, we introduce a new DEM-based shadow coarse detection method to obtain the DEM rough shadow mask, which uses a relationship between the magnitude of terrain height angle and solar elevation angle to determine shadow properties. Then, we use the MC3 (modified C3 component) index-based shadow fine detection method to obtain an MC3 mean map, which includes image enhancement with a stretching process and multi-scale superpixel segmentation. We then derive the Shadow pixel Proportion Map (SPM) by counting the DEM rough shadow mask in terms of superpixels. The Joint Shadow probability Map (JSM) is obtained by combining the SPM and the MC3 mean map with specific weights. Finally, a multi-level Otsu threshold method is applied to the JSM to generate the shadow mask. We compare the proposed CTS method against several state-of-the-art algorithms through both qualitative assessments and quantitative metrics. The results show that the CTS method demonstrates superior accuracy and consistency in detecting true shadows, achieving an average overall accuracy of 95.81% on mountainous remote sensing images.

Recently, superpixel segmentation has been widely employed in hyperspectral image (HSI) classification of remote sensing. However, the structures of land-covers in HSI commonly vary greatly, which makes it difficult to fully fit the boundaries of land-covers by single-scale superpixel segmentation. Moreover, the shape-irregularity of superpixel brings challenge for depth feature extraction. To overcome these issues, a multiscale superpixel depth feature extraction (MSDFE) method is proposed for HSI classification in this article, which effectively explores and integrates the spatial-spectral information of land-covers by adopting multiscale superpixel segmentation, constructing statistical features of superpixel, and conducting depth feature extraction. Specifically, to exploit rich spatial information of HSI, multiscale superpixel segmentation is firstly applied on the HSI. Once superpixels on different scales are obtained, two-dimensional statistical features with a united form are constructed for these superpixels with different spatial shapes. Based on these two-dimensional statistical features, a convolutional neural network is utilized to learn deeper features and classify these depth features. Finally, an adaptive strategy is adopted to fuse the multiscale classification results. Experiments on three real hyperspectral datasets indicate the superiority of the proposed MSDFE method over several state-of-the-art methods.

Lithology classification stands as a pivotal research domain within geological Remote Sensing (RS). In recent years, extracting lithology information from multi-source RS data has become an inevitable trend. Various classification image primitives yield distinct outcomes in lithology classification. The current research on lithology classification utilizing RS data has predominantly concentrated on pixel-level classification, which suffers from a long classification time and high sensitivity to noise. In order to explore the application potential of superpixel segmentation in lithology classification, this study proposed the Multi-scale superpixel Segmentation Integrating Multi-source RS data (MSIMRS), and conducted a lithology classification study in Duolun County, Inner Mongolia Autonomous Region, China combining MSIMRS and the Support Vector Machine (MSIMRS-SVM). In addition, pixel-level K-Nearest Neighbor (KNN), Random Forest (RF) and SVM classification algorithms, as well as deep-learning models including Resnet50 (Res50), Efficientnet_B8 (Effi_B8), and Vision Transformer (ViT) were chosen for a comparative analysis. Among these methods, our proposed MSIMRS-SVM achieved the highest accuracy in lithology classification in a typical semi-arid area, Duolun County, with an overall accuracy and Kappa coefficient of 92.9% and 0.92. Moreover, the findings indicate that incorporating superpixel segmentation into lithology classification resulted in notably fewer fragmented patches and significantly improved the visualization effect. The results showcase the application potential of superpixel primitives in lithology information extraction within semi-arid areas.

ABSTRACT Hyperspectral image (HI) analysis approaches have recently become increasingly complex and sophisticated. Recently, the combination of spectral-spatial information and superpixel techniques have addressed some hyperspectral data issues, such as the higher spatial variability of spectral signatures and dimensionality of the data. However, most existing superpixel approaches do not account for specific HI characteristics resulting from its high spectral dimension. In this work, we propose a multiscale superpixel method that is computationally efficient for processing hyperspectral data. The Simple Linear Iterative Clustering (SLIC) oversegmentation algorithm, on which the technique is based, has been extended hierarchically. Using a novel robust homogeneity testing, the proposed hierarchical approach leads to superpixels of variable sizes but with higher spectral homogeneity when compared to the classical SLIC segmentation. For validation, the proposed homogeneity-based hierarchical method was applied as a preprocessing step in the spectral unmixing and classification tasks carried out using, respectively, the Multiscale sparse Unmixing Algorithm (MUA) and the CNN-Enhanced Graph Convolutional Network (CEGCN) methods. Simulation results with both synthetic and real data show that the technique is competitive with state-of-the-art solutions.

Semi-supervised graph convolutional networks (SSGCNs) have been proven to be effective in hyperspectral image classification (HSIC). However, limited training data and spectral uncertainty restrict the classification performance, and the computational demands of a graph convolution network (GCN) present challenges for real-time applications. To overcome these issues, a dual-branch fusion of a GCN and convolutional neural network (DFGCN) is proposed for HSIC tasks. The GCN branch uses an adaptive multi-scale superpixel segmentation method to build fusion adjacency matrices at various scales, which improves the graph convolution efficiency and node representations. Additionally, a spectral feature enhancement module (SFEM) enhances the transmission of crucial channel information between the two graph convolutions. Meanwhile, the CNN branch uses a convolutional network with an attention mechanism to focus on detailed features of local areas. By combining the multi-scale superpixel features from the GCN branch and the local pixel features from the CNN branch, this method leverages complementary features to fully learn rich spatial-spectral information. Our experimental results demonstrate that the proposed method outperforms existing advanced approaches in terms of classification efficiency and accuracy across three benchmark data sets.

SPMUNet: Semantic segmentation of citrus surface defects driven by superpixel feature

Superpixel-based multi-scale multi-instance learning for hyperspectral image classification

Polarimetric synthetic aperture radar (PolSAR) has attracted more attentions because of its excellent observation ability, and PolSAR image classification has become one of the significant tasks in remote sensing interpretation. Various types and sizes of land cover objects lead to misclassification, especially in the boundaries of different categories. To solve these issues, a multiscale superpixel-guided weighted graph convolutional network (MSGWGCN) is proposed for classifying PolSAR images. In the proposed MSGWGCN, multiscale superpixel features are imported into the weighted graph convolutional network to obtain higher-level representation, which can make full use of land cover object information in PolSAR images. Moreover, to fuse pixel-level features at different scales, a multiscale feature cascade fusion module is built, which plays an important role in preserving classification details. Experiments on three PolSAR datasets indicate that the proposed MSGWGCN performs better than other advanced methods on PolSAR classification task.

A superpixel is a group of pixels with similar low-level and mid-level properties, which can be seen as a basic unit in the pre-processing of remote sensing images. Therefore, superpixel segmentation can reduce the computation cost largely. However, all the deep-learning-based methods still suffer from the under-segmentation and low compactness problem of remote sensing images. To fix the problem, we propose EAGNet, an enhanced atrous extractor and self-dynamic gate network. The enhanced atrous extractor is used to extract the multi-scale superpixel feature with contextual information. The multi-scale superpixel feature with contextual information can solve the low compactness effectively. The self-dynamic gate network introduces the gating and dynamic mechanisms to inject detailed information, which solves the under-segmentation effectively. Massive experiments have shown that our EAGNet can achieve the state-of-the-art performance between k-means and deep-learning-based methods. Our methods achieved 97.61 in ASA and 18.85 in CO on the BSDS500. Furthermore, we also conduct the experiment on the remote sensing dataset to show the generalization of our EAGNet in remote sensing fields.

Pellet image segmentation model of superpixel feature-based support vector machine in digital twin

A spatially correlated fractional integral-based method for denoising geiger-mode avalanche photodiode light detection and ranging depth images

Although a wide variety of background subtraction methods has been proposed in recent years, none has been able to fully address multi-scale moving objects and dynamic background in real surveillance tasks. In this paper, a novel and effective background subtraction method, named regional multi-feature-frequency (RMFF), is proposed to detect multi-scale moving objects under dynamic background. Unlike many existing methods construct background model using simple multi-feature combinations, RMFF exploits the spatiotemporal cues of multi-feature as well as superpixels at each scale, thus allowing for more robust information to be exploited for background modeling. Specifically, the spatial relationship between pixels in a neighborhood and the frequencies of features over time are first exploited, enabling accurate detection of moving objects while ignoring most dynamic background changes. Then, the use of multi-scale superpixels for exploiting the structural information existing in real-world scenes further enhances robustness to multi-scale objects and environmental variations. Finally, an adaptive strategy is employed to dynamically adjust the foreground/background segmentation threshold for each region without user intervention. This adaptive threshold is defined for each region separately, and can adjust dynamically based on continuous monitoring of the background changes, thereby effectively reducing potential segmentation noise. Experiments on the 2014 version of the ChangeDetection.net dataset demonstrate that the proposed method outperforms the 12 state-of-the-art algorithms in terms of overall F-Measure and performs effectively in many complex scenes. Consequently, it is verified that the developed approach is feasible and useful for robust application in practical video surveillance.

Multi-scale Superpixel based Hierarchical Attention model for brain CT classification

The very high spatial resolution (VHR) remote sensing images have been an extremely valuable source for monitoring changes occurring on the Earth’s surface. However, precisely detecting relevant changes in VHR images still remains a challenge, due to the complexity of the relationships among ground objects. To address this limitation, a dual neighborhood hypergraph neural network is proposed in this article, which combines multiscale superpixel segmentation and hypergraph convolution to model and exploit the complex relationships. First, the bi-temporal image pairs are segmented under two scales and fed to a pre-trained U-net to obtain node features by treating each object under the fine scale as a node. The dual neighborhood is then defined using the father-child and adjacent relationships of the segmented objects to construct the hypergraph, which permits models to represent higher-order structured information far more complex than the conventional pairwise relationships. The hypergraph convolutions are conducted on the constructed hypergraph to propagate the label information from a small amount of labeled nodes to the other unlabeled ones by the node-edge-node transformation. Moreover, to alleviate the problem of imbalanced sampling, the focal loss function is adopted to train the hypergraph neural network. The experimental results on optical, SAR and heterogeneous optical/SAR data sets demonstrate that the proposed method offersbetter effectiveness and robustness compared to many state-of-the-art methods.

Prior studies using graph neural networks (GNNs) for image classification have focused on graphs generated from a regular grid of pixels or similar-sized superpixels. In the latter, a single target number of superpixels is defined for an entire dataset irrespective of differences across images and their intrinsic multiscale structure. On the contrary, this study investigates image classification using graphs generated from an image-specific number of multiscale superpixels. We propose WaveMesh, a new wavelet-based superpixeling algorithm, where the number and sizes of superpixels in an image are systematically computed based on its content. WaveMesh superpixel graphs are structurally different from similar-sized superpixel graphs. We use SplineCNN, a state-of-the-art network for image graph classification, to compare WaveMesh and similar-sized superpixels. Using SplineCNN, we perform extensive experiments on three benchmark datasets under three local-pooling settings: 1) no pooling, 2) GraclusPool, and 3) WavePool, a novel spatially heterogeneous pooling scheme tailored to WaveMesh superpixels. Our experiments demonstrate that SplineCNN learns from multiscale WaveMesh superpixels on-par with similar-sized superpixels. In all WaveMesh experiments, GraclusPool performs poorer than no pooling / WavePool, showing that poor cluster assignment negatively affects the performance of the network while learning from multiscale superpixels.

Abstract A backpack is a type of carried object (CO) widely used for various purposes because of its practicality. Various valuable items such as wallets, laptops, cameras, and cellphones may be kept in backpacks. Detecting backpacks in video surveillance is challenging due to their varying shapes, sizes, and colors. The process of localizing the area of the backpack in the image is a critical stage and dramatically influences the success of detection. This paper focuses on the process of localizing the backpack area through a multi-scale segmentation approach, where different scales are intended to detect the various size of the backpacks. Based on the assumption that the backpack is generally located above the bend line, the body-part method is then used to select superpixels. The selected superpixel feature is then extracted and used to train the model. Model testing is carried out in two scenarios. In the first scenario, the model is tested using the HOG (histogram of oriented gradients) feature, while in the second scenario, the model is tested using a combination of the HOG and histogram features. The experiment results show that on the DIKE20 dataset, the proposed model obtained an average F1 score of 69%. On PETS2006 and i-LIDS datasets, the proposed model shows an average F1 score of 68%, better than the average F1 score obtained by the state-of-the-art method.

Recently, the isolation forest (IF) methods have received increasing attention for their promising performance in hyperspectral anomaly detection (HAD). However, limited by the ability of exploiting spatial-spectral information, existing IF-based methods suffer from a lot of false alarms and disappointing performance of detecting local anomalies. To overcome the two problems, a multiscale superpixel guided discriminative forest method is proposed for HAD. First, the multiscale superpixel segmentation is employed to generate some homogeneous regions, and it can effectively extract spatial information to guide anomaly detection for the discriminative forest in local areas. Then, a novel discriminative forest (DF) model with the gain split criterion is designed, which enhances the sensitivity of the DF to local anomalies by the utilization of multi-dimension spectral bands for node division; meanwhile, the acceptable range of hyperplane attribute values is introduced to capture any unseen anomaly pixels that are out-of-range in the evaluation stage. Finally, for the high false alarm rate situation in the existing IF-based algorithms, the multiscale fusion with guided filtering is put forward to refine the initial detection results from the DF. In addition, the extensive experimental results on four real hyperspectral datasets demonstrate the effectiveness of the proposed method.

Superpixel segmentation for polarimetric synthetic aperture radar (PolSAR) images plays a key role in remote-sensing tasks, such as ship detection and land-cover classification. However, the existing methods cannot directly generate multi-scale superpixels in a hierarchical style and they will take a long time when multi-scale segmentation is executed separately. In this article, we propose an effective and accurate hierarchical superpixel segmentation method, by introducing a minimum spanning tree (MST) algorithm called the Boruvka algorithm. To accurately measure the difference between neighboring pixels, we obtain the scattering mechanism information derived from the model-based refined 5-component decomposition (RFCD) and construct a comprehensive dissimilarity measure. In addition, the edge strength map and homogeneity measurement are considered to make use of the structural and spatial distribution information in the PolSAR image. On this basis, we can generate superpixels using the distance metric along with the MST framework. The proposed method can maintain good segmentation accuracy at multiple scales, and it generates superpixels in real time. According to the experimental results on the ESAR and AIRSAR datasets, our method is faster than the current state-of-the-art algorithms and preserves somewhat more image details in different segmentation scales.