Superpixel Algorithm Research Articles

An Object-Aware Network Embedding Deep Superpixel for Semantic Segmentation of Remote Sensing Images

Semantic segmentation forms the foundation for understanding very high resolution (VHR) remote sensing images, with extensive demand and practical application value. The convolutional neural networks (CNNs), known for their prowess in hierarchical feature representation, have dominated the field of semantic image segmentation. Recently, hierarchical vision transformers such as Swin have also shown excellent performance for semantic segmentation tasks. However, the hierarchical structure enlarges the receptive field to accumulate features and inevitably leads to the blurring of object boundaries. We introduce a novel object-aware network, Embedding deep SuperPixel, for VHR image semantic segmentation called ESPNet, which integrates advanced ConvNeXt and the learnable superpixel algorithm. Specifically, the developed task-oriented superpixel generation module can refine the results of the semantic segmentation branch by preserving object boundaries. This study reveals the capability of utilizing deep convolutional neural networks to accomplish both superpixel generation and semantic segmentation of VHR images within an integrated end-to-end framework. The proposed method achieved mIoU scores of 84.32, 90.13, and 55.73 on the Vaihingen, Potsdam, and LoveDA datasets, respectively. These results indicate that our model surpasses the current advanced methods, thus demonstrating the effectiveness of the proposed scheme.

Color detection of printing based on improved superpixel segmentation algorithm

We propose an improved superpixel segmentation algorithm based on visual saliency and color entropy for online color detection in printed products. This method addresses the issues of low accuracy and slow speed in detecting color deviations in print quality control. The improved superpixel segmentation algorithm consists of three main steps: Firstly, simulating human visual perception to obtain visually salient regions of the image, thereby achieving region-based superpixel segmentation. Secondly, adaptively determining the superpixel size within the salient regions using color information entropy. Finally, the superpixel segmentation method is optimized using hue angle distance based on chromaticity, ultimately achieving a region-based adaptive superpixel segmentation algorithm. Color detection of printed products compares the color mean values of post-printing images under the same superpixel labels, outputting labels with color deviations to identify areas of color differences. The experimental results show that the improved superpixel algorithm introduces color phase distance with better segmentation accuracy, and combines it with human visual perception to better reproduce the color information of printed materials. Using the method described in this article for printing color quality inspection can reduce data computation, quickly detect and mark color difference areas, and provide the degree of color deviation.

SMALE: Hyperspectral Image Classification via Superpixels and Manifold Learning

As an extremely efficient preprocessing tool, superpixels have become more and more popular in various computer vision tasks. Nevertheless, there are still several drawbacks in the application of hyperspectral image (HSl) processing. Firstly, it is difficult to directly apply superpixels because of the high dimension of HSl information. Secondly, existing superpixel algorithms cannot accurately classify the HSl objects due to multi-scale feature categorization. For the processing of high-dimensional problems, we use the principle of PCA to extract three principal components from numerous bands to form three-channel images. In this paper, a novel superpixel algorithm called Seed Extend by Entropy Density (SEED) is proposed to alleviate the seed point redundancy caused by the diversified content of HSl. It also focuses on breaking the dilemma of manually setting the number of superpixels to overcome the difficulty of classification imprecision caused by multi-scale targets. Next, a space–spectrum constraint model, termed Hyperspectral Image Classification via superpixels and manifold learning (SMALE), is designed, which integrates the proposed SEED to generate a dimensionality reduction framework. By making full use of spatial context information in the process of unsupervised dimension reduction, it could effectively improve the performance of HSl classification. Experimental results show that the proposed SEED could effectively promote the classification accuracy of HSI. Meanwhile, the integrated SMALE model outperforms existing algorithms on public datasets in terms of several quantitative metrics.

MAS-Net: Multi-Attention Hybrid Network for Superpixel Segmentation

Superpixels, as essential mid-level image representations, have been widely used in computer vision due to their computational efficiency and redundant compression. Compared with traditional superpixel methods, superpixel algorithms based on deep learning frameworks demonstrate significant advantages in segmentation accuracy. However, existing deep learning-based superpixel algorithms suffer from a loss of details due to convolution and upsampling operations in their encoder–decoder structure, which weakens their semantic detection capabilities. To overcome these limitations, we propose a novel superpixel segmentation network based on a multi-attention hybrid network (MAS-Net). MAS-Net is still based on an efficient symmetric encoder–decoder architecture. First, utilizing residual structure based on a parameter-free attention module at the feature encoding stage enhanced the capture of fine-grained features. Second, adoption of a global semantic fusion self-attention module was used at the feature selection stage to reconstruct the feature map. Finally, fusing the channel with the spatial attention mechanism at the feature-decoding stage was undertaken to obtain superpixel segmentation results with enhanced boundary adherence. Experimental results on real-world image datasets demonstrated that the proposed method achieved competitive results in terms of visual quality and metrics, such as ASA and BR-BP, compared with the state-of-the-art approaches.

Spherical Superpixel Segmentation with Context Identity and Contour Intensity

Superpixel segmentation is a popular preprocessing tool in the field of image processing. Nevertheless, conventional planar superpixel generation algorithms are inadequately suited for segmenting symmetrical spherical images due to the distinctive geometric differences. In this paper, we present a novel superpixel algorithm termed context identity and contour intensity (CICI) that is specifically tailored for spherical scene segmentation. By defining a neighborhood range and regional context identity, we propose a symmetrical spherical seed-sampling method to optimize both the quantity and distribution of seeds, achieving evenly distributed seeds across the panoramic surface. Additionally, we integrate the contour prior to superpixel correlation measurements, which could significantly enhance boundary adherence across different scales. By implementing the two-fold optimizations on the non-iterative clustering framework, we achieve synergistic CICI to generate higher-quality superpixels. Extensive experiments on the public dataset confirm that our work outperforms the baselines and achieves comparable results with state-of-the-art superpixel algorithms in terms of several quantitative metrics.

Learning Invariant Inter-pixel Correlations for Superpixel Generation

Deep superpixel algorithms have made remarkable strides by substituting hand-crafted features with learnable ones. Nevertheless, we observe that existing deep superpixel methods, serving as mid-level representation operations, remain sensitive to the statistical properties (e.g., color distribution, high-level semantics) embedded within the training dataset. Consequently, learnable features exhibit constrained discriminative capability, resulting in unsatisfactory pixel grouping performance, particularly in untrainable application scenarios. To address this issue, we propose the Content Disentangle Superpixel (CDS) algorithm to selectively separate the invariant inter-pixel correlations and statistical properties, i.e., style noise. Specifically, We first construct auxiliary modalities that are homologous to the original RGB image but have substantial stylistic variations. Then, driven by mutual information, we propose the local-grid correlation alignment across modalities to reduce the distribution discrepancy of adaptively selected features and learn invariant inter-pixel correlations. Afterwards, we perform global-style mutual information minimization to enforce the separation of invariant content and train data styles. The experimental results on four benchmark datasets demonstrate the superiority of our approach to existing state-of-the-art methods, regarding boundary adherence, generalization, and efficiency. Code and pre-trained model are available at https://github.com/rookiie/CDSpixel.

Brain MR Image Classification Using Superpixel-Based Deep Transfer Learning.

Nowadays, brain MR (Magnetic Resonance) images are widely used by clinicians to examine the brain's anatomy to look into various pathological conditions like cerebrovascular incidents and neuro-degenerative diseases. Generally, these diseases can be identified with the MR images as "normal" and "abnormal" brains in a two-class classification problem or as disease-specific classes in a multi-class problem. This article presents an ensemble transfer learning-inspired deep architecture that uses the simple linear iterative clustering (SLIC)-based superpixel algorithm along with convolutional neural network (CNN) to classify the MR images as normal or abnormal. Superpixel algorithm segments the input MR images into clusters of regions defined by similarity measures using perceptual feature space. These superpixel images are beneficial as they can provide a compact and meaningful role in computationally demanding applications. The superpixel images are then fed to the deep convolutional neural network (CNN) to classify the images. Three brain MR image datasets, NITR-DHH, DS-75, and DS-160, are used to conduct the experimentation. Through the use of deep transfer learning, the model achieves performance accuracy of 88.15% (NITR-DHH), 98.15% (DS-160), and 98.33% (DS-75) even with the small-scale medical image dataset. The experimentally obtained results demonstrate that the proposed method is promising and efficient for clinical applications for diagnosing different brain diseases via MR images.

Fast Automatic Fuzzy C-Means Knitting Pattern Color-Separation Algorithm Based on Superpixels.

Patterns entered into knitting CAD have thousands or tens of thousands of different colors, which need to be merged by color-separation algorithms. However, for degraded patterns, the current color-separation algorithms cannot achieve the desired results, and the clustering quantity parameter needs to be managed manually. In this paper, we propose a fast and automatic FCM color-separation algorithm based on superpixels, which first uses the Real-ESRGAN blind super-resolution network to clarify the degraded patterns and obtain high-resolution images with clear boundaries. Then, it uses the improved MMGR-WT superpixel algorithm to pre-separate the high-resolution images and obtain superpixel images with smooth and accurate edges. Subsequently, the number of superpixel clusters is automatically calculated by the improved density peak clustering (DPC) algorithm. Finally, the superpixels are clustered using fast fuzzy c-means (FCM) based on a color histogram. The experimental results show that not only is the algorithm able to automatically determine the number of colors in the pattern and achieve the accurate color separation of degraded patterns, but it also has lower running time. The color-separation results for 30 degraded patterns show that the segmentation accuracy of the color-separation algorithm proposed in this paper reaches 95.78%.

ROI adaptive lossy compression in dermatology using deep learning and superpixels algorithm

ROI adaptive lossy compression in dermatology using deep learning and superpixels algorithm

A deep-learning approach for dynamic region merging applied to feature extraction from borehole microresistivity images

The primary purpose of processing borehole resistivity images is to identify and extract high (or low) resistivity anomalous areas, which are associated with resistive fractures and dissolved pores. To improve the accuracy and applicability of these models, a new intelligent method combining dynamic-region merging (DRM) with a deep-learning network (U-net) is proposed. The superpixel method, also referred to as linear spectral clustering (LSC), was applied to segment fractures and dissolved pores that are represented by a resistivity contrast in the original resistivity images. The sequential probability ratio test technique was then used to implement the DRM procedure to group many oversegmented small regions. A convolutional neural network (U-net architecture) model was used to automatically identify geologic features with various scales. The trained neural network was then used to identify the segmented resistivity images that had been processed by the DRM. The results showed that the superpixel algorithm and U-net combination significantly improved the accuracy of the classification and identification of fractures and dissolved pores in different test data sets. Moreover, it solved the problem of few-shot learning, enabling a pretrained model to generalize over new data categories.

Orchard classification based on super-pixels and deep learning with sparse optical images

Reliable and accurate classification of orchards is important for the dynamic monitoring of large-scale orchards and food security evaluation. At present, the very similar spectral profiles of different fruit trees and the high susceptibility of optical data to interference by weather conditions limit the resolution of orchard classification. Synthetic Aperture Radar (SAR) imagery provides an advanced solution to this problem, with the advantages of being immune to weather conditions and advances in deep learning techniques. This paper presents an orchard classification model (STCM) with optical and SAR fusion, which integrates the advantages of the Simple Non-Iterative Clustering (SNIC) super-pixel algorithm with a deep learning algorithm based on a multi-headed attention mechanism, and it achieves the best classification accuracy of above 0.82 in comparison with existing deep learning models. Meanwhile, the model has exceptional classification accuracy and robustness in a study area with fragmented plots, many fruit tree species, and various distributions of optical images. In the absence of optical data, the classification accuracy of the STCM model using only SAR data is around 0.70, which makes the model have a promising potential application value. This study provides a technical solution for accurately obtaining different orchard categories in high-resolution remotely sensed orchard images. The study helps to improve the management and operation of orchards and provides a strong basis for decision-making in the fruit industry to enhance the sustainability of the global fruit industry.

Deep Hyperspherical Clustering for Skin Lesion Medical Image Segmentation.

Diagnosis of skin lesions based on imaging techniques remains a challenging task because data (knowledge) uncertainty may reduce accuracy and lead to imprecise results. This paper investigates a new deep hyperspherical clustering (DHC) method for skin lesion medical image segmentation by combining deep convolutional neural networks and the theory of belief functions (TBF). The proposed DHC aims to eliminate the dependence on labeled data, improve segmentation performance, and characterize the imprecision caused by data (knowledge) uncertainty. First, the SLIC superpixel algorithm is employed to group the image into multiple meaningful superpixels, aiming to maximize the use of context without destroying the boundary information. Second, an autoencoder network is designed to transform the superpixels' information into potential features. Third, a hypersphere loss is developed to train the autoencoder network. The loss is defined to map the input to a pair of hyperspheres so that the network can perceive tiny differences. Finally, the result is redistributed to characterize the imprecision caused by data (knowledge) uncertainty based on the TBF. The proposed DHC method can well characterize the imprecision between skin lesions and non-lesions, which is particularly important for the medical procedures. A series of experiments on four dermoscopic benchmark datasets demonstrate that the proposed DHC yields better segmentation performance, increasing the accuracy of the predictions while can perceive imprecise regions compared to other typical methods.

Tree Species Classification in a Complex Brazilian Tropical Forest Using Hyperspectral and LiDAR Data

This study experiments with different combinations of UAV hyperspectral data and LiDAR metrics for classifying eight tree species found in a Brazilian Atlantic Forest remnant, the most degraded Brazilian biome with high fragmentation but with huge structural complexity. The selection of the species was done based on the number of tree samples, which exist in the plot data and in the fact the UAV imagery does not acquire information below the forest canopy. Due to the complexity of the forest, only species that exist in the upper canopy of the remnant were included in the classification. A combination of hyperspectral UAV images and LiDAR point clouds were in the experiment. The hyperspectral images were photogrammetric and radiometric processed to obtain orthomosaics with reflectance factor values. Raw spectra were extracted from the trees, and vegetation indices (VIs) were calculated. Regarding the LiDAR data, both the point cloud—referred to as Peak Returns (PR)—and the full-waveform (FWF) LiDAR were included in this study. The point clouds were processed to normalize the intensities and heights, and different metrics for each data type (PR and FWF) were extracted. Segmentation was preformed semi-automatically using the superpixel algorithm, followed with manual correction to ensure precise tree crown delineation before tree species classification. Thirteen different classification scenarios were tested. The scenarios included spectral features and LiDAR metrics either combined or not. The best result was obtained with all features transformed with principal component analysis with an accuracy of 76%, which did not differ significantly from the scenarios using the raw spectra or VIs with PR or FWF LiDAR metrics. The combination of spectral data with geometric information from LiDAR improved the classification of tree species in a complex tropical forest, and these results can serve to inform management and conservation practices of these forest remnants.

Multilevel Multiobjective Particle Swarm Optimization Guided Superpixel Algorithm for Histopathology Image Detection and Segmentation.

Histopathology image analysis is considered as a gold standard for the early diagnosis of serious diseases such as cancer. The advancements in the field of computer-aided diagnosis (CAD) have led to the development of several algorithms for accurately segmenting histopathology images. However, the application of swarm intelligence for segmenting histopathology images is less explored. In this study, we introduce a Multilevel Multiobjective Particle Swarm Optimization guided Superpixel algorithm (MMPSO-S) for the effective detection and segmentation of various regions of interest (ROIs) from Hematoxylin and Eosin (H&E)-stained histopathology images. Several experiments are conducted on four different datasets such as TNBC, MoNuSeg, MoNuSAC, and LD to ascertain the performance of the proposed algorithm. For the TNBC dataset, the algorithm achieves a Jaccard coefficient of 0.49, a Dice coefficient of 0.65, and an F-measure of 0.65. For the MoNuSeg dataset, the algorithm achieves a Jaccard coefficient of 0.56, a Dice coefficient of 0.72, and an F-measure of 0.72. Finally, for the LD dataset, the algorithm achieves a precision of 0.96, a recall of 0.99, and an F-measure of 0.98. The comparative results demonstrate the superiority of the proposed method over the simple Particle Swarm Optimization (PSO) algorithm, its variants (Darwinian particle swarm optimization (DPSO), fractional order Darwinian particle swarm optimization (FODPSO)), Multiobjective Evolutionary Algorithm based on Decomposition (MOEA/D), non-dominated sorting genetic algorithm 2 (NSGA2), and other state-of-the-art traditional image processing methods.

Superpixel-Based Grain Segmentation in Sandstone Thin-Section

Mineral segmentation is an equally important and difficult task in the quantification of mineral composition. Difficulties come from the process of determining boundaries of distinctive mineral grains necessary for further analysis and mineral identification. Done by hand, the task is very time-consuming and higher accuracies are burdened with the possible human fatigue factor. The presented method is a fully automated solution to the problem that uses a superpixel approach and feature-based merging. The method is validated by comparison with the manual approach. Analyzed data consist of photos taken by a Nikon Eclipse LV100N POL polarizing microscope at 200× magnification, in transmitted light, with crossed polarizers. Images are first prepared by Gaussian filter and meanshift operations, then the initial segmentation is provided by the superpixel algorithm. Oversegmentation is resolved by feature-based merging. The last step consists of counting the individual grain boundaries and preparing the results as easily readable visual data.

H. pylori Related Atrophic Gastritis Detection Using Enhanced Convolution Neural Network (CNN) Learner.

Atrophic gastritis (AG) is commonly caused by the infection of the Helicobacter pylori (H. pylori) bacteria. If untreated, AG may develop into a chronic condition leading to gastric cancer, which is deemed to be the third primary cause of cancer-related deaths worldwide. Precursory detection of AG is crucial to avoid such cases. This work focuses on H. pylori-associated infection located at the gastric antrum, where the classification is of binary classes of normal versus atrophic gastritis. Existing work developed the Deep Convolution Neural Network (DCNN) of GoogLeNet with 22 layers of the pre-trained model. Another study employed GoogLeNet based on the Inception Module, fast and robust fuzzy C-means (FRFCM), and simple linear iterative clustering (SLIC) superpixel algorithms to identify gastric disease. GoogLeNet with Caffe framework and ResNet-50 are machine learners that detect H. pylori infection. Nonetheless, the accuracy may become abundant as the network depth increases. An upgrade to the current standards method is highly anticipated to avoid untreated and inaccurate diagnoses that may lead to chronic AG. The proposed work incorporates improved techniques revolving within DCNN with pooling as pre-trained models and channel shuffle to assist streams of information across feature channels to ease the training of networks for deeper CNN. In addition, Canonical Correlation Analysis (CCA) feature fusion method and ReliefF feature selection approaches are intended to revamp the combined techniques. CCA models the relationship between the two data sets of significant features generated by pre-trained ShuffleNet. ReliefF reduces and selects essential features from CCA and is classified using the Generalized Additive Model (GAM). It is believed the extended work is justified with a 98.2% testing accuracy reading, thus providing an accurate diagnosis of normal versus atrophic gastritis.

An Efficient Hybrid Linear Clustering Superpixel Decomposition Framework for Traffic Scene Semantic Segmentation

Superpixel decomposition could reconstruct an image through meaningful fragments to extract regional features, thus boosting the performance of advanced computer vision tasks. To further optimize the computational efficiency as well as segmentation quality, a novel framework is proposed to generate superpixels from the perspective of hybridizing two existing linear clustering frameworks. Instead of conventional grid sampling seeds for region clustering, a fast convergence strategy is first introduced to center the final superpixel clusters, which is based on an accelerated convergence strategy. Superpixels are then generated from a center-fixed online average clustering, which adopts region growing to label all pixels in an efficient one-pass manner. The experiments verify that the integration of this two-step implementation could generate a synergistic effect and that it becomes more well-rounded than each single method. Compared with other state-of-the-art superpixel algorithms, the proposed framework achieves a comparable overall performance in terms of segmentation accuracy, spatial compactness and running efficiency; moreover, an application on image segmentation verifies its facilitation for traffic scene analysis.

UAV-based weed detection in Chinese cabbage using deep learning

Weeds are unwanted plants on agricultural soil. They always competing for sunlight, nutrient, space and water with economic crops. Uncontrolled weed growth can cause both significant economic and ecological loss. Hence, weeds should be efficiently differentiated from the crops for the smart spraying solution. In this study, the Convolutional Neural Network (CNN) was used to perform weed detection amongst the commercial crop of Chinese cabbage, using the acquired images by Unmanned Aerial Vehicles. The acquired images were pre-processed and subsequently segmented into the crop, soil, and weed classes using the Simple Linear Iterative Clustering Superpixel algorithm. The segmented images were then used to construct the CNN-based classifier. The Random Forest (RF) was applied to compare with the performance of CNN. The results showed that the CNN achieved a higher overall accuracy of 92.41% than the 86.18% attained by RF.

An artificial intelligence method for vessel detection in images of cardiac catheterization

Coronary artery disease (CAD) is one of the top ten leading causes of death in countries worldwide. There are several methods used by physicians to diagnose CAD. One common method is the use of X-ray angiogram (XRA) images. When employing a computer-aided automatic analysis method to locate blood vessels in XRA images, problems such as complex background noise, low contrast, and difficulty in separating blood vessels from the background are encountered, especially for blood vessels with a low contrast to the background noise. In this study, we used various image preprocessing methods to highlight blood vessels from complex backgrounds and different types of blood vessel, followed by the employment of four superpixel segmentation methods to partition an image into multiple segments, and finally the utilization of Weighted Voting to determine the best superpixel algorithm. A genetic algorithm was used to identify the best weights by which to determine the locations of blood vessels in images of cardiac catheterization. Our proposed method achieved an average F1-score of 73.4% in the analysis of five patients, which was better than the other superpixel algorithms tested. The results of this research will be helpful for future SYNTAX score analysis.

Multiple object tracking with appearance feature prediction and similarity fusion

Object tracking is a crucial research area within the field of intelligent transportation, providing a vital foundation for anomalous behavior analysis and traffic statistics. Although pedestrian detectors have shown impressive results, leading to the advancement of detection-based tracking methods, target association in complex scenarios remains a difficult and less efficient task due to the lack of feature robustness in the presence of partial occlusions. In the proposed tracking method, we extract convolutional features on each entire object and its local blocks, segmented by the superpixel algorithm. Aiming to emphasize the global and local information respectively, the global features for each entire object are extracted from the last layer of the backbone network, while local features are derived from a specific intermediate layer of the backbone network. The association between tracked targets and detected pedestrian candidates relies on fused similarity degrees. Furthermore, we use the transformer’s self-attention mechanism to predict features for the current frame based on the information within past frames, aiming to eliminate the effects of target appearance variations. Additionally, we remove redundant background pixels in the detected rectangles of pedestrian candidates by using a background modeling algorithm. Experimental results demonstrate that the tracker proposed in this paper outperforms other trackers across five publicly available datasets, indicating its effectiveness and potential for further development.