Superpixel Methods Research Articles

MAS-Net: Multi-Attention Hybrid Network for Superpixel Segmentation

Superpixels, as essential mid-level image representations, have been widely used in computer vision due to their computational efficiency and redundant compression. Compared with traditional superpixel methods, superpixel algorithms based on deep learning frameworks demonstrate significant advantages in segmentation accuracy. However, existing deep learning-based superpixel algorithms suffer from a loss of details due to convolution and upsampling operations in their encoder–decoder structure, which weakens their semantic detection capabilities. To overcome these limitations, we propose a novel superpixel segmentation network based on a multi-attention hybrid network (MAS-Net). MAS-Net is still based on an efficient symmetric encoder–decoder architecture. First, utilizing residual structure based on a parameter-free attention module at the feature encoding stage enhanced the capture of fine-grained features. Second, adoption of a global semantic fusion self-attention module was used at the feature selection stage to reconstruct the feature map. Finally, fusing the channel with the spatial attention mechanism at the feature-decoding stage was undertaken to obtain superpixel segmentation results with enhanced boundary adherence. Experimental results on real-world image datasets demonstrated that the proposed method achieved competitive results in terms of visual quality and metrics, such as ASA and BR-BP, compared with the state-of-the-art approaches.

Learning Invariant Inter-pixel Correlations for Superpixel Generation

Deep superpixel algorithms have made remarkable strides by substituting hand-crafted features with learnable ones. Nevertheless, we observe that existing deep superpixel methods, serving as mid-level representation operations, remain sensitive to the statistical properties (e.g., color distribution, high-level semantics) embedded within the training dataset. Consequently, learnable features exhibit constrained discriminative capability, resulting in unsatisfactory pixel grouping performance, particularly in untrainable application scenarios. To address this issue, we propose the Content Disentangle Superpixel (CDS) algorithm to selectively separate the invariant inter-pixel correlations and statistical properties, i.e., style noise. Specifically, We first construct auxiliary modalities that are homologous to the original RGB image but have substantial stylistic variations. Then, driven by mutual information, we propose the local-grid correlation alignment across modalities to reduce the distribution discrepancy of adaptively selected features and learn invariant inter-pixel correlations. Afterwards, we perform global-style mutual information minimization to enforce the separation of invariant content and train data styles. The experimental results on four benchmark datasets demonstrate the superiority of our approach to existing state-of-the-art methods, regarding boundary adherence, generalization, and efficiency. Code and pre-trained model are available at https://github.com/rookiie/CDSpixel.

Wavefront imaging of a biological sample using DMD-based single-pixel phase-shifting interferometric techniques: An experimental comparison

Single-pixel wavefront imaging (SWI) is an innovative technique for simultaneously measuring the amplitude and phase of an unknown field, which is particularly important for wavefront imaging in invisible wavelengths and under low light conditions. However, most of the existing SWI techniques are still in the technical development stage. Here, we provide a comprehensive analysis and comparison of the SWI methods using DMD-based single-pixel multi-step phase-shifting interferometric techniques. Through experimental demonstrations, we first compare the performances of the techniques using the Hadamard basis, Fourier basis, and discrete cosine transform (DCT) basis. Further, the performances of the Hadamard basis-based SWI (HSWI) technique using phase-shifting interferometry with the different reference strategies are investigated while the binary grating, Lee, and super-pixel methods are applied to control the DMD for realizing the necessary complex amplitude modulations there. Experimental results suggest the grating-phase-shifting-based HSWI performs best, in which one can use the checkerboard reference for the maximum field of view or select the peripheral reference for a better imaging resolution; and it can do 2 fps wavefront imaging of the sample at 10 % downsampling rate with a resolution of 128 × 128 pixels. Our work provides researchers with a guideline to choose a suitable SWI technique for their applications.

System for quantitative diagnosis of COVID-19 associated Pneumonia based on Superpixels with deep learning and chest CT

COVID-19 is a disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that can lead to complications such as acute respiratory distress syndrome, acute heart injury and secondary infections in a relatively high proportion of patients and, consequently, lead to significant mortality. The definitive diagnosis of COVID-19 is performed by real-time Polymerase Chain Reaction (RT-PCR). However, as RT-PCR results usually take longer to be completed than computed tomography (CT), the latter has had an important role in the detection of patients infected with COVID-19. A rough estimate of the extent of lung involvement by the disease is also important and considered an additional criterion for deciding on discharge or hospitalization. Recent research has adopted deep neural networks and other machine learning approaches to detect the presence of lung infection caused by COVID-19. However, the extent of lung involvement (volume) caused by the disease has been only superficially investigated. In this work, we created an end-to-end computer vision system to automatically quantify the Percentage of Infection (POI) in chest CT images of COVID-19 cases confirmed by laboratorial analysis. We evaluated the performance of three well-known deep neural networks, trained using three different strategies: (1) no transfer learning with randomly initialized weights; (2) transfer learning using with ImageNet weights without fine-tuning; and (3) transfer learning with weight fine-tuning. Data augmentation and dropout were used during training to reduce overfitting and increase the generalization capacity of the models. Our approach consists of segmenting a chest CT with the SLIC Superpixels method and classifying each segment (superpixel) into a specific class (COVID or non-COVID). We used the weights of the best deep neural network to classify the superpixels and quantify the COVID-19 infection by calculating the POI on chest CT. The results indicate that deep learning models can be successfully used to support radiologists in the quantitative diagnosis of lung infection caused by COVID-19, reaching an accuracy of up to 98.4% with Inception-Resnet-v2 architecture.

Inshore Ship Detection Based on Multi-Modality Saliency for Synthetic Aperture Radar Images

Synthetic aperture radar (SAR) ship detection is of significant importance in military and commercial applications. However, a high similarity in intensity and spatial distribution of scattering characteristics between the ship target and harbor facilities, along with a fuzzy sea-land boundary due to the strong speckle noise, result in a low detection accuracy and high false alarm rate for SAR ship detection with complex inshore scenes. In this paper, a new inshore ship detection method based on multi-modality saliency is proposed to overcome these challenges. Four saliency maps are established from different perspectives: an ocean-buffer saliency map (OBSM) outlining more accurate coastline under speckle noises; a local stability saliency map (LSSM) addressing pixel spatial distribution; a super-pixel saliency map (SPSM) extracting critical region-based features for inshore ship detection; and an intensity saliency map (ISM) to highlight target pixels with intensity distribution. By combining these saliency maps, ship targets in complex inshore scenes can be successfully detected. The method provides a novel interdisciplinary perspective (surface metrology) for SAR image segmentation, discovers the difference in spatial characteristics of SAR image elements, and proposes a novel robust CFAR procedure for background clutter fitting. Experiments on a public SAR ship detection dataset (SSDD) shows that our method achieves excellent detection performance, with a low false alarm rate, in offshore scenes, inshore scenes, inshore scenes with confusing metallic port facilities, and large-scale scenes. The results outperform several widely used methods, such as CFAR-based methods and super-pixel methods.

Double U-Net (W-Net): A change detection network with two heads for remote sensing imagery

Recently, the deep learning algorithms have been increasingly utilized in remote sensing change detection. However, incomplete buildings and the blurred edges caused by the complex scenes in change detection applications make the detection results fail to describe the real land cover changes. Superpixels can be used to alleviate edge blurring, but the existing superpixel methods cannot be trained jointly with the models in change detection. In this work, we investigated an innovative double-head method using deep learning, called double U-Net (W-Net), which consists of a superpixel module and a change detection module. Due to the superpixel module, W-Net can handle building edges very well. In order to solve problem that multiple subtasks fail to achieve the optimal results, a two-branch multi-task coupling framework of change detection and superpixels is designed for W-Net, which enables the model to achieve a globally optimal detection performance. The advancement of the W-Net was demonstrated using three public datasets. The F1score on LEVIR-CD dataset was 0.9031 and kappa coefficient was 0.8969. The F1-score on WHU building dataset was 0.9172 and kappa coefficient was 0.9142. The F1-score on SYSU-CD dataset was 0.8167and and kappa coefficient was 0.7724. The experiments confirmed that the W-Net is capable to detect the edges of changed area better and outperforms the other advanced change detection methods.

FMRI-Based Alzheimer's Disease Detection Using the SAS Method with Multi-Layer Perceptron Network.

In the present scenario, Alzheimer's Disease (AD) is one of the incurable neuro-degenerative disorders, which accounts for nearly 60% to 70% of dementia cases. Currently, several machine-learning approaches and neuroimaging modalities are utilized for diagnosing AD. Among the available neuroimaging modalities, functional Magnetic Resonance Imaging (fMRI) is extensively utilized for studying brain activities related to AD. However, analyzing complex brain structures in fMRI is a time-consuming and complex task; so, a novel automated model was proposed in this manuscript for early diagnosis of AD using fMRI images. Initially, the fMRI images are acquired from an online dataset: Alzheimer's Disease Neuroimaging Initiative (ADNI). Further, the quality of the acquired fMRI images was improved by implementing a normalization technique. Then, the Segmentation by Aggregating Superpixels (SAS) method was implemented for segmenting the brain regions (AD, Normal Controls (NC), Mild Cognitive Impairment (MCI), Early Mild Cognitive Impairment (EMCI), Late Mild Cognitive Impairment (LMCI), and Significant Memory Concern (SMC)) from the denoised fMRI images. From the segmented brain regions, feature vectors were extracted by employing Gabor and Gray Level Co-Occurrence Matrix (GLCM) techniques. The obtained feature vectors were dimensionally reduced by implementing Honey Badger Optimization Algorithm (HBOA) and fed to the Multi-Layer Perceptron (MLP) model for classifying the fMRI images as AD, NC, MCI, EMCI, LMCI, and SMC. The extensive investigation indicated that the presented model attained 99.44% of classification accuracy, 88.90% of Dice Similarity Coefficient (DSC), 90.82% of Jaccard Coefficient (JC), and 88.43% of Hausdorff Distance (HD). The attained results are better compared with the conventional segmentation and classification models.

Novel superpixel method to visualize fundus blood flow resistivity in healthy adults

We aimed to perform superpixel segmentation of ocular blood flow maps obtained using laser speckle flowgraphy (LSFG) and investigate the effects of systemic parameters such as body weight, height, and sex on ocular blood flow resistivity. We studied 757 healthy participants (583 men, 174 women). We calculated the average beat strength over mean blur rate (BOM) as a LSFG resistivity index, as a function of age and sex using ordinary regions of interest (ROI) centered on the optic nerve head (ONH), the retinal vessels region and tissue around the ONH, and the choroid (CHD). We compared the ROI and superpixel-based methods, which are segmented based on image processing, for calculating the BOM. The sex differences in the BOM for the ONH, retinal-vessels region and tissue region of the ONH and CHD were significant for individuals aged ≤ 50 years (P < 0.01) but not those > 50 years old (P > 0.05). The average BOMs calculated using the ROI and superpixel methods were strongly correlated in the ONH (coefficient = 0.87, R2 = 0.8, P < 0.0001, n = 5465). In summary, a superpixel-segmented BOM map is suitable for two-dimensional visualization of ocular blood flow resistivity.

Superpixel-Based PSO Algorithms for Color Image Quantization.

Nature-inspired artificial intelligence algorithms have been applied to color image quantization (CIQ) for some time. Among these algorithms, the particle swarm optimization algorithm (PSO-CIQ) and its numerous modifications are important in CIQ. In this article, the usefulness of such a modification, labeled IDE-PSO-CIQ and additionally using the idea of individual difference evolution based on the emotional states of particles, is tested. The superiority of this algorithm over the PSO-CIQ algorithm was demonstrated using a set of quality indices based on pixels, patches, and superpixels. Furthermore, both algorithms studied were applied to superpixel versions of quantized images, creating color palettes in much less time. A heuristic method was proposed to select the number of superpixels, depending on the size of the palette. The effectiveness of the proposed algorithms was experimentally verified on a set of benchmark color images. The results obtained from the computational experiments indicate a multiple reduction in computation time for the superpixel methods while maintaining the high quality of the output quantized images, slightly inferior to that obtained with the pixel methods.

A fire detection and localisation method based on keyframes and superpixels for large-space buildings

Relevant fire detection and localisation methods suffer from the problems of slow detection speeds, low detection accuracies and low localisation precisions when applied to large-space buildings. To address these problems, a fire detection and localisation method based on keyframes and superpixels is proposed herein. Initially, we improve the SuperPoint method to extract video keyframes; then, we detect and locate fires only on these extracted video keyframes, thus extensively improving the detection speed. Then, we perform fire detection and localisation via superpixel and multi-feature fusion methods applied to the extracted video keyframes, and the results are more accurate than those obtained with single features, thus realising precise localisation. The experimental results obtained using a publicly available fire dataset reveal that our method realises good keyframe extraction, fire detection and fire localisation performances and is thus suitable for detecting and localising fires under large-building surveillance.

Generation of structured beams with optional turbulence levels using a digital micromirror device

The dynamic spatial control of light fields, including spatial shaping and turbulence simulation, is essential for applications such as free-space optical communication. We propose a method for generating and rapidly switching structured beams with coherently superposed modes and optional turbulence levels using a digital micromirror device (DMD). The composite holograms encoded by Lee and Superpixel methods are quantitatively analyzed in terms of fidelity and efficiency, and both can generate structured beams under turbulence with high fidelity. The loaded turbulence intensity is verified by measuring the Strehl Ratio of generated beams, and the propagation characteristics is verified by reconstructing the three-dimensional spatial field through a stack of cross-sectional images. A more compact and robust experimental system is adopted, which reduces the power loss than traditional two consecutive modulations and improves the switching speed than phase-only modulators. The measurements consist with theoretical simulations, and the experimental method will promote applications of DMD in structured beam generation.

Efficient COVID-19 super pixel segmentation algorithm using MCFO-based SLIC.

In computer vision segmentation field, super pixel identity has become an important index in the recently segmentation algorithms especially in medical images. Simple Linear Iterative Clustering (SLIC) algorithm is one of the most popular super pixel methods as it has a great robustness, less sensitive to the image type and benefit to the boundary recall in different kinds of image processing. Recently, COVID-19 severity increased with the lack of an effective treatment or vaccine. As the Corona virus spreads in an unknown manner, th-ere is a strong need for segmenting the lungs infected regions for fast tracking and early detection, no matter how small. This may consider difficult to be achieved with traditional segmentation techniques. From this perspective, this paper presents an efficient modified central force optimization (MCFO)-based SLIC segmentation algorithm to discuss chest CT images for detecting the positive COVID-19 cases. The proposed MCFO-based SLIC segmentation algorithm performance is evaluated and compared with the thresholding segmentation algorithm using different evaluation metrics such as accuracy, boundary recall, F-measure, similarity index, MCC, Dice, and Jaccard. The outcomes demonstrated that the proposed MCFO-based SLIC segmentation algorithm has achieved better detection for the small infected regions in CT lung scans than the thresholding segmentation.

An enhanced image quality assessment by synergizing superpixels and visual saliency

Superpixel and saliency-based evaluation methods play important roles in full reference image quality assessment (FR IQA). However, we find that these methods have one complementary principle and three limitations: (1) the weighted maps of superpixel-based methods conflict with the perception of the human visual system; (2) saliency-based methods are inefficient in terms of the block distortion; (3) the general two-direction gradient extraction factor must be extended to be multidirectional. To address these limitations, we propose an enhanced image quality assessment by synergizing superpixels and visual saliency. Specifically, the calculation of a newly proposed framework involves three similarities and two strategies: the saliency, superpixel and multidirectional gradient similarities of the neighborhoods, and the saliency pooling strategy, the fusion strategy of these similarities. Theoretical analysis and experimental results show that the proposed method can effectively address the limitations noted above and outperform the existing methods.

CMSuG: Competitive mechanism-based superpixel generation method for image segmentation

During the last years, object-based image segmentation (OBIA) has seen a considerable increase in the image segmentation. OBIA is generally based on superpixel methods, in which the clustering-based method plays an increasingly important role. Most clustering methods for generating superpixels suffer from inaccurate classification points with inappropriate cluster centers. To solve the problem, we propose a competitive mechanism-based superpixel generation (CMSuG) method, which both accelerates convergence and promotes robustness for noise sensitivity. Then, image segmentation results will be obtained by a region adjacent graph (RAG)-based merging algorithm after constructing an RAG. However, high segmentation accuracy is customarily accompanied by expensive time-consuming costs. To improve computational efficiency, we address a parallel CMSuG algorithm, the time of which is much less than the CMSuG method. In addition, we present a parallel RAG method to decrease the expensive time-consuming cost in serial RAG construction. By leveraging parallel techniques, the running time of the whole image segmentation method decline with the time complexity from O (N) + O (K2) to O (N/K) or O (K2), in which N is the size of an input image and K is the given number of the superpixel. In the experiments, both nature image and remote sensing image segmentation results demonstrate that our CMSuG method outperforms the state-of-the-art superpixel generation methods, and then performs well for image segmentation in turn. Compared with the serial segmentation method, our parallel techniques gain more than four times acceleration in both remote sensing image dataset and nature image dataset.

Direct binary search method for high-resolution holographic image projection.

Complex-amplitude modulation of light fields with a digital micromirror device (DMD) has been widely used in holographic image projection. DMD is a binary-amplitude modulator, and its use for complex field modulation in a 4f configuration requires low-pass filtering. However, the reconstructed fields suffer from low resolution due to the limited bandwidth for the existing methods such as the Lee and superpixel methods. Here, we report a direct binary search (DBS) method to design high-resolution complex-amplitude holograms. The method is able to increase the spatial bandwidth up to twice that of the superpixel method. Numerical simulations and experiments are presented to demonstrate the method, which show that the errors are reduced by about 60% and 40% respectively for the test fields compared to the superpixel method. Furthermore, the measured efficiency of laser light can be improved by a maximum of 60%.

Ship detection based on a superpixel-level CFAR detector for SAR imagery

ABSTRACT Ship detection for SAR imagery plays a crucial rule in the field of remote-sensing image processing. Superpixel methods have attracted enormous interest in the recent years, resulting in superpixel-based CFAR (Constant False Alarm Rate) ship detection methods for SAR images become a hot research issue. However, existing methods take superpixel generation as a preprocessing, and CFAR detection is essentially carried out at the pixel level, and the influence of speckle noise has not been fundamentally overcome. This paper innovatively proposed a real superpixel-level CFAR detection method, which takes superpixels instead of pixels as the minimum processing primitives on the basis of reliable superpixel segmentation. Firstly, in order to obtain the background superpixel region of each under test superpixel more accurately and efficiently, we designed a ring topology, which was specially developed for non-Euclidean structure data like superpixels. Secondly, the Johnsonsu function is used to fit the superpixel clutters from the background ring, and the detection threshold was obtained by a given false alarm rate. Finally, the candidate targets were further optimized to carry out accurate detection results. Experimental results based on multiple sets of real SAR images shown that our method has significant advantages in terms of detection rate and false alarm rate compared to the traditional K-CFAR and the existing superpixel-based CFAR methods.

Detection of ring cell cancer in histopathological images with region of interest determined by SLIC superpixels method

Detection of ring cell cancer in histopathological images with region of interest determined by SLIC superpixels method

Spectral-Spatial Classification of Hyperspectral Images Using BERT-Based Methods With HyperSLIC Segment Embeddings

The classification performance is highly affected because hyperspectral images include many bands, have high dimensions, and have few labeled training samples. This challenge is reduced by using rich spatial information and an effective classifier. The classifiers in this study are BERT-based (Bidirectional Encoder Representations from Transformers) models, which have recently been applied in natural language processing. The BERT model and its performance-improved version, the ALBERT (A Lite BERT) model, are utilized as transformer-based models. Because of their structure, these models can also accept spatial information via &#x2018;segment embeddings&#x2019;. Segmentation algorithms are commonly used in the literature to get spatial information. Superpixel methods have shown superior results in the segmentation literature due to the utility of working at the superpixel level rather than the conventional pixel level. HyperSLIC, a modified version of the SLIC superpixel method for hyperspectral images, is employed as input to BERT-based models in this study. In addition, HyperSLIC segmentation results are merged with the DBSCAN algorithm for similar superpixels to increase the size of spatially similar areas and called as HyperSLIC-DBSCAN. The effects of segment embedding information on classification accuracy in BERT-based models is studied experimentally. Experimental results show that BERT-based models outperform conventional and deep learning-based 1D/2D convolutional neural network classifiers when spatial information is used with the help of segment embedding information.

SMBFT: A Modified Fuzzy c-Means Algorithm for Superpixel Generation.

Most traditional superpixel segmentation methods used binary logic to generate superpixels for natural images. When these methods are used for images with significantly fuzzy characteristics, the boundary pixels sometimes cannot be correctly classified. In order to solve this problem, this paper proposes a Superpixel Method Based on Fuzzy Theory (SMBFT), which uses fuzzy theory as a guide and traditional fuzzy c-means clustering algorithm as a baseline. This method can make full use of the advantage of the fuzzy clustering algorithm in dealing with the images with the fuzzy characteristics. Boundary pixels which have higher uncertainties can be correctly classified with maximum probability. The superpixel has homogeneous pixels. Meanwhile, the paper also uses the surrounding neighborhood pixels to constrain the spatial information, which effectively alleviates the negative effects of noise. The paper tests on the images from Berkeley database and brain MR images from the Brain web. In addition, this paper proposes a comprehensive criterion to measure the weights of two kinds of criterions in choosing superpixel methods for color images. An evaluation criterion for medical image data sets employs the internal entropy of superpixels which is inspired by the concept of entropy in the information theory. The experimental results show that this method has superiorities than traditional methods both on natural images and medical images.

Cluster-based fine-to-coarse superpixel segmentation

As an image preprocessing technology, superpixel segmentation has become an important tool in the field of computer vision. How to obtain a more accurate, faster, and easier-to-apply superpixel segmentation algorithm is a problem faced by researchers. In this paper, a cluster-based fine-to-coarse superpixel segmentation (FCSS) algorithm is proposed. By introducing color thresholds and depth thresholds with practical physical meanings as algorithm parameters, high-quality segmentation with fewer superpixels is achieved. It not only reduces the complexity of the upper application, but also provides an easy to understand interface. Superpixel segmentation methods often cannot achieve high-quality segmentation through a set of parameters. Experimental results show that FCSS can achieve finer segmentation by setting different parameters, and the segmentation results are superior to other algorithms. When the number of superpixels is 100, the segmentation performance of FCSS is better than that of existing state-of-the-art methods.