Image Superpixel Research Articles

USNet: underwater image superpixel segmentation via multi-scale water-net

Underwater images commonly suffer from a variety of quality degradations, such as color casts, low contrast, blurring details, and limited visibility. Existing superpixel segmentation algorithms face challenges in achieving superior performance when directly applied to underwater images with quality degradation. In this paper, to alleviate the limitations of superpixel segmentation when applied to underwater scenes, we propose the first underwater superpixel segmentation network (USNet), specifically designed according to the intrinsic characteristics of underwater images. Considering the quality degradation, we propose a multi-scale water-net module (MWM) aimed at enhancing the quality of underwater images before superpixel segmentation. The degradation-aware attention (DA) mechanism is then created and incorporated into MWM to solve light scattering and absorption, which can decrease object visibility and cause blurred edges. By effectively directing the network to prioritize locations that exhibit a considerable decrease in quality, this method enhances the visibility of those specific areas. Additionally, we extract the deep spatial features using the coordinate attention method. Finally, these features are fused with the shallow spatial information using the dynamic spatiality embedding module to embed comprehensive spatial features. Training and testing were conducted on the SUIM dataset, the underwater change detection dataset, and UIEB dataset. Experimental results show that our method achieves the best scores in terms of achievable segmentation accuracy, undersegmentation error, and boundary recall evaluation metrics compared to other methods. Both quantitative and qualitative evaluations demonstrate that our method can handle complicated underwater scenes and outperform existing state-of-the-art segmentation methods.

LCAMix: Local-and-contour aware grid mixing based data augmentation for medical image segmentation

Medical image segmentation often faces challenges related to overfitting, primarily due to the limited and complex training samples. This challenge often prompts the use of self-supervised learning and data augmentation. However, self-supervised learning requires well-defined hand-crafted tasks and multiple training stages. On the other hand, basic image augmentation techniques like cropping, rotation, and flipping, effective for natural scene images, have limited efficacy for medical images due to their isotropic nature.While regional dropout regularization data augmentation methods have proven effective in image recognition tasks, their application in image segmentation is not as extensively studied. Additionally, existing augmentation methods often operate on square regions, leading to the loss of crucial contour information. This is particularly problematic for medical image segmentation tasks dealing with regions of interest characterized by intricate shapes. In this work, we introduce LCAMix, a novel data augmentation approach designed for medical image segmentation. LCAMix operates by blending two images and their segmentation masks based on their superpixels, incorporating a local-and-contour-aware strategy. The training process on augmented images adopts two auxiliary pretext tasks: firstly, classifying local superpixels in augmented images using an adaptive focal margin, leveraging segmentation ground truth masks as prior knowledge; secondly, reconstructing the two source images using mixed superpixels as mutual masks, emphasizing spatial sensitivity. Our method stands out as a simple, one-stage, model-agnostic, and plug-and-play data augmentation solution applicable to various segmentation tasks. Notably, it requires no external data or additional models. Extensive experiments validate its superior performance across diverse medical segmentation datasets and tasks. The source codes are available at https://github.com/DanielaPlusPlus/DataAug4Medical.

Enhanced Graph Structure Representation for Unsupervised Heterogeneous Change Detection

Heterogeneous change detection (CD) is widely applied in various fields such as urban planning, environmental monitoring, and disaster management. It enhances the accuracy and comprehensiveness of surface change monitoring by integrating multi-sensor remote sensing data. Scholars have proposed many graph-based methods to address the issue of incomparable heterogeneous images caused by imaging differences. However, these methods often overlook the influence of changes in vertex status on the graph structure, which limits their ability to represent image structural features. To tackle this problem, this paper presents an unsupervised heterogeneous CD method based on enhanced graph structure representation (EGSR). This method enhances the representation capacity of the graph structure for image structural features by measuring the unchanged probabilities of vertices, thereby making it easier to detect changes in heterogeneous images. Firstly, we construct the graph structure using image superpixels and measure the structural graph differences of heterogeneous images in the same image domain. Then, we calculate the unchanged probability of each vertex in the structural graph and reconstruct the graph structure using this probability. To accurately represent the graph structure, we adopt an iterative framework for enhancing the representation of the graph structure. Finally, at the end of the iteration, the final change map (CM) is obtained by binary segmentation of the graph vertices based on their unchanged probabilities. The effectiveness of this method is validated through experiments on four sets of heterogeneous image datasets and two sets of homogeneous image datasets.

Feature descriptors using super-pixels as fuzzy numbers

ABSTRACT The objective of this paper is to extract directly local important region descriptors using image super-pixels and fuzzy numbers. Previous works are based on extracting important feature points like corners in an image then region descriptors are formed around these features. Our novel contribution is to consider directly the most discriminative super-pixels as region descriptors. First, each super-pixel is considered as a fuzzy number. Then the alpha-cut which best represents the fuzzy number is obtained. Finally, according to these alpha-cuts and the cardinality of each fuzzy number the region descriptors are formed. Matching is done according to distances between fuzzy numbers. The Palm-print recognition problem was chosen to show the effectiveness of this approach.

Modern garden design thinking and practice based on spatial information technology

Abstract The construction of a garden image classification model is done by combining spatial information technology and analyzing the process of hyperspectral classification of garden images is the main focus of this paper. The linear transformation of the image is performed by principal component analysis to achieve the effect of reducing image dimension. The SVM classifier is used to classify the garden images, and the hyperplane is found in the sample space to distinguish between positive and negative cases. Using a simple linear iterative algorithm, the image superpixels are segmented, and the information contained in them is fused with the features of the hyperspectral image. The positioning of the hyperpixel block impacts the calculation of the mean hyperspectral feature value for each hyperpixel region. The results show that a well-rounded designer needs to achieve 70% aesthetics and 80% rationality to present modern garden design.

Melanoma Classification via Hybrid Saliency and Conditional Random Field with Bottleneck to Optimize DeepLab

Neural networks overcome drawbacks of vision tasks by becoming convolutional in a wide range of layers. The salient map is affected by multilevels of strong pixels (superpixels) in global images and that is dependent on the hard threshold for their dividing. Deep neural networks have been established for saliency prediction of segmentation because the feature extraction must be suited to the input data. The convolutional neural network (CNN) also endures conflict between spatial pattern and a likeness of salient objects. Semantic segmentation is one of the approaches to continue classification based on these features. Therefore, upgrading the extraction process can be of use in saliency. In this work, we optimize DeepLab based on an atrous convolutional and a conditional random field (CRF) with a bottleneck in the semantic segmentation method, which serves for classification. The backbone of deep feature extraction is atrous convolution and the bottleneck based on CRF for hybrid saliency in the encoder-decoder system. The classification results are compared with some approaches for saliency prediction of recent deeper methods in an ISIC 2017 dataset. The results give better values not only for saliency prediction for segmentation but also for training and testing for classification.

Online small-object anti-fringe sorting of tobacco stem impurities based on hyperspectral superpixels

The use of tobacco stems as raw material for cigarettes reduces cost and improves the flammability of cigarettes. However, various impurities, such as plastic, reduce the purity of tobacco stems, degrade the quality of cigarettes, and endanger the health of smokers. Therefore, the correct classification of tobacco stems and impurities is crucial. This study proposes a method based on hyperspectral image superpixels and the use of light gradient boosting machine (LightGBM) classifier to categorize tobacco stems and impurities. First, the hyperspectral image is segmented using superpixels. Second, the gray-level co-occurrence matrix extracts the texture features of superpixels. Subsequently, an improved LightGBM is applied and trained with the spectral and textural features of superpixels as a classification model. Several experiments were implemented to evaluate the performance of the proposed method. The results show that the classification performance based on superpixels is better than that based on single-pixel points. The classification model based on superpixels (10 × 10 px) achieved the highest impurity recognition rate (93.8%). This algorithm has already been applied to industrial production in cigarette factories. It exhibits considerable potential in overcoming the influence of interference fringes to promote the intelligent industrial application of hyperspectral imaging.

Adaptive Density Spatial Clustering Method Fusing Chameleon Swarm Algorithm.

The density-based spatial clustering of application with noise (DBSCAN) algorithm is able to cluster arbitrarily structured datasets. However, the clustering result of this algorithm is exceptionally sensitive to the neighborhood radius (Eps) and noise points, and it is hard to obtain the best result quickly and accurately with it. To solve the above problems, we propose an adaptive DBSCAN method based on the chameleon swarm algorithm (CSA-DBSCAN). First, we take the clustering evaluation index of the DBSCNA algorithm as the objective function and use the chameleon swarm algorithm (CSA) to iteratively optimize the evaluation index value of the DBSCAN algorithm to obtain the best Eps value and clustering result. Then, we introduce the theory of deviation in the data point spatial distance of the nearest neighbor search mechanism to assign the identified noise points, which solves the problem of over-identification of the algorithm noise points. Finally, we construct color image superpixel information to improve the CSA-DBSCAN algorithm's performance regarding image segmentation. The simulation results of synthetic datasets, real-world datasets, and color images show that the CSA-DBSCAN algorithm can quickly find accurate clustering results and segment color images effectively. The CSA-DBSCAN algorithm has certain clustering effectiveness and practicality.

Fast Single-Parameter Energy Function Thresholding for Image Segmentation Based on Region Information

To solve the problems of image threshold segmentation based on weak continuous constraint theory, the running time is long, and the two parameters need to be selected manually, and therefore a fast single-parameter energy function thresholding for image segmentation based on region information (FSEFTISRI) is proposed in this paper. The proposed FSEFTISRI algorithm uses simple linear iterative clustering (SLIC) technology to pre-block the image, extract the image super-pixels, and then map the image super-pixels to the interval type-2 fuzzy set (IT2FS), so as to construct the single-parameter energy function to search the optimal threshold, and adaptively select the penalty parameters in the energy function through the class uncertainty theory. On a non-destructive testing (NDT) database and Berkeley segmentation datasets and benchmarks (BSDS), the proposed FSEFTISRI is compared with five related algorithms. The average misclassification error (ME) of the proposed FSEFTISRI algorithm on NDT and BSDS are 0.0466 and 0.0039, respectively. The results show that the proposed FSEFTISRI has acquired more satisfactory results in visual effect and evaluation index, and the running time of the proposed FSEFTISRI algorithm is shorter, which shows the effectiveness of the proposed FSEFTISRI.

Efficient Superpixel Generation for Polarimetric SAR Images with Cross-Iteration and Hexagonal Initialization

Clustering-based methods of polarimetric synthetic aperture radar (PolSAR) image superpixel generation are popular due to their feasibility and parameter controllability. However, these methods pay more attention to improving boundary adherence and are usually time-consuming to generate satisfactory superpixels. To address this issue, a novel cross-iteration strategy is proposed to integrate various advantages of different distances with higher computational efficiency for the first time. Therefore, the revised Wishart distance (RWD), which has better boundary adherence but is time-consuming, is first integrated with the geodesic distance (GD), which has higher efficiency and more regular shape, to form a comprehensive similarity measure via the cross-iteration strategy. This similarity measure is then utilized alternately in the local clustering process according to the difference between two consecutive ratios of the current number of unstable pixels to the total number of unstable pixels, to achieve a lower computational burden and competitive accuracy for superpixel generation. Furthermore, hexagonal initialization is adopted to further reduce the complexity of searching pixels for relabelling in the local regions. Extensive experiments conducted on the AIRSAR, RADARSAT-2 and simulated data sets demonstrate that the proposed method exhibits higher computational efficiency and a more regular shape, resulting in a smooth representation of land cover in homogeneous regions and better-preserved details in heterogeneous regions.

Graph Cardinality Preserved Attention Network for Fault Diagnosis of Induction Motor Under Varying Speed and Load Condition

During the long-term operation of motors, their working conditions are changing due to the industrial demands or declining health status, and traditional diagnosis methods perform poorly in that case. This article proposes a fault diagnosis method based on graph cardinality preserved attention network (GCPAT), which can work under varying working conditions, and can be generalized to the transient state. Diagnosis results are obtained by analyzing signal-converting graphs, which are composed of nodes and edges. First, the vibration signals are converted into symmetrical snowflake images by symmetrized dot pattern (SDP) method. Second, SLIC is developed to make homogeneous super-pixels in SDP images as nodes, and form graphs according to color, texture, and distance features. Finally, the GCPAT is utilized to distinguish motor status. Compared with other state-of-art methods, the results show the out-performance of GCPAT under varying working conditions both in steady and transient state.

Multitask Learning-Based for SAR Image Superpixel Generation

Most of the existing synthetic aperture radar (SAR) image superpixel generation methods are designed based on the raw SAR images or artificially designed features. However, such methods have the following limitations: (1) SAR images are severely affected by speckle noise, resulting in unstable pixel distance estimation. (2) Artificially designed features cannot be well-adapted to complex SAR image scenes, such as the building regions. Aiming to overcome these shortcomings, we propose a multitask learning-based superpixel generation network (ML-SGN) for SAR images. ML-SGN firstly utilizes a multitask feature extractor to extract deep features, and constructs a high-dimensional feature space containing intensity information, deep semantic informantion, and spatial information. Then, we define an effective pixel distance measure based on the high-dimensional feature space. In addition, we design a differentiable soft assignment operation instead of the non-differentiable nearest neighbor operation, so that the differentiable Simple Linear Iterative Clustering (SLIC) and multitask feature extractor can be combined into an end-to-end superpixel generation network. Comprehensive evaluations are performed on two real SAR images with different bands, which demonstrate that our proposed method outperforms other state-of-the-art methods.

Infrared Image Superpixel Segmentation Based on Seed Strategy of Contour Encoding

Superpixel segmentation has become a crucial pre-processing tool to reduce computation in many computer vision applications. In this paper, a superpixel extraction algorithm based on a seed strategy of contour encoding (SSCE) for infrared images is presented, which can generate superpixels with high boundary adherence and compactness. Specifically, SSCE can solve the problem of superpixels being unable to self-adapt to the image content. First, a contour encoding map is obtained by ray scanning the binary edge map, which ensures that each connected domain belongs to the same homogeneous region. Second, according to the seed sampling strategy, each seed point can be extracted from the contour encoding map. The initial seed set, which is adaptively scattered based on the local structure, is capable of improving the capability of boundary adherence, especially for small regions. Finally, the initial superpixels limited by the image contour are generated by clustering and refined by merging similar adjacent superpixels in the region adjacency graph (RAG) to reduce redundant superpixels. Experimental results on a self-built infrared dataset and the public datasets BSD500 and 3Dircadb demonstrate the generalization ability in grayscale and medical images, and the superiority of the proposed method over several state-of-the-art methods in terms of accuracy and compactness.

Unsupervised Change Detection of Multispectral Images Based on PCA and Low-Rank Prior

In this letter, we propose a new unsupervised change detection method based on low-rank prior for multispectral images. It is assumed that the changed and unchanged pixels are from different subspaces due to different appearance and statistical properties. So, low-rank representation (LRR) is employed to find informative pixels from the superpixels of the difference image (DI). Besides, taking the sparsity of changed pixels in the observed scenes into consideration, the selection rule is designed to distinguish these pixels. Then, principal component analysis (PCA) is used for the training of changed and unchanged dictionaries from these pixels. Finally, the change map is estimated by comparing the reconstruction error of each pixel in DI on changed and unchanged dictionaries. By LRR, more representative pixels are found for subsequent dictionary learning, which can efficiently improve the performance of the proposed method. Experiments on multitemporal images from the Landsat satellite demonstrate the effectiveness of the proposed method.

Adaptive and fast image superpixel segmentation approach

Superpixel is one of the most popular image over-segmentations with broad applications in the computer vision field to reduce their computations by replacing pixels as primitives. The main concerns of one superpixel generation algorithm are its accuracy and efficiency. One of the most important things in superpixel accuracy is to fit the image boundaries tightly with a few pixels as possible (namely minimal contour density, which is measured by the percent of superpixel contour pixels in the whole image). In this paper, we propose a new fast algorithm based on the clustering method to produce superpixels accurately with low contour density. First, we adopt the linear path from a pixel to one superpixel seed to define a regular term and propose a new distance measurement between them. In addition, we introduce the gradient and Local Binary Pattern (LBP) features and propose formulas of parameters in the proposed method adaptively. In this way, we can use the new distance measurement to group pixels as initial regions adaptively and produce the final superpixels by merging those small ones. Finally, we test the new algorithm on two public datasets and compare it with the state-of-the-art. Our method can generate superpixels with lower contour density while being competitive in accuracy and computational time.

Extended set of superpixel features

Superpixel-based image processing and analysis methods usually use a small set of superpixel features. Expanding the description of superpixels can improve the quality of processing algorithms. In the paper, a set of 25 basic superpixel features of shape, intensity, geometry, and location is proposed. The features meet the requirements of low computational complexity in the process of image superpixel segmentation and sufficiency for solving a wide class of application tasks. Applying the set, we present a modification of the well-known approach to the superpixel generation. It consists of fast primary superpixel segmentation of the image with a strict homogeneity predicate, which provides superpixels preserving the intensity information of the original image with high accuracy, and the subsequent enlargement of the superpixels with softer homogeneity predicates. The experiments show that the approach can significantly reduce the number of image elements, which helps to reduce the complexity of processing algorithms, meanwhile the expanded superpixels more accurately correspond to the image objects.

Dense Haze Removal by Nonlinear Transformation

Images captured in hazy or foggy weather conditions, suffer from various problems, such as limited visibility, low contrast, color distortions. These images are used in many computer vision applications, such as video surveillance, transportation, remote sensing. The elimination of the haze effect from these images is essential to ensure the perfect working of these applications. The degradation of a captured image is expressed by the physical model of hazy image formation. The physical model requires the estimation of transmission to restore a haze-free image, which is one of the most important parameters of single image dehazing (SID). Due to the ill-posed nature of SID, lots of priors/assumptions have been used. However, traditional methods fail when these priors do not hold, especially for varying haze concentrations, which lead to many issues such as incomplete haze removal or over enhancement in long-range regions. In this paper, a single image dehazing method based on a superpixel and nonlinear transformation is proposed. The proposed method transforms the minimum filtering on superpixels of a hazy image into the minimum filtering on superpixels of a haze-free image using nonlinear transformation. The nonlinear transformation prevents over enhancement in the long-range regions, while the superpixels reduce the halo artifacts in the dehazed image. The experimental results on challenging real hazy images, dense-hazy images, and synthetic images have proved that a combination of nonlinear transformation and superpixels provide the strength to the proposed method. The obtained dehazed results are evaluated qualitatively and quantitatively and it is found that the proposed method has tremendous performance as compared to state-of-the-art dehazing approaches.

Ship Detection in SAR Images via Enhanced Nonnegative Sparse Locality-Representation of Fisher Vectors

As a powerful coding strategy for superpixels in synthetic aperture radar (SAR) images, Fisher vector (FV) lies in a low-dimensional subspace and can be sparsely represented as a linear combination of training samples. The existing ship detection methods based on FVs often consider the Euclidean distances between target FVs and clutter FVs, where the subspace features of FVs are generally not exploited. In this article, we propose a new ship detection algorithm based on nonnegative sparse locality-representation (NSLR) to exploit the subspace features of FVs. The proposed NSLR method is based on the assumption that FVs of superpixels in SAR images are sparsely represented by the dictionary of background sea clutter only under a null hypothesis. In addition, we propose two FV-based filters to enhance the robustness of our newly developed NSLR to heterogeneous sea clutter environments by further exploiting the intrinsic features of ship targets in terms of intensity and spatiality. The experimental results based on Gaofen-3 SAR images demonstrate that the proposed NSLR detection method provides higher target-to-clutter contrast and achieves better detection performance than other commonly used ship detection algorithms.

A Hierarchical Superpixel-Based Approach for DIBR View Synthesis.

View synthesis allows observers to explore static scenes using aligned color images and depth maps captured in a preset camera path. Among the options, depth-image-based rendering (DIBR) approaches have been effective and efficient since only one pair of color and depth map is required, saving storage and bandwidth. The present work proposes a novel DIBR pipeline for view synthesis that properly tackles the different artifacts that arise from 3D warping, such as cracks, disocclusions, ghosts, and out-of-field areas. A key aspect of our contributions relies on the adaptation and usage of a hierarchical image superpixel algorithm that helps to maintain structural characteristics of the scene during image reconstruction. We compare our approach with state-of-the-art methods and show that it attains the best average results in two common assessment metrics under public still-image and video-sequence datasets. Visual results are also provided, illustrating the potential of our technique in real-world applications.

An Improved Image Segmentation Algorithm CT Superpixel Grid Using Active Contour

The traditional CT image segmentation algorithm is easy to ignore image contour initialization, which leads to the problem of long time consuming and low accuracy. A superpixel mesh CT image improved segmentation algorithm using active contour was proposed. CT image superpixel gridding was carried out first; secondly, on the basis of gridding, the region growth criterion was improved by superpixel processing, the region growth graph was established, the image edge salient graph was calculated based on the growth graph, and the target edge was obtained as the initial contour; finally, the Mumford‐Shah model in the active contour model was improved; the energy functional was constructed based on the improved model and transformed into the symbol distance function. The results show that the proposed algorithm takes less time to mesh superpixels, the accuracy of image edge calculation is high, the correct classification coefficient is as high as 0.9, and the accuracy of CT image segmentation is always higher than 90%, which has superiority.