Number Of Superpixels Research Articles

Clustering classifier of FRP strengthened concrete beams using superpixels and principal component analysis

Nowadays, the application of Fiber Reinforced Polymer (FRP) materials in concrete structures is more and more extended because of their advantages. However, premature and brittle failure caused by debonding of FRP materials from the concrete surface may be critical and needs to be detected at its earliest stages to avoid possible catastrophic events. Digital image correlation (DIC) has been used in the past for the detection of abnormalities from surface digital images. In this work, an efficient computer-aided diagnosis (CAD) system for damage identification for this type of FRP strengthening is developed. It is based on the combination of cells of strain maps built with a DIC system and an automated clustering classifier optimized with the use of superpixels and principal component analysis. Additionally, preprocessing of the images is also done with some filters to enhance the accuracy of the methodology. The procedure has been validated with experimental results on a progressively damaged FRP externally strengthened beam. Results show that the proposed detection scheme achieves an effective automated classification method for the damage detection of this kind of strengthened specimens. Furthermore, the optimal choice of the subset size, number of superpixels and principal components contributed to the success of the method. Simultaneously, an attempt to quantify damage from the proposed method and its validation with electromechanical impedance method has been developed.

SMALE: Hyperspectral Image Classification via Superpixels and Manifold Learning

As an extremely efficient preprocessing tool, superpixels have become more and more popular in various computer vision tasks. Nevertheless, there are still several drawbacks in the application of hyperspectral image (HSl) processing. Firstly, it is difficult to directly apply superpixels because of the high dimension of HSl information. Secondly, existing superpixel algorithms cannot accurately classify the HSl objects due to multi-scale feature categorization. For the processing of high-dimensional problems, we use the principle of PCA to extract three principal components from numerous bands to form three-channel images. In this paper, a novel superpixel algorithm called Seed Extend by Entropy Density (SEED) is proposed to alleviate the seed point redundancy caused by the diversified content of HSl. It also focuses on breaking the dilemma of manually setting the number of superpixels to overcome the difficulty of classification imprecision caused by multi-scale targets. Next, a space–spectrum constraint model, termed Hyperspectral Image Classification via superpixels and manifold learning (SMALE), is designed, which integrates the proposed SEED to generate a dimensionality reduction framework. By making full use of spatial context information in the process of unsupervised dimension reduction, it could effectively improve the performance of HSl classification. Experimental results show that the proposed SEED could effectively promote the classification accuracy of HSI. Meanwhile, the integrated SMALE model outperforms existing algorithms on public datasets in terms of several quantitative metrics.

Forecasting water resources from satellite image time series using a graph-based learning strategy

Abstract. In the context of climate change, it is important to monitor the dynamics of the Earth’s surface in order to prevent extreme weather phenomena such as floods and droughts. To this end, global meteorological forecasting is constantly being improved, with a recent breakthrough in deep learning methods. In this paper, we propose to adapt a recent weather forecasting architecture, called GraphCast, to a water resources forecasting task using high-resolution satellite image time series (SITS). Based on an intermediate mesh, the data geometry used within the network is adapted to match high spatial resolution data acquired in two-dimensional space. In particular, we introduce a predefined irregular mesh based on a segmentation map to guide the network’s predictions and bring more detail to specific areas. We conduct experiments to forecast water resources index two months ahead on lakes and rivers in Italy and Spain. We demonstrate that our adaptation of GraphCast outperforms the existing frameworks designed for SITS analysis. It also showed stable results for the main hyperparameter, i.e., the number of superpixels. We conclude that adapting global meteorological forecasting methods to SITS settings can be beneficial for high spatial resolution predictions.

A Comprehensive Review and New Taxonomy on Superpixel Segmentation

Superpixel segmentation consists of partitioning images into regions composed of similar and connected pixels. Its methods have been widely used in many computer vision applications, since it allows for reducing the workload, removing redundant information, and preserving regions with meaningful features. Due to the rapid progress in this area, the literature fails to catch up on more recent works among the compared ones and to categorize the methods according to all existing strategies. This work fills this gap by presenting a comprehensive review with a new taxonomy for superpixel segmentation, in which methods are classified according to their processing steps and processing levels of image features. We revisit the recent and popular literature according to our taxonomy and evaluate 23 strategies and a grid segmentation baseline based on nine criteria: connectivity, compactness, delineation, control over the number of superpixels, color homogeneity, robustness, running time, stability, and visual quality. Our experiments show the trends of each approach in superpixel segmentation and discuss individual trade-offs. Finally, we provide a new benchmark for superpixel assessment, available at https://github.com/IMScience-PPGINF-PucMinas/superpixel-benchmark .

Use of superpixels for improvement of inter-rater and intra-rater reliability during annotation of medical images

In the context of automatic medical image segmentation based on statistical learning, raters’ variability of ground truth segmentations in training datasets is a widely recognized issue. Indeed, the reference information is provided by experts but bias due to their knowledge may affect the quality of the ground truth data, thus hindering creation of robust and reliable datasets employed in segmentation, classification or detection tasks. In such a framework, automatic medical image segmentation would significantly benefit from utilizing some form of presegmentation during training data preparation process, which could lower the impact of experts’ knowledge and reduce time-consuming labeling efforts. The present manuscript proposes a superpixels-driven procedure for annotating medical images. Three different superpixeling methods with two different number of superpixels were evaluated on three different medical segmentation tasks and compared with manual annotations. Within the superpixels-based annotation procedure medical experts interactively select superpixels of interest, apply manual corrections, when necessary, and then the accuracy of the annotations, the time needed to prepare them, and the number of manual corrections are assessed. In this study, it is proven that the proposed procedure reduces inter- and intra-rater variability leading to more reliable annotations datasets which, in turn, may be beneficial for the development of more robust classification or segmentation models. In addition, the proposed approach reduces time needed to prepare the annotations.

Multiobjective Evolutionary Superpixel Segmentation for PolSAR Image Classification

Superpixel segmentation has been widely used in the field of computer vision. The generations of PolSAR superpixels have also been widely studied for their feasibility and high efficiency. The initial numbers of PolSAR superpixels are usually designed manually by experience, which has a significant impact on the final performance of superpixel segmentation and the subsequent interpretation tasks. Additionally, the effective information of PolSAR superpixels is not fully analyzed and utilized in the generation process. Regarding these issues, a multiobjective evolutionary superpixel segmentation for PolSAR image classification is proposed in this study. It contains two layers, an automatic optimization layer and a fine segmentation layer. Fully considering the similarity information within the superpixels and the difference information among the superpixels simultaneously, the automatic optimization layer can determine the suitable number of superpixels automatically by the multiobjective optimization for PolSAR superpixel segmentation. Considering the difficulty of the search for accurate boundaries of complex ground objects in PolSAR images, the fine segmentation layer can further improve the qualities of superpixels by fully using the boundary information of good-quality superpixels in the evolution process for generating PolSAR superpixels. The experiments on different PolSAR image datasets validate that the proposed approach can automatically generate high-quality superpixels without any prior information.

Memory-efficient semantic segmentation oflarge microscopy images using graph-based neural networks.

We present a graph neural network (GNN)-based framework applied to large-scale microscopy image segmentation tasks. While deep learning models, like convolutional neural networks (CNNs), have become common for automating image segmentation tasks, they are limited by the image size that can fit in the memory of computational hardware. In a GNN framework, large-scale images are converted into graphs using superpixels (regions of pixels with similar color/intensity values), allowing us to input information from the entire image into the model. By converting images with hundreds of millions of pixels to graphs with thousands of nodes, we can segment large images using memory-limited computational resources. We compare the performance of GNN- and CNN-based segmentation in terms of accuracy, training time and required graphics processing unit memory. Based on our experiments with microscopy images of biological cells and cell colonies, GNN-based segmentation used one to three orders-of-magnitude fewer computational resources with only a change in accuracy of ‒2 % to +0.3 %. Furthermore, errors due to superpixel generation can be reduced by either using better superpixel generation algorithms or increasing the number of superpixels, thereby allowing for improvement in the GNN framework's accuracy. This trade-off between accuracy and computational cost over CNN models makes the GNN framework attractive for many large-scale microscopy image segmentation tasks in biology.

Adaptive window based collaborative representation for hyperspectral anomaly detection with fusion of local and global information

Hyperspectral anomaly detection using collaborative representation (CR) has attracted high interest in recent years. Ignoring global information and the use of fixed dual window, which is inappropriate for targets with different sizes, are some disadvantages of the existing methods. In this paper, the adaptive window based CR, called as AWCR, is proposed, which utilizes the results of two segmentation maps with different numbers of superpixels to find appropriate size of inner and outer windows for each test pixel. In addition to local information contained in adaptive dual windows, two individual dictionaries are obtained for background and anomaly subspaces from the whole image to provide the global information. Both local and global residual terms are fused to result in the final residual term in AWCR. The experiments show high detection performance with a reasonable computation time for AWCR compared to several serious competitors.

High Quality Superpixel Generation Through Regional Decomposition

Superpixel generation is increasingly an important area for computer vision tasks. While superpixels with highly regular shapes are preferred to make the subsequent processing easier, the accuracy of the superpixel boundaries is also necessary. Previous methods usually depend on a distance function considering both spatial and color coherency regularization on the whole image, which however is hard to balance between shape regularity and boundary adherence, especially when the desired number of superpixels is small. In addition, non-adaptive parameters and insufficient contour information also affect the performance of segmentation. To mitigate these problems, we propose a robust divide-and-conquer superpixel segmentation method, of which the core idea is that we apply a new contour information extraction and a pixel clustering to separate the input image into flat and non-flat regions, where the former targets shape regularity and the latter emphasizes boundary adherence, followed by an efficient hierarchical merging to clean up tiny and dangling superpixels. Our algorithm requires no additional parameter tuning except the desired number of superpixels since our internal parameters are self-adaptive to the image contents. Experimental results demonstrate that for public benchmark datasets, our algorithm consistently generates more regular superpixels with stronger boundary adherence than state-of-the-art methods while maintaining a competitive efficiency. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/YunyangXu/HQSGRD</uri> .

Superpixel segmentation based on image density

Superpixel segmentation can get the middle features in image processing, effectively reduce the dimensionality of the image, and is widely used in image processing fields. To get the regular and compact superpixels in real-time, a superpixel segmentation algorithm based on image density is proposed in this paper. Firstly, the image is uniformly divided according to the number of superpixels to be obtained. Secondly, to get the clustering ability of the pixels, the density image is produced. Thirdly, the seed is chosen in each sub-region block according to the density and then the superpixels are obtained by clustering. During the clustering process, the pixel around the seed should be added into the superpixel if it meets the conditions, and the small supeipixels are merged into the big superpixels around them. Finally, the result shows that the proposed algorithm has the best segmentation effect, and a good balance in accuracy, regularity, and time cost.

Ensemble-based genetic algorithm explainer with automized image segmentation: A case study on melanoma detection dataset

Explainable Artificial Intelligence (XAI) makes AI understandable to the human user particularly when the model is complex and opaque. Local Interpretable Model-agnostic Explanations (LIME) has an image explainer package that is used to explain deep learning models. The image explainer of LIME needs some parameters to be manually tuned by the expert in advance, including the number of top features to be seen and the number of superpixels in the segmented input image. This parameter tuning is a time-consuming task. Hence, with the aim of developing an image explainer that automizes image segmentation, this paper proposes Ensemble-based Genetic Algorithm Explainer (EGAE) for melanoma cancer detection that automatically detects and presents the informative sections of the image to the user. EGAE has three phases. First, the sparsity of chromosomes in GAs is determined heuristically. Then, multiple GAs are executed consecutively. However, the difference between these GAs are in different number of superpixels in the input image that result in different chromosome lengths. Finally, the results of GAs are ensembled using consensus and majority votings. This paper also introduces how Euclidean distance can be used to calculate the distance between the actual explanation (delineated by experts) and the calculated explanation (computed by the explainer) for accuracy measurement. Experimental results on a melanoma dataset show that EGAE automatically detects informative lesions, and it also improves the accuracy of explanation in comparison with LIME efficiently. The python codes for EGAE, the ground truths delineated by clinicians, and the melanoma detection dataset are available at https://github.com/KhaosResearch/EGAE.

Research on spatiotemporal characteristics and trends of urban housing sites based on remote sensing analysis technology

In order to improve the analysis effect of the spatiotemporal characteristics of urban homesteads, this paper combines remote sensing analysis technology to study and analyze the spatiotemporal characteristics and trends of urban homesteads, segment the spatiotemporal characteristics of urban homesteads, and use the watershed transformation based on multi-scale morphological reconstruction to generate superpixels. The watershed transformation based on multi-scale morphological gradient reconstruction can obtain superpixel segmentation with good local information with less time consumption and a smaller number of superpixels. The research shows that the spatiotemporal feature analysis model of urban homestead based on remote sensing analysis technology proposed in this paper can effectively improve the analysis effect of urban homestead spatiotemporal feature, and can predict the spatiotemporal feature trend of urban homestead on this basis.

Superpixel-Based PSO Algorithms for Color Image Quantization.

Nature-inspired artificial intelligence algorithms have been applied to color image quantization (CIQ) for some time. Among these algorithms, the particle swarm optimization algorithm (PSO-CIQ) and its numerous modifications are important in CIQ. In this article, the usefulness of such a modification, labeled IDE-PSO-CIQ and additionally using the idea of individual difference evolution based on the emotional states of particles, is tested. The superiority of this algorithm over the PSO-CIQ algorithm was demonstrated using a set of quality indices based on pixels, patches, and superpixels. Furthermore, both algorithms studied were applied to superpixel versions of quantized images, creating color palettes in much less time. A heuristic method was proposed to select the number of superpixels, depending on the size of the palette. The effectiveness of the proposed algorithms was experimentally verified on a set of benchmark color images. The results obtained from the computational experiments indicate a multiple reduction in computation time for the superpixel methods while maintaining the high quality of the output quantized images, slightly inferior to that obtained with the pixel methods.

Image classification using graph neural network and multiscale wavelet superpixels

Prior studies using graph neural networks (GNNs) for image classification have focused on graphs generated from a regular grid of pixels or similar-sized superpixels. In the latter, a single target number of superpixels is defined for an entire dataset irrespective of differences across images and their intrinsic multiscale structure. On the contrary, this study investigates image classification using graphs generated from an image-specific number of multiscale superpixels. We propose WaveMesh, a new wavelet-based superpixeling algorithm, where the number and sizes of superpixels in an image are systematically computed based on its content. WaveMesh superpixel graphs are structurally different from similar-sized superpixel graphs. We use SplineCNN, a state-of-the-art network for image graph classification, to compare WaveMesh and similar-sized superpixels. Using SplineCNN, we perform extensive experiments on three benchmark datasets under three local-pooling settings: 1) no pooling, 2) GraclusPool, and 3) WavePool, a novel spatially heterogeneous pooling scheme tailored to WaveMesh superpixels. Our experiments demonstrate that SplineCNN learns from multiscale WaveMesh superpixels on-par with similar-sized superpixels. In all WaveMesh experiments, GraclusPool performs poorer than no pooling / WavePool, showing that poor cluster assignment negatively affects the performance of the network while learning from multiscale superpixels.

ESCNet: An End-to-End Superpixel-Enhanced Change Detection Network for Very-High-Resolution Remote Sensing Images.

Change detection (CD), as one of the central problems in Earth observation, has attracted a lot of research interest over recent decades. Due to the rapid development of satellite sensors in recent years, we have witnessed an enrichment of the CD source data with the availability of very-high-resolution (VHR) multispectral imagery, which provides abundant change clues. However, precisely locating real changed areas still remains a challenge. In this article, we propose an end-to-end superpixel-enhanced CD network (ESCNet) for VHR images, which combines differentiable superpixel segmentation and a deep convolutional neural network (DCNN). Two weight-sharing superpixel sampling networks (SSNs) are tailored for the feature extraction and superpixel segmentation of bitemporal image pairs. A UNet-based Siamese neural network is then employed to mine the different information. The superpixels are then leveraged to reduce the latent noise in the pixel-level feature maps while preserving the edges, where a novel superpixelation module is used to serve this purpose. Furthermore, to compensate for the dependence on the number of superpixels, we propose an innovative adaptive superpixel merging (ASM) module, which has a concise form and is fully differentiable. A pixel-level refinement module making use of the multilevel decoded features is also appended to the end of the framework. Experiments on two public datasets confirmed the superiority of ESCNet compared to the traditional and state-of-the-art (SOTA) deep learning-based CD (DLCD) methods.

Adaptive hypergraph superpixels

Superpixel segmentation, which amounts to partitioning an image into a number of superpixels each of which is a set of pixels sharing common visual meanings, requires specific needs for different computer vision tasks. Graph based methods, as a kind of popular superpixel segmentation method, regard an image as a weighted graph whose nodes correspond to pixels of the image, and partition all pixels into superpixels according to the similarity between pixels over various feature spaces. Despite their improvement of the performance of segmentation, these methods ignore high-order relationship between them incurred from either locally neighboring pixels or structured layout of the image. Moreover, they measure the similarity of pairwise pixels using Gaussian kernel where a robust radius parameter is difficult to find for pixels which exhibit multiple features (e.g., texture, color, brightness). In this paper, we propose an adaptive hypergraph superpixel segmentation (AHS) of intensity images for solving both issues. AHS constructs a hypergraph by building the hyperedges with an adaptive neighborhood scheme, which explores an intrinsic relationship of pixels. Afterwards, AHS encodes the relationship between pairwise pixels using characteristics of current two pixels as well as their neighboring pixels defined by hyperedges. Essentially, AHS models the relationship of pairwise pixels in a high-order group fashion while graph based methods evaluate it in a one-vs-one fashion. Experiments on four datasets demonstrate that AHS achieves higher or comparable performance compared with state-of-the-art methods.

A Hybrid Linear Iterative Clustering and Bayes Classification-Based GrabCut Segmentation Scheme for Dynamic Detection of Cervical Cancer

Cervical cancer earlier detection remains indispensable for enhancing the survival rate probability among women patients worldwide. The early detection of cervical cancer is done relatively by using the Pap Smear cell Test. This method of detection is challenged by the degradation phenomenon within the image segmentation task that arises when the superpixel count is minimized. This paper introduces a Hybrid Linear Iterative Clustering and Bayes classification-based GrabCut Segmentation Technique (HLC-BC-GCST) for the dynamic detection of Cervical cancer. In this proposed HLC-BC-GCST approach, the Linear Iterative Clustering process is employed to cluster the potential features of the preprocessed image, which is then combined with GrabCut to prevent the issues that arise when the number of superpixels is minimized. In addition, the proposed HLC-BC-GCST scheme benefits of the advantages of the Gaussian mixture model (GMM) on the extracted features from the iterative clustering method, based on which the mapping is performed to describe the energy function. Then, Bayes classification is used for reconstructing the graph cut model from the extracted energy function derived from the GMM model-based Linear Iterative Clustering features for better computation and implementation. Finally, the boundary optimization method is utilized to considerably minimize the roughness of cervical cells, which contains the cytoplasm and nuclei regions, using the GrabCut algorithm to facilitate improved segmentation accuracy. The results of the proposed HLC-BC-GCST scheme are 6% better than the results obtained by other standard detection approaches of cervical cancer using graph cuts.

BACA: Superpixel Segmentation with Boundary Awareness and Content Adaptation

Superpixels could aggregate pixels with similar properties, thus reducing the number of image primitives for subsequent advanced computer vision tasks. Nevertheless, existing algorithms are not effective enough to tackle computing redundancy and inaccurate segmentation. To this end, an optimized superpixel generation framework termed Boundary Awareness and Content Adaptation (BACA) is presented. Firstly, an adaptive seed sampling method based on content complexity is proposed in the initialization stage. Different from the conventional uniform mesh initialization, it takes content differentiation into consideration to incipiently eliminate the redundancy of seed distribution. In addition to the efficient initialization strategy, this work also leverages contour prior information to strengthen the boundary adherence from whole to part. During the similarity calculation of inspecting the unlabeled pixels in the non-iterative clustering framework, a multi-feature associated measurement is put forward to ameliorate the misclassification of boundary pixels. Experimental results indicate that the two optimizations could generate a synergistic effect. The integrated BACA achieves an outstanding under-segmentation error (3.34%) on the BSD dataset over the state-of-the-art performances with a minimum number of superpixels (345). Furthermore, it is not limited to image segmentation and can be facilitated by remote sensing imaging analysis.

Scale‐adaptive superpixels for medical images

BackgroundSuperpixel segmentation is a powerful preprocessing tool to reduce the complexity of image processing. Traditionally, size uniformity is one of the significant features of superpixels. However, in medical images, in which subjects scale varies greatly and background areas are often flat, size uniformity rarely conforms to the varying content. To obtain the fewest superpixels with retaining important details, the size of superpixel should be chosen carefully.MethodsWe propose a scale‐adaptive superpixel algorithm relaxing the size‐uniformity criterion for medical images, especially pathological images. A new path‐based distance measure and superpixel region growing schema allow our algorithm to generate superpixels with different scales according to the complexity of image content, that is smaller (larger) superpixels in color‐riching areas (flat areas).ResultsThe proposed superpixel algorithm can generate superpixels with boundary adherence, insensitive to noise, and with extremely big sizes and extremely small sizes on one image. The number of superpixels is much smaller than size‐uniformly superpixel algorithms while retaining more details of images.ConclusionWith the proposed algorithm, the choice of superpixel size is automatic, which frees the user from the predicament of setting suitable superpixel size for a given application. The results on the nuclear dataset show that the proposed superpixel algorithm superior to the respective state‐of‐the‐art algorithms on both quantitative and quantitative comparisons.

Efficient Color Quantization Using Superpixels.

We propose three methods for the color quantization of superpixel images. Prior to the application of each method, the target image is first segmented into a finite number of superpixels by grouping the pixels that are similar in color. The color of a superpixel is given by the arithmetic mean of the colors of all constituent pixels. Following this, the superpixels are quantized using common splitting or clustering methods, such as median cut, k-means, and fuzzy c-means. In this manner, a color palette is generated while the original pixel image undergoes color mapping. The effectiveness of each proposed superpixel method is validated via experimentation using different color images. We compare the proposed methods with state-of-the-art color quantization methods. The results show significantly decreased computation time along with high quality of the quantized images. However, a multi-index evaluation process shows that the image quality is slightly worse than that obtained via pixel methods.