Number Of Superpixels Research Articles

Oil Spill Detection with Dual-Polarimetric Sentinel-1 SAR Using Superpixel-Level Image Stretching and Deep Convolutional Neural Network

Synthetic aperture radar (SAR) has been widely applied in oil spill detection on the sea surface due to the advantages of wide area coverage, all-weather operation, and multi-polarization characteristics. Sentinel-1 satellites can provide dual-polarized SAR data, and they have high potential for successful application to oil spill detection. However, the characteristics of the sea surface and oil film on different images are not the same when imaging at different locations and in different conditions, which leads to the inconsistent accuracy of these images with the application of the current oil spill detection methods. In order to avoid the above limitation, we propose an oil spill detection method using image stretching based on superpixels and a convolutional neural network. Experiments were carried out on eight Sentinel-1 dual-pol data, and the optimal superpixel number and image stretching parameters are discussed. Mean intersection over union (MIoU) was used to evaluate classification accuracy. The proposed method could effectively improve the classification accuracy; when the expansion and inhibition coefficients of image stretching were set to 1.6 and 1.2 respectively, the experiments achieved a maximum MIoU of 85.4%, 7.3% higher than that without image stretching.

HIGH PRECISION AUTOMATIC EXTRACTION OF CULTURAL RELIC DISEASES BASED ON IMPROVED SLIC AND AP CLUSTERING

Abstract. Automatic and high-precision detection and quantitative expression of cultural relics diseases are important contents of cultural relics science and technology protection. Aiming at the problem of automatic extraction of boundary cultural relics diseases, this paper proposes an adaptive SLIC0 combined with AP clustering method to achieve high-precision detection of disease areas. Firstly, based on the SLIC0 algorithm, the selected area of the disease orthophoto frame is segmented, and the Canny edge detection is used as the true value. The number of superpixels is iterative until the accuracy meets the requirements, so as to achieve the best fitting of superpixel edges. Then the AP clustering method is used to merge the superpixels of the disease area to obtain the edge information. Finally, taking the surface shedding disease of painted cultural relics as an example, this method is applied to realize the high precision extraction and quantitative expression of the edge of the disease. The correctness, feasibility and advancement of the algorithm are proved by comparing with the existing manual methods. The method in this paper provides an efficient and high-precision means for the quantitative expression of cultural relics diseases, and can provide accurate data support for the scientific and technological restoration of cultural relics.

Recognition and segmentation of complex texture images based on superpixel algorithm and deep learning

Recognizing and segmenting complex texture images such as materials is of great significance to industrial design and production. Due to the lack of sufficient training samples and fuzzy boundaries in material images, it is difficult to segment material images by using deep learning methods. In material images, the pixels of each phase have a high degree of similarity, so if partial pixels’ features in each phase are learned, the whole phase can be recognized. In this paper, we propose a method based on deep learning for recognizing and segmenting material images with complex textures. Firstly, the simple linear iterative cluster(SLIC) algorithm is used to obtain different numbers of superpixels which are a group of pixels with similar texture features. Then we extract the largest inscribed rectangular block in each superpixel. Next, put these rectangular blocks into the classical convolutional neural network(CNN)—DenseNet to recognize them. To retain the key texture features and reduce redundant information, we increase the receptive field in the key layers of DenseNet. In addition, due to the uneven distribution of phases in the material images, we improve focal loss to fit the material image. We make extensive comparative and ablation experiments to confirm the effectiveness of our method.

A segmentation algorithm incorporating superpixel block and holistically nested edge for sugarcane aphids images under natural light conditions

Image-based automatic detection systems illustrate the efficient capability of the pest ( e.g. , sugarcane aphids) identification. However, most of the algorithms were developed using the images captured under stable light conditions and the natural illumination has been rarely considered. Here, we developed a fusion superpixel method and compared it with eight other state-of-the-art superpixel algorithms on a developed dataset of images under various natural light conditions of strong light, diffuse light, weak light, and direct sunlight. We evaluated the algorithm performance according to visual quality, ground truth deviation ( e.g. , target compactness, regularity index, boundary recall, and under-segmentation error), and segmentation runtime and quality. We found 400 as the best number of superpixels for segmenting the pest image with the resolution of 408 × 306 pixels and captured with the distance of ∼0.2 m between target and camera lens. Compared with others, the developed fusion algorithm illustrated a relatively better performance in target compactness (0.61), boundary recall (0.76), regularity index (16.4), and runtime (0.13 s). Furthermore, diffuse light was the ideal light condition in the sugarcane aphids’ identification task. Our study suggested the developed algorithm as an appropriate method concerning various natural light conditions and pest distribution in the field. • A Holistically-Nested Edge was introduced in superpixel for image segmentation. • The proposed superpixel algorithm performed better compared with previous methods. • Diffuse light was the ideal light condition in pest identification task. • K = 400 is the best number of superpixels for segmenting our sugarcane aphid images.

Graph-based modelling of superpixels for automatic identification of empty shelves in supermarkets

Automatic detection of empty spaces (gaps) between the displayed products as seen in the images of shelves of a supermarket is an interesting image segmentation problem. This paper presents the first known attempt to solve this commercially relevant challenge. The shelf image is first over-segmented into a number of superpixels to construct a graph of superpixels (SG). Subsequently, a graph convolutional network and a Siamese network are built to process the SG. Finally, a structural support vector machine based inference model is formulated based on SG for segmenting the gap and non-gap regions. In order to validate our method, we manually annotate the images of shelves of three benchmark datasets of retail products. We have achieved ∼70 to ∼85% segmentation accuracy (in terms of mean intersection-over-union) on the annotated datasets. A part of the annotated data is released at https://github.com/gapDetection/gapDetectionDatasets.

Hyperspectral Classification Based on Coupling Multiscale Super-Pixels and Spatial Spectral Features

Hyperspectral remotely sensed images contain not only the spatial information of ground objects, but also their rich spectral information. Effectively improve the classification accuracy of hyperspectral images (HSIs), the purpose is to accurately grasp the current land resource utilization information, which is of great significance to the formulation and implementation of future land and space planning. Existing research on the classification of hyperspectral remotely sensed images uses a single-scale superpixel method for image segmentation. The optimal number of superpixels cannot be determined, image details may get missed, and a single kernel matrix cannot represent the information from multiple features, which results in reduced classification accuracy. Therefore, this study intends to use the superpixel segmentation method to perform multiscale superpixel segmentation on the principal components of HSIs at multiple scales, and to couple the multiscale superpixel spatial spectral kernel (SSK) with the original spectral kernel to form a synthetic kernel using weights for HSI classification. The three HSIs of Pavia University, Pavia Center, and Washington DC Mall are used as the experimental data to test and analyze this method. The experimental results show that the effective classification accuracies for the three datasets obtained by this method are 5.28%, 5.90%, and 7.71% higher than the five comparison methods, at best. The results prove that this method can effectively solve the problem of imprecise image feature extraction and an unknown number of initial superpixels and can significantly improve the classification accuracy of HSIs.

Fast SAR Image Segmentation With Deep Task-Specific Superpixel Sampling and Soft Graph Convolution

Since the number of superpixels is lower than that of pixels, superpixels can substantially speed up subsequent processing steps and have been widely used in synthetic aperture radar (SAR) image segmentation. However, in most of the existing superpixel-wise segmentation algorithms, superpixel prediction is an isolated preprocessing step and is independent of the segmentation task. The performance of the segmentation results is determined by the accuracy of superpixels. Once superpixels are generated, their shape cannot be changed in the following segmentation stage, even if the same superpixels contain pixels of different landcovers. To address this, we propose an end-to-end trainable superpixel-wise segmentation method for single-polarization SAR images. First, we design a differentiable boundary-ware clustering method for estimating task-specific superpixels. Instead of the hard association between pixels and superpixels in the existing superpixel algorithms, this method introduces the soft association map to make the clustering differentiable. Hence, it can be implemented using a simple deep fully convolutional network. In the segmentation part, we propose a novel soft graph convolution network (Soft-GCN), which takes the association map as input and performs superpixel-wise segmentation. The advantage of our method is that superpixel generation and graph convolution parts can be trained under a unified framework, until two parts obtain the optimum parameters. In the training process, it can adaptively adjust the shape of the superpixels according to the segmentation results, ensuring the superpixels correctly adhere the boundaries. Experimental results with simulated and real SAR images demonstrate that our method outperforms other state-of-the-art segmentation algorithms, while also being faster.

Superpixel Sizes using Topology Preserved Regular Superpixel Algorithm and their Impacts on Interactive Segmentation

Interactive Image Segmentation is a type of semi-automated segmentation that uses user input to extract the object of interest. It is possible to speed up and improve the end result of segmentation by using pre-processing steps. The use of superpixels is an example of a pre-processing step. A superpixel is a collection of pixels with similar properties such as texture and colour. Previous research was conducted to assess the impact of the number of superpixels (based on SEEDS superpixel aglorithms) required to achieve the best segmentation results. The study, however, only examined one type of input (strokes) and a small number of images. As a result, the goal of this study is to extend previous work by performing interactive segmentation with input strokes and a combination of bounding box and strokes on images from Grabcut image data sets generated by Topology preserved regular superpixel (TPRS). Based on our findings, an image with 1000 to 2500 superpixels and a combination of bounding box and strokes will help the interactive segmentation algorithm produce a good segmentation result. Finally, the size of the superpixels would influence the final segmentation results as well as the input type.

Video object segmentation via random walks on two-frame graphs comprising superpixels

We propose a novel video object segmentation method employing random walkers to travel on graphs constructed on two consecutive frames. First, we estimate the initial foreground and background distributions by minimising an energy function that incorporates the stationary distributions of the random walks. The random walkers frequently travel between similar nodes of the graph constructed on two adjacent frames, which enables the incorporation of the inter-frame information into the energy function effectively and elegantly. Then, we refine the initial results by simulating the movements of multiple random walkers. We process the sequence in a recursive manner, which naturally propagates the previous segmentation labels to the subsequent frames. Additionally, we develop a strategy for adjusting the superpixel number using region similarity and the average Frobenius norm of optical flow gradient. This strategy can improve performance significantly. Furthermore, we discuss the feature selection problem in the method to select a more effective feature representation. Extensive and comparable experiments on Segtrack and Segtrack v2 demonstrate that the proposed algorithm yields higher performance than several recent state-of-the-art approaches.

Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images

Tissue/region segmentation of pathology images is essential for quantitative analysis in digital pathology. Previous studies usually require full supervision (e.g., pixel-level annotation) which is challenging to acquire. In this paper, we propose a weakly-supervised model using joint Fully convolutional and Graph convolutional Networks (FGNet) for automated segmentation of pathology images. Instead of using pixel-wise annotations as supervision, we employ an image-level label (i.e., foreground proportion) as weakly-supervised information for training a unified convolutional model. Our FGNet consists of a feature extraction module (with a fully convolutional network) and a classification module (with a graph convolutional network). These two modules are connected via a dynamic superpixel operation, making the joint training possible. To achieve robust segmentation performance, we propose to use mutable numbers of superpixels for both training and inference. Besides, to achieve strict supervision, we employ an uncertainty range constraint in FGNet to reduce the negative effect of inaccurate image-level annotations. Compared with fully-supervised methods, the proposed FGNet achieves competitive segmentation results on three pathology image datasets (i.e., HER2, KI67, and H&E) for cancer region segmentation, suggesting the effectiveness of our method. The code is made publicly available at https://github.com/zhangjun001/FGNet.

Content-Sensitive Superpixel Generation for SAR Images With Edge Penalty and Contraction–Expansion Search Strategy

In this article, we present a content-sensitive superpixel generation method with edge penalty and the contraction–expansion search strategy (EPCES) for synthetic aperture radar (SAR) images. Specifically, the edge information can be obtained by our previously proposed ratio-based edge detector with recurrent guidance filter, which has been proven to be robust to speckle noise and capable of detecting weak edges in low-contrast areas. The content-sensitive superpixel seeds’ initialization method is proposed with respect to the heterogeneous state of the SAR imagery, benefiting from which EPCES can generate an exact number of superpixels set by the user and the fine details can be preserved well. In EPCES, a new dissimilarity with edge penalty is defined to generate the superpixels with better edge adherence. Rather than adopting the conventional clustering method based on local <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means, we propose the contraction–expansion search strategy (CES), which explicitly utilizes the continuity information contained in neighboring pixels and enforces the connectivity of the superpixel without any postprocessing step. With the aid of the CES, our proposed method can attain superpixels with low computational cost and high edge adherence. Experimental results on both synthetic and real-world SAR images verify that the proposed method consistently performs favorably against several state-of-the-art methods in terms of both quality and efficiency.

Cluster-based fine-to-coarse superpixel segmentation

As an image preprocessing technology, superpixel segmentation has become an important tool in the field of computer vision. How to obtain a more accurate, faster, and easier-to-apply superpixel segmentation algorithm is a problem faced by researchers. In this paper, a cluster-based fine-to-coarse superpixel segmentation (FCSS) algorithm is proposed. By introducing color thresholds and depth thresholds with practical physical meanings as algorithm parameters, high-quality segmentation with fewer superpixels is achieved. It not only reduces the complexity of the upper application, but also provides an easy to understand interface. Superpixel segmentation methods often cannot achieve high-quality segmentation through a set of parameters. Experimental results show that FCSS can achieve finer segmentation by setting different parameters, and the segmentation results are superior to other algorithms. When the number of superpixels is 100, the segmentation performance of FCSS is better than that of existing state-of-the-art methods.

Adaptive tuning of SLIC parameter K

The well-known simple linear iterative clustering (SLIC) is the most effective among the existing algorithms for superpixel segmentation, which requires manual tuning of the number of superpixels K. The optimal value of the parameter K of the SLIC algorithm for a given image is yet an open issue. In this work, we present granulometry and quality metrics based methods for adaptive tuning of the parameter K. The proposed granulometric method exploits the weighted average of the image pattern spectrum for the adaptive tuning of the parameter K. In the quality metrics method, we use majority voting scheme based on information, texture and ground truth independent quality metrics. The experimental results demonstrate that the K SLIC superpixels from the proposed methods achieved good boundary adherence of the ground truth for the images with high value of the compactness.

A Graph Neural Network for superpixel image classification

The classification of superpixel images by graph neural networks has gradually become a research hotspot. It is a crucial issue to embed super-pixel images from lowdimensional to high-dimensional so as to turn complex image information into graph signals. This paper proposes a method for image classification using a graph neural network (GNN) model. We convert the input image into a region adjacency graph (RAG) composed of superpixels as nodes, and use residual and concat structure to extract deep features. Finally, the loss function that increases the distance between classes and compactness within classes is used as supervision. Experiments have been tested with different numbers of superpixels on multiple datasets, and the results show that our method has a great performance in superpixel images classification.

In-process monitoring of the melt-pool motion during continuous-wave laser metal deposition

Abstract In the field of additive manufacturing (AM), the implementation of real-time monitoring systems is essential to guarantee the quality and repeatability of the manufactured parts. Particularly in the laser metal deposition technique, the dynamics and stability of the molten pool, both geometrically and thermally, have an impact on the microstructural properties and the formation of defects. This article presents a novel technique to track the molten pool in real-time. A middle wavelength infrared (MWIR) camera is used, with a spectral range between 3 and 5 μm. The acquired frames are processed to reduce their dimension using a new strategy based on Superpixels. Traditional pixels are grouped into block-structures that are similar in temperature, optical flow, and spatial position. This guarantees a better adherence to the edges improving the segmentation task. Then, the optical flow of the molten pool is calculated. Throughout this tracking, geometric and thermal features are extracted. The proposed method was evaluated using qualitative and quantitative criteria. The performance of the proposal is compared against benchmark algorithms using standard metrics such as Sub-segmentation Error, Compactness, and Achievable Segmentation Accuracy obtaining better results. The proposal presented is robust to the number of superpixels generated as well as the size of the images. Which provides better performance for in situ monitoring tasks. Our proposal has significant industrial potential for monitoring additive manufacturing processes, as it requires minimal modifications to commercially available industrial machines.

Multiscale Adjacent Superpixel-Based Extended Multi-Attribute Profiles Embedded Multiple Kernel Learning Method for Hyperspectral Classification

In this paper, superpixel features and extended multi-attribute profiles (EMAPs) are embedded in a multiple kernel learning framework to simultaneously exploit the local and multiscale information in both spatial and spectral dimensions for hyperspectral image (HSI) classification. First, the original HSI is reduced to three principal components in the spectral domain using principal component analysis (PCA). Then, a fast and efficient segmentation algorithm named simple linear iterative clustering is utilized to segment the principal components into a certain number of superpixels. By setting different numbers of superpixels, a set of multiscale homogenous regional features is extracted. Based on those extracted superpixels and their first-order adjacent superpixels, EMAPs with multimodal features are extracted and embedded into the multiple kernel framework to generate different spatial and spectral kernels. Finally, a PCA-based kernel learning algorithm is used to learn an optimal kernel that contains multiscale and multimodal information. The experimental results on two well-known datasets validate the effectiveness and efficiency of the proposed method compared with several state-of-the-art HSI classifiers.

Adaptive superpixel-based multi-object pedestrian recognition

This paper proposed an adaptive multi-object pedestrian recognition algorithm based on SLIC. First, we used SLIC to superpixel the pre-segmentation processing on the image. Then, the hash distance is added as the superpixel point aggregation parameter based on the traditional superpixel measurement unit of LAB color space distance and position distance. Finally, we identified the clustering subject by using the extreme learning machine neural network. The proposed method can adaptively determine the number of superpixels to achieve high recognition performance. This method simply needs to preset the number of pre-segments, which can reduce the number of detection targets, improve the segmentation efficiency, and shorten the image identification time.

Image superpixel segmentation based on hierarchical multi-level LI-SLIC

In computer vision, superpixel segmentation has been widely used as a very important preprocessing to reduce the number of image primitives for subsequent image processing tasks. To improve the segmentation accuracy and the robustness to noise, a hierarchical multi-level segmentation framework is developed in this paper. First, original image is initially segmented by a local information based simple liner iterative clustering (LI-SLIC) method. Then, the initial generated superpixels are further segmented hierarchically by LI-SLIC to ensure that all pixels contained within each superpixel belong to a same object. Finally, to eliminate over-segmentation and reduce the number of superpixels, adjacent superpixels belonging to a same object are merged based on the probability distribution similarity. The proposed method does not require setting the seeds or number of the superpixels to be generated in advance, which can segment image into an appropriate number of superpixels without under- or over- segmentation automatically according to its content. Experiments are conducted on two public datasets Berkeley and 3Dircadb, and the results demonstrate that our method is more effective and accurate than many existing superpixel methods and shows a great advantage in dealing with images corrupted by various noises.

Efficient Superpixel Generation Method Based on Image Complexity

Superpixel methods are widely used in the preprocessing stage as a method to reduce computational complexity by simplifying images while maintaining the characteristics of the images in the computer vision applications. It is common to generate superpixels of similar size and shape based on the pixel values rather than considering the characteristics of the image. In this paper, we propose a method to control the sizes and shapes of generated superpixels, considering the contents of an image. The proposed method consists of two steps. The first step is to over-segment an image so that the boundary information of the image is well preserved. In the second step, generated superpixels are merged based on similarity to produce the target number of superpixels, where the shapes of superpixels are controlled by limiting the maximum size and the proposed roundness metric. Experimental results show that the proposed method preserves the boundaries of the objects in an image more accurately than the existing method.

Fuzzy SLIC: Fuzzy Simple Linear Iterative Clustering

Most superpixel methods are sensitive to noise and cannot control the superpixel number precisely. To solve these problems, in this paper, we propose a robust superpixel method called fuzzy simple linear iterative clustering (Fuzzy SLIC), which adopts a local spatial fuzzy C-means clustering and dynamic fuzzy superpixels. We develop a fast and precise superpixel number control algorithm called onion peeling (OP) algorithm. Fuzzy SLIC is insensitive to most types of noise, including Gaussian, salt and pepper, and multiplicative noise. The OP algorithm can control the superpixel number accurately without reducing much computational efficiency. In the validation experiments, we tested the Fuzzy SLIC and OP algorithm and compared them with state-of-the-art methods on the BSD500 and Pascal VOC2007 benchmarks. The experiment results show that our methods outperform state-of-the-art techniques in both noise-free and noisy environments.