Superpixel Segmentation Algorithm Research Articles

A Cloud Removal Algorithm to Generate Cloud and Cloud Shadow Free Images Using Information Cloning

One of the main problems of optical remote sensing is clouds and cloud shadows caused by specific atmospheric conditions during data acquisition. These features limit the usage of acquired images and increase the difficulty in data analysis, such as normalized difference vegetation index values, misclassification, and atmospheric correction. Accurate detection and reliable cloning of cloud and cloud shadow features in satellite images are very useful processes for optical remote sensing applications. In this study, an automated cloud removal algorithm to generate cloud and cloud shadow free images from multitemporal Landsat-8 images is introduced. Cloud and cloud shadow areas are classified by using process-based rule set developed by using spectral and spatial features after applying simple linear iterative clustering superpixel segmentation algorithm to the image to find cloud pixel groups easily and correctly. Segmentation-based cloud detection method gives better results than pixel-based for detection of cloud and cloud shadow patches. After detection of clouds and cloud shadows, cloud-free images are created by cloning cloudless regions from multitemporal dataset. Spectral and structural consistency are preserved by considering spectral features and seasonal effects while cloning process. Statistical similarity tests are applied to find best cloud-free image to use for cloning process. Cloning results are tested with the structural similarity index metric to evaluate the performance of cloning algorithm.

Use of Spectral Clustering Combined with Normalized Cuts (N-Cuts) in an Iterative k-Means Clustering Framework (NKSC) for Superpixel Segmentation with Contour Adherence

Superpixel segmentation methods are generally used as a pre-processing step to speed up image processing tasks. They group the pixels of an image into homogeneous regions while trying to respect existing contours. In this paper, we propose a fast Superpixels segmentation algorithm with Contour Adherence using spectral clustering, combined with normalized cuts in an iterative k-means clustering framework. It produces compact and uniform superpixels with low computational costs. Normalized cut is adapted to measure the color similarity and space proximity between image pixels. We have used a kernel function to estimate the similarity metric. Kernel function maps the pixel values and coordinates into a high dimensional feature space. The objective functions of weighted K-means and normalized cuts share the same optimum point in this feature space. So it is possible to optimize the cost function of normalized cuts by iteratively applying simple K-means clustering algorithm. The proposed framework produces regular and compact superpixels that adhere to the image contours. On segmentation comparison benchmarks it proves to be equally well or better than the state-of-the-art super pixel segmentation algorithms in terms of several commonly used evaluation metrics in image segmentation. In addition, our method is computationally very efficient and its computational complexity is linear.

Content-Adaptive Superpixel Segmentation.

Superpixel segmentation targets at grouping pixels in an image into atomic regions whose boundaries align well with the natural object boundaries. This paper first proposes a new feature representation for superpixel segmentation that holistically embraces color, contour, texture, and spatial features. Then, we introduce a clustering-based discriminability measure to iteratively evaluate the importance of different features. Integrating the feature representation and the discriminability measure, we propose a novel content-adaptive superpixel (CAS) segmentation algorithm. CAS is able to automatically and iteratively adjust the weights of different features to fit various properties of image instances. Experiments on several challenging datasets demonstrate that the proposed CAS outperforms the state-of-the-art methods and has a low computational cost.

Fast Superpixel Based Subspace Low Rank Learning Method for Hyperspectral Denoising

Sequential data, such as video frames and event data, have been widely applied in the real-world. As a special kind of sequential data, hyperspectral images (HSIs) can be regarded as a sequence of 2-D images in the spectral dimension, which can be effectively utilized for distinguishing different land-covers according to the spectral sequences. This paper presents a novel noise reduction method based on superpixel-based subspace low rank representation for hyperspectral imagery. First, under the framework of a linear mixture model, the original hyperspectral cube is assumed to be low rank in the spectral domain, which could be represented by decomposing HSI data into two sub-matrices of lower ranks. Meanwhile, due to the high correlation of neighboring pixels, the spectra within each neighborhood would also promote low rankness, and the local spatial low rankness could be exploited by enforcing the nuclear norm within superpixel-based regions in the decomposed subspace. The superpixels are easily and effectively generated by utilizing state-of-the-art superpixel segmentation algorithms in the first principal component of the original HSI. Moreover, benefiting from the subspace decomposition, the proposed method has an overwhelming superiority in computational cost than the state-of-the-art LR-based methods. The final model could be efficiently solved by the augmented Lagrangian method. Experimental results on simulated and real hyperspectral data sets validate that the proposed method produces superior performance than other state-of-the-art denoising methods in terms of quantitative assessment and visual quality.

Minimum barrier superpixel segmentation

Superpixel has become an essential processing unit in many computer vision systems, and superpixel segmentation of images is one of the most important step. In this paper, an efficient superpixel segmentation algorithm was proposed. We introduce a new compact-aware minimum barrier distance for superpixel segmentation (MBS), and a propagation scheme for the cluster centers between adjacent levels on a hierarchical architecture. Experiments show that it achieves state-of-the-art performance and can be configured with simple trade-off between performance and efficiency. Furthermore, the compactness of segmented superpixel could be flexibly controlled continuously by only one parameter, which could be easily integrated in other computer vision tasks. The source code of MBS is available at https://github.com/YinlinHu/MBS.

Epithelium and Stroma Identification in Histopathological Images Using Unsupervised and Semi-Supervised Superpixel-Based Segmentation

We present superpixel-based segmentation frameworks for unsupervised and semi-supervised epithelium-stroma identification in histopathological images or oropharyngeal tissue micro arrays. A superpixel segmentation algorithm is initially used to split-up the image into binary regions (superpixels) and their colour features are extracted and fed into several base clustering algorithms with various parameter initializations. Two Consensus Clustering (CC) formulations are then used: the Evidence Accumulation Clustering (EAC) and the voting-based consensus function. These combine the base clustering outcomes to obtain a more robust detection of tissue compartments than the base clustering methods on their own. For the voting-based function, a technique is introduced to generate consistent labellings across the base clustering results. The obtained CC result is then utilized to build a self-training Semi-Supervised Classification (SSC) model. Unlike supervised segmentations, which rely on large number of labelled training images, our SSC approach performs a quality segmentation while relying on few labelled samples. Experiments conducted on forty-five hand-annotated images of oropharyngeal cancer tissue microarrays show that (a) the CC algorithm generates more accurate and stable results than individual clustering algorithms; (b) the clustering performance of the voting-based function outperforms the existing EAC; and (c) the proposed SSC algorithm outperforms the supervised methods, which is trained with only a few labelled instances.

Hyperspectral Image Segmentation via Frequency-Based Similarity for Mixed Noise Estimation

Accurate approximation of the signal-independent (SI) and signal-dependent (SD) mixed noise from hyperspectral (HS) images is a critical task for many image processing applications where the detection of homogeneous regions plays a key role. Most of the conventional methods empirically divide images into rectangular blocks and then select the homogeneous ones, but it might result in erroneous homogeneity detection, especially for highly textured HS images. To address this challenge, a superpixel segmentation algorithm is proposed in this paper, which can decompose a noisy HS image into patches that adhere to the local structures and hence persist in homogeneous characteristic. A novel spectral similarity measure is defined in the frequency domain to make the superpixel segmentation algorithm more robust to the mixed noise. Combined with an improved scatter-plot-based homogeneous superpixel selection and a multiple linear regression-based noise parameter calculation, our method can accurately estimate SD and SI noise variances from HS images with different noise conditions and various image complexities. We evaluate the proposed method with both synthetic and real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images. Experimental results demonstrate that the proposed noise estimation method outperforms the state-of-the-art methods.

Weed detection in soybean crops using ConvNets

Weeds are undesirable plants that grow in agricultural crops, such as soybean crops, competing for elements such as sunlight and water, causing losses to crop yields. The objective of this work was to use Convolutional Neural Networks (ConvNets or CNNs) to perform weed detection in soybean crop images and classify these weeds among grass and broadleaf, aiming to apply the specific herbicide to weed detected. For this purpose, a soybean plantation was carried out in Campo Grande, Mato Grosso do Sul, Brazil, and the Phantom DJI 3 Professional drone was used to capture a large number of crop images. With these photographs, an image database was created containing over fifteen thousand images of the soil, soybean, broadleaf and grass weeds. The Convolutional Neural Networks used in this work represent a Deep Learning architecture that has achieved remarkable success in image recognition. For the training of Neural Network the CaffeNet architecture was used. Available in Caffe software, it consists of a replication of the well known AlexNet, network which won the ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012). A software was also developed, Pynovisão, which through the use of the superpixel segmentation algorithm SLIC, was used to build a robust image dataset and classify images using the model trained by Caffe software. In order to compare the results of ConvNets, Support Vector Machines, AdaBoost and Random Forests were used in conjunction with a collection of shape, color and texture feature extraction techniques. As a result, this work achieved above 98% accuracy using ConvNets in the detection of broadleaf and grass weeds in relation to soil and soybean, with an accuracy average between all images above 99%.

A Novel Trail Detection and Scene Understanding Framework for a Quadrotor UAV With Monocular Vision

Vision-based trail detection and autonomous scene understanding play a key role for unmanned aerial vehicles (UAVs) working in complex outdoor environments such as isolated disaster sites. This paper investigates the problems associated with trail detection and tracking, as well as autonomous scene understanding using a quadrotor UAV. A framework that integrates support vector machine-based trail detection with a trail tracker is proposed to accomplish trail direction estimation and tracking at a low cost of computation and in real time. To accurately perform online parameter estimation, a performance test is designed and implemented to evaluate the accuracy. Moreover, the simple linear iterative clustering superpixel segmentation algorithm is utilized in the proposed system framework to guarantee the scene segmentation accuracy. Visual detection for significant objects or people is implemented by using single shot multibox detector algorithm. A series of experiments are conducted by using a quadrotor platform DJI M100 and experimental results show the validity and practicality of the proposed approach.

Cellular matrix model for parallel combinatorial optimization algorithms in Euclidean plane

We propose a parallel computation model, called cellular matrix model (CMM), to address large-size Euclidean graph matching problems in the plane. The parallel computation takes place by partitioning the plane into a regular grid of cells, each cell being affected to a single processor. Each processor operates on local data, starting from its cell location and extending its search to the neighborhood cells in a spiral search way. In order to deal with large-size problems, memory size and processor number are fixed as O(N), where N is the problem size. Then one key point is that closest point searching in the plane is performed in O(1) expected time for uniform or bounded distribution, for each processor independently. We define a generic loop that models the parallel projection between graphs and their matching, as executed by the many cells at a given level of computation granularity. To illustrate its efficacy and versatility, we apply the CMM, on GPU platforms, to two problems in image processing: superpixel segmentation and stereo matching energy minimization. Firstly, we propose an extended version of the well-known SLIC superpixel segmentation algorithm, which we call SPASM algorithm, by using a parallel 2D self-organizing map instead of k-means algorithm. Secondly, we investigate the idea of distributed variable neighborhood search, and propose a parallel search heuristic, called distributed local search (DLS), for global energy minimization of stereo matching problem. We evaluate the approach with regards to the state-of-the-art graph cut and belief propagation algorithms. For each problem, we argue that the parallel GPU implementation provides new competitive quality/time trade-offs, with substantial acceleration factors as the problem size increases.

Local Competition-Based Superpixel Segmentation Algorithm in Remote Sensing.

Remote sensing technologies have been widely applied in urban environments’ monitoring, synthesis and modeling. Incorporating spatial information in perceptually coherent regions, superpixel-based approaches can effectively eliminate the “salt and pepper” phenomenon which is common in pixel-wise approaches. Compared with fixed-size windows, superpixels have adaptive sizes and shapes for different spatial structures. Moreover, superpixel-based algorithms can significantly improve computational efficiency owing to the greatly reduced number of image primitives. Hence, the superpixel algorithm, as a preprocessing technique, is more and more popularly used in remote sensing and many other fields. In this paper, we propose a superpixel segmentation algorithm called Superpixel Segmentation with Local Competition (SSLC), which utilizes a local competition mechanism to construct energy terms and label pixels. The local competition mechanism leads to energy terms locality and relativity, and thus, the proposed algorithm is less sensitive to the diversity of image content and scene layout. Consequently, SSLC could achieve consistent performance in different image regions. In addition, the Probability Density Function (PDF), which is estimated by Kernel Density Estimation (KDE) with the Gaussian kernel, is introduced to describe the color distribution of superpixels as a more sophisticated and accurate measure. To reduce computational complexity, a boundary optimization framework is introduced to only handle boundary pixels instead of the whole image. We conduct experiments to benchmark the proposed algorithm with the other state-of-the-art ones on the Berkeley Segmentation Dataset (BSD) and remote sensing images. Results demonstrate that the SSLC algorithm yields the best overall performance, while the computation time-efficiency is still competitive.

Linear Spectral Clustering Superpixel.

In this paper, we present a superpixel segmentation algorithm called linear spectral clustering (LSC), which is capable of producing superpixels with both high boundary adherence and visual compactness for natural images with low computational costs. In LSC, a normalized cuts-based formulation of image segmentation is adopted using a distance metric that measures both the color similarity and the space proximity between image pixels. However, rather than directly using the traditional eigen-based algorithm, we approximate the similarity metric through a deliberately designed kernel function such that pixel values can be explicitly mapped to a high-dimensional feature space. We then apply the conclusion that by appropriately weighting each point in this feature space, the objective functions of the weighted K-means and the normalized cuts share the same optimum points. Consequently, it is possible to optimize the cost function of the normalized cuts by iteratively applying simple K-means clustering in the proposed feature space. LSC possesses linear computational complexity and high memory efficiency, since it avoids both the decomposition of the affinity matrix and the generation of the large kernel matrix. By utilizing the underlying mathematical equivalence between the two types of seemingly different methods, LSC successfully preserves global image structures through efficient local operations. Experimental results show that LSC performs as well as or even better than the state-of-the-art superpixel segmentation algorithms in terms of several commonly used evaluation metrics in image segmentation. The applicability of LSC is further demonstrated in two related computer vision tasks.

Region-Based Relaxed Multiple Kernel Collaborative Representation for Hyperspectral Image Classification

This paper presents a region-based relaxed multiple kernel collaborative representation method for the spatial-spectral classification of hyperspectral images. The proposed method consists of three steps. In the first step, a multiscale method achieved by extending a superpixel segmentation algorithm is designed to capture the spatial-spectral information of hyperspectral images. For each scale, a hyperspectral image can be segmented into several nonoverlapping spectrally similar regions that consist of some spatially adjacent pixels. In the second step, two criteria (i.e., the first two moments) are computed within the regions of each scale to generate the corresponding spatial features. In the final step, a relaxed multiple kernel technique is proposed to fuse the obtained spatial multiscale features and original spectral features in the framework of column generation kernel collaborative representation classification. Experimental results obtained from two real hyperspectral images demonstrate the effectiveness of the proposed method as compared with some popular spatial-spectral techniques.

GLSC: LSC superpixels at over 130 FPS

Superpixel has been successfully applied in various computer vision tasks, and many algorithms have been proposed to generate superpixel map. Recently, a superpixel algorithm called “superpixel segmentation using linear spectral clustering” (LSC) has been proposed, and it performs equally well or better than state-of-the art superpixel segmentation algorithms in terms of several commonly used evaluation metrics in superpixel segmentation. Although LSC is of linear complexity, its original implementation runs in few hundreds of milliseconds for images with resolution of 481 × 321 stated by the authors, which is a limitation for some real-time applications such as visual tracking which may needs, for instance, 30 FPS for standard image resolution (e.g., 480 × 320, 640 × 480, 1280 × 720 and 1920 × 1080). Instead of inventing new algorithms with lower complexity than LSC, we will explore LSC to modify its structure and make it suitable to be implemented by parallel technique. The modified LSC algorithm is implemented in CUDA and tested on several NVIDIA graphics processing unit. Our implementation of the proposed modified LSC algorithm achieves speedups of up to 80× from the original sequential implementation, and the quality, measured by two commonly used evaluation metrics, of our implementation keeps being similar to the original one. The source code will be made publicly available.

A Fast Superpixel Segmentation Algorithm for PolSAR Images Based on Edge Refinement and Revised Wishart Distance.

The superpixel segmentation algorithm, as a preprocessing technique, should show good performance in fast segmentation speed, accurate boundary adherence and homogeneous regularity. A fast superpixel segmentation algorithm by iterative edge refinement (IER) works well on optical images. However, it may generate poor superpixels for Polarimetric synthetic aperture radar (PolSAR) images due to the influence of strong speckle noise and many small-sized or slim regions. To solve these problems, we utilized a fast revised Wishart distance instead of Euclidean distance in the local relabeling of unstable pixels, and initialized unstable pixels as all the pixels substituted for the initial grid edge pixels in the initialization step. Then, postprocessing with the dissimilarity measure is employed to remove the generated small isolated regions as well as to preserve strong point targets. Finally, the superiority of the proposed algorithm is validated with extensive experiments on four simulated and two real-world PolSAR images from Experimental Synthetic Aperture Radar (ESAR) and Airborne Synthetic Aperture Radar (AirSAR) data sets, which demonstrate that the proposed method shows better performance with respect to several commonly used evaluation measures, even with about nine times higher computational efficiency, as well as fine boundary adherence and strong point targets preservation, compared with three state-of-the-art methods.

Superpixel-guided multiscale kernel collaborative representation for hyperspectral image classification

ABSTRACTThis article presents a superpixel-guided multiscale kernel collaborative representation method for robust classification of hyperspectral images. This novel method first exploits the spatial multiscale information of hyperspectral images by extending a superpixel segmentation algorithm, and then proposes a spatial-spectral information fusion technique to encode the spatial multiscale similarities and the spectral similarities between the pixels in the framework of kernel collaborative representation classification. The advantages of it mainly consist in (1) avoiding choosing empirical parameters in the spatial feature extraction process of superpixels and (2) enhanced classification accuracy as compared to traditional spatial-spectral kernel techniques. Experimental results with two widely used hyperspectral images demonstrate the effectiveness of the proposed method.

Acceleration of simple linear iterative clustering using early candidate cluster exclusion

For superpixel segmentation that partitions an image into multiple homogeneous regions, simple linear iterative clustering (SLIC) has been widely used as a preprocessing step in various image processing and computer vision applications due to its outstanding performance in terms of speed and accuracy. However, determining the segment where each pixel belongs still requires tedious, repeated computation to measure the distance between the pixel and every candidate segment. In this paper, by applying the Cauchy–Schwarz inequality, we derive an approximate distance metric and a simple condition using the metric to get rid of unnecessary computational operations from the cluster inspection procedure. Candidate clusters can be excluded if they satisfy a condition requiring much less computation than the normal cluster inspection involving the distance measure. We refer to the condition as early candidate cluster exclusion (ECCE). To maximize the success rate of ECCE, we propose a method to predict the best cluster for each pixel. In the experimental results, we analyzed various properties of the proposed approximate distance metric, including the required computational operations and the success rate of ECCE. We confirmed that the proposed superpixel segmentation algorithm improves SLIC’s computational efficiency by 219 %, on average, without any degradation in segmentation accuracy.

Subsampling-based acceleration of simple linear iterative clustering for superpixel segmentation

Simple linear iterative clustering (SLIC) that partitions an image into multiple homogeneous regions, superpixels, has been widely used as a preprocessing step in various image processing and computer vision applications due to its outstanding performance in terms of speed and accuracy. However, determining a segment that each pixel belongs to still requires tedious, iterative computation, which hinders real-time execution of SLIC. In this paper, we propose an accelerated SLIC superpixel segmentation algorithm where the number of candidate segments for each pixel is reduced effectively by exploiting high spatial redundancy within natural images. Because all candidate segments should be inspected in order to choose the best one, candidate reduction significantly improves computational efficiency. Various characteristics of the proposed acceleration algorithm are investigated. The experimental results confirmed that the proposed superpixel segmentation algorithm runs up to about five times as fast as SLIC while producing almost the same superpixel segmentation performance, sometimes better than SLIC, with respect to under-segmentation error and boundary recall.

Uniformity-Based Superpixel Segmentation of Hyperspectral Images

Superpixel segmentation algorithms attempt to group contiguous image pixels which are in homogeneous regions into segments (superpixels). Superpixel segmentation maps have proven successful in improving the performance of unmixing algorithms on hyperspectral images. For hyperspectral images (HSIs), segment members must contain spectrally similar pixels, a requirement we refer to as segment uniformity. Existing superpixel segmentation algorithms which have been applied to HSIs provide no guarantees on the uniformity inside segments. In the absence of such guarantees, the only viable option is to make the segments small enough that uniformity is always ensured; this leads to an oversegmentation of the image. An accurate uniformity measure would lead to a more accurate segmentation. We propose a graph-based agglomerative approach that enforces segment uniformity by setting a threshold for maximum variability inside segments. The threshold is computed by a statistical analysis of the within-class and between-class spectral divergences of several mineral families of interest. We show that the proposed algorithm can be used to generate parsimonious segmentations and facilitate the computation of accurate mineralogical summaries for several simulated and real HSIs of terrestrial and planetary geological surfaces.

Image Segmentation by Using Linear Spectral Clustering

In this paper a super pixel segmentation algorithm called Linear Spectral Clustering (LSC), which produces compact and uniform super pixels with low computational costs. Basically, a normalized cuts formulation of the super pixel segmentation is adopted based on a similarity metric that measures the colour similarity and space proximity between image pixels. However, instead of using the traditional Eigen-based algorithm, we approximate the similarity metric using a kernel function leading to an explicitly mapping of pixel values and coordinates into a high dimensional feature space. We revisit the conclusion that by appropriately weighting each point in this feature space, the objective functions of weighted K-means and normalized cuts share the same optimum point. As such, it is possible to optimize the cost function of normalized cuts by iteratively applying simple K-means clustering in the proposed feature space. LSC is of linear computational complexity and high memory efficiency and is able to preserve global properties of images. Experimental results show that LSC performs equally well or better than state of the art super pixel segmentation algorithms in terms of several commonly used evaluation metrics in image segmentation.