Adjacent Superpixels Research Articles

A raster-based method for the hierarchical selection of river networks based on stream characteristics

Computer screens often constrain the level of detail and clarity of displays. High-density data require a predefined strategy to select significant features hierarchically to allow interactive data zooming. Although many methods are available for hierarchically selecting rivers from vector data, some approaches for raster data are better than others for maintaining accuracy when the original river data are in a raster format during generalization. In this study, a raster-based approach is proposed to allow hierarchical superpixel selection in river networks. Linear spectral clustering segmentation was applied to divide the original raster river networks into superpixels at multiple levels. A graph was constructed to organize the generated river network superpixels based on the distances between adjacent superpixels by considering the weights determined by the four types of rules. Finally, the total weight values were ranked, the river-network superpixels were selected according to their weights, and the redundant pixels at the river-network intersections were removed. Compared with the traditional vector selection method, the proposed superpixel river network selection method can effectively consider the characteristics of river width without artificial river grading and preserve the main structure and connectivity features during hierarchical mapping. Notably, the average geometry and density changes decreased by 15.8% and 5.1%, respectively.

SAR Image Classification Using Markov Random Fields with Deep Learning

Classification algorithms integrated with convolutional neural networks (CNN) display high accuracies in synthetic aperture radar (SAR) image classification. However, their consideration of spatial information is not comprehensive and effective, which causes poor performance in edges and complex regions. This paper proposes a Markov random field (MRF)-based algorithm for SAR image classification which fully considers the spatial constraints between superpixel regions. Firstly, the initialization of region labels is obtained by the CNN. Secondly, a probability field is constructed to improve the distribution of spatial relationships between adjacent superpixels. Thirdly, a novel region-level MRF is employed to classify the superpixels, which combines the intensity field and probability field in one framework. In our algorithm, the generation of superpixels reduces the misclassification at the pixel level, and region-level misclassification is rectified by the improvement of spatial description. Experimental results on simulated and real SAR images confirm the efficacy of the proposed algorithm for classification.

A Superpixel Spatial Intuitionistic Fuzzy C-Means Clustering Algorithm for Unsupervised Classification of High Spatial Resolution Remote Sensing Images

This paper proposes a superpixel spatial intuitionistic fuzzy C-means (SSIFCM) clustering algorithm to address the problems of misclassification, salt and pepper noise, and classification uncertainty arising in the pixel-level unsupervised classification of high spatial resolution remote sensing (HSRRS) images. To reduce information redundancy and ensure noise immunity and image detail preservation, we first use a superpixel segmentation to obtain the local spatial information of the HSRRS image. Secondly, based on the bias-corrected fuzzy C-means (BCFCM) clustering algorithm, the superpixel spatial intuitionistic fuzzy membership matrix is constructed by counting an intuitionistic fuzzy set and spatial function. Finally, to minimize the classification uncertainty, the local relation between adjacent superpixels is used to obtain the classification results according to the spectral features of superpixels. Four HSRRS images of different scenes in the aerial image dataset (AID) are selected to analyze the classification performance, and fifteen main existing unsupervised classification algorithms are used to make inter-comparisons with the proposed SSIFCM algorithm. The results show that the overall accuracy and Kappa coefficients obtained by the proposed SSIFCM algorithm are the best within the inter-comparison of fifteen algorithms, which indicates that the SSIFCM algorithm can effectively improve the classification accuracy of HSRRS image.

Fusion Information Multi-View Classification Method for Remote Sensing Cloud Detection

In recent years, many studies have been carried out to detect clouds on remote sensing images. Due to the complex terrain, the variety of clouds, the density, and content of clouds are various, and the current model has difficulty accurately detecting the cloud in the image. In our strategy, a multi-view data training set based on super pixel is constructed. View A uses multi-level network to extract the boundary, texture, and deep abstract feature of super pixels. View B is the statistical feature of the three channels of the image. Privilege information View P contains the cloud content of super pixels and the tag status of adjacent super pixels. Finally, we propose a cloud detection method for remote sensing image classification based on multi-view support vector machine (SVM). The proposed method is tested on images of different terrain and cloud distribution in GF-1_WHU and Cloud-38 remote sensing datasets. Visual performance and quantitative analysis show that the method has excellent cloud detection performance.

Infrared Image Superpixel Segmentation Based on Seed Strategy of Contour Encoding

Superpixel segmentation has become a crucial pre-processing tool to reduce computation in many computer vision applications. In this paper, a superpixel extraction algorithm based on a seed strategy of contour encoding (SSCE) for infrared images is presented, which can generate superpixels with high boundary adherence and compactness. Specifically, SSCE can solve the problem of superpixels being unable to self-adapt to the image content. First, a contour encoding map is obtained by ray scanning the binary edge map, which ensures that each connected domain belongs to the same homogeneous region. Second, according to the seed sampling strategy, each seed point can be extracted from the contour encoding map. The initial seed set, which is adaptively scattered based on the local structure, is capable of improving the capability of boundary adherence, especially for small regions. Finally, the initial superpixels limited by the image contour are generated by clustering and refined by merging similar adjacent superpixels in the region adjacency graph (RAG) to reduce redundant superpixels. Experimental results on a self-built infrared dataset and the public datasets BSD500 and 3Dircadb demonstrate the generalization ability in grayscale and medical images, and the superiority of the proposed method over several state-of-the-art methods in terms of accuracy and compactness.

Superpixel-Level Hybrid Discriminant Analysis for Hyperspectral Image Feature Extraction

For hyperspectral image (HSI) classification, it is an challenging problem to learn highly discriminative features, since the complex local/non-local spatial-spectral association is difficult to be accurately characterized. Focused on this issue, a novel superpixel-level hybrid discriminant analysis (SHDA) method is proposed, in this paper. Here, the SHDA method takes advantage of superpixel’s merit in characterizing spatial-spectral shape-adaptive structure and the powerful capability of discriminant analysis in enhancing class-separability to learn the feature representation. Moreover, the local/non-local spatial-spectral correlation information among/between superpixels is effectively excavated and fused in one framework to further improve the classification performance of features. This is achieved by first designing two specific discriminant analysis modules, i.e., superpixel-level local discriminant analysis (SLDA) and superpixel-level non-local discriminant analysis (SNDA). In the SLDA, adaptively weighted scatter matrices are defined to characterize the local spectral similarity within each superpixel and the discrepancy among adjacent superpixels. In the SNDA, superpixel-level graphs are built for capturing the non-local contextual information, where the weights of graphs are estimated based on the most similar and dissimilar superpixels. Then, the SLDA and the SNDA are effectively fused to construct the total intra/inter-superpixel scatter matrices. Finally, a joint projection transformation is obtained via solving a simple generalized eigenvalue problem. By this way, the HSI data can be projected from a high-dimensional data space into a low-dimensional feature space, where different classes of land-covers can be more accurately distinguished. Experimental results on three real hyperspectral data sets indicate that the proposed SHDA method outperforms several state-of-the-art techniques.

Multi-resolution Pyramid Enhanced Non-local Feature Extraction for Hyperspectral Classification

Hyperspectral image (HSI) is applied to accurately distinguish ground objects in many fields, owing to its abundant structural and spectral information. To fully mine the spectral-spatial information, feature extraction methods are employed to further improve the classification performance in numerous researches. Among them, integrating contextual information into pixels is an active topic and achieved significant performance. However, it is inevitable to come with unwanted and superfluous distracting contents in the process of introducing contextual information. This paper propose a multi-resolution pyramid enhanced non-local feature extraction (MRPEN) method by exploiting non-local information from different resolutions to alleviate the effects of noise contents. The feature extraction task is implemented through by first constructing a multi-resolution feature pyramid via performing spectral-spatial reduction and resize operation on the HSI. With the feature pyramid, three kinds of feature extraction manners (i.e., short-range, long-range, and non-local) are utilized to extract specific features for each layers. In particular, non-local feature is extracted by introducing adjacent superpixels from upper layer into our feature extraction operation. After this, the extracted features of each layer are concatenated together and fed on a sparse autoencoders. Finally, a decision rule is introduced to blend the prediction results. Experiments on three public HSI datasets and a Gaofen-5 satellite dataset demonstrate that our method outperforms several state-of-the-art classification approaches.

Multilevel Superpixel Structured Graph U-Nets for Hyperspectral Image Classification

Limited by the shape-fixed kernels, convolutional neural networks (CNNs) are usually difficult to model difform land covers in hyperspectral images (HSIs), leading to inadequate land use. Recently, benefiting from the ability to conduct shape-adaptive convolutions and model complex patterns in graph-structured data, graph convolutional networks (GCNs) have been applied to HSI classification. However, due to the massive computation in GCNs, HSI is usually pretreated into a graph based on a specific superpixel segmentation, which limits the modeling of spatial topologies to the same scale. To break this limitation, we propose a multilevel superpixel structured graph U-Net (MSSGU) to learn multiscale features on multilevel graphs. Specifically, we construct several hierarchical segmentations from fine to coarse by progressively merging adjacent superpixels and then convert them into multilevel graphs. Meanwhile, based on the merging relations between hierarchical superpixels, we establish the pooling and unpooling functions to transfer features from one graph to another, thereby enabling different-level graphs to collaborate in a single network. Different from concatenating different-scale features straightforwardly in the feature fusion stage, MSSGU fuses them in a coarse-to-fine progressive manner, which can generate subtler fusion features adaptive to the pixelwise classification task. Moreover, we use a CNN instead of GCN to extract and fuse the pixel-level features, which greatly reduces the computation. Such a hybrid U-Net can exploit features of HSIs from a multiscale hierarchical perspective, and its performance has been proven competitive with other deep-learning-based methods by extensive experiments on three benchmark datasets.

CNN‐combined graph residual network with multilevel feature fusion for hyperspectral image classification

The application of graph convolutional networks (GCN) in hyperspectral image (HSI) classification has become a promising method, thanks to its flexible convolution operation in any irregular image region. For the classification of HSI, GCN can extract more superpixel-level features with a topological structure, in comparison to the traditional convolutional neural networks (CNNs) using fixed square kernels distilling pixel-level features. To fully leverage the different levels of features, this study proposes a novel deep network referred to as a CNN-combined graph residual network ( C 2 GRN), which integrates the multilevel graph residual module and spectral-spatial features continuous learning module. During the extraction of topology information using the former module, HSI pixels are divided into superpixels and served as input nodes of the module to reduce the computational complexity and obtain the multilevel spatial relevance between adjacent superpixels. Besides, for the latter module, the spectral-spatial features are learnt continuously, which could obtain the finer pixel-level features. Finally, the captured spectral-spatial features of different levels are concatenated. This strategy could not only adequately utilize the correlation and difference of adjacent spatial but also obtain the finer and more valuable spectral-spatial information, which makes a significant boost in the HSI classification. Additionally, the experiment results demonstrate the superiority and availability of the C 2 GRN on three benchmark datasets of HSI, compared with the state-of-the-art methods for the classification of HSI.

Multiscale Weighted Adjacent Superpixel-Based Composite Kernel for Hyperspectral Image Classification

This paper presents a composite kernel method (MWASCK) based on multiscale weighted adjacent superpixels (ASs) to classify hyperspectral image (HSI). The MWASCK adequately exploits spatial-spectral features of weighted adjacent superpixels to guarantee that more accurate spectral features can be extracted. Firstly, we use a superpixel segmentation algorithm to divide HSI into multiple superpixels. Secondly, the similarities between each target superpixel and its ASs are calculated to construct the spatial features. Finally, a weighted AS-based composite kernel (WASCK) method for HSI classification is proposed. In order to avoid seeking for the optimal superpixel scale and fuse the multiscale spatial features, the MWASCK method uses multiscale weighted superpixel neighbor information. Experiments from two real HSIs indicate that superior performance of the WASCK and MWASCK methods compared with some popular classification methods.

Melanoma Detection Using Spatial and Spectral Analysis on Superpixel Graphs.

Melanoma is the most fatal type of skin cancer. Detection of melanoma from dermoscopic images in an early stage is critical for improving survival rates. Numerous image processing methods have been devised to discriminate between melanoma and benign skin lesions. Previous studies show that the detection performance depends significantly on the skin lesion image representations and features. In this work, we propose a melanoma detection approach that combines graph-theoretic representations with conventional dermoscopic image features to enhance the detection performance. Instead of using individual pixels of skin lesion images as nodes for complex graph representations, superpixels are generated from the skin lesion images and are then used as graph nodes in a superpixel graph. An edge of such a graph connects two adjacent superpixels where the edge weight is a function of the distance between feature descriptors of these superpixels. A graph signal can be defined by assigning to each graph node the output of some single-valued function of the associated superpixel descriptor. Features are extracted from weighted and unweighted graph models in the vertex domain at both local and global scales and in the spectral domain using the graph Fourier transform (GFT). Other features based on color, geometry and texture are extracted from the skin lesion images. Several conventional and ensemble classifiers have been trained and tested on different combinations from those features using two datasets of dermoscopic images from the International Skin Imaging Collaboration (ISIC) archive. The proposed system achieved an AUC of [Formula: see text], an accuracy of [Formula: see text], a specificity of [Formula: see text] and a sensitivity of [Formula: see text].

Multiscale Adjacent Superpixel-Based Extended Multi-Attribute Profiles Embedded Multiple Kernel Learning Method for Hyperspectral Classification

In this paper, superpixel features and extended multi-attribute profiles (EMAPs) are embedded in a multiple kernel learning framework to simultaneously exploit the local and multiscale information in both spatial and spectral dimensions for hyperspectral image (HSI) classification. First, the original HSI is reduced to three principal components in the spectral domain using principal component analysis (PCA). Then, a fast and efficient segmentation algorithm named simple linear iterative clustering is utilized to segment the principal components into a certain number of superpixels. By setting different numbers of superpixels, a set of multiscale homogenous regional features is extracted. Based on those extracted superpixels and their first-order adjacent superpixels, EMAPs with multimodal features are extracted and embedded into the multiple kernel framework to generate different spatial and spectral kernels. Finally, a PCA-based kernel learning algorithm is used to learn an optimal kernel that contains multiscale and multimodal information. The experimental results on two well-known datasets validate the effectiveness and efficiency of the proposed method compared with several state-of-the-art HSI classifiers.

Segmentation of natural images based on super pixel and graph merging

The task of natural image segmentation is one of the most researched topics of computer vision. There are mainly two principal approaches for the task, the statistical approach and the supervised approach. The proposed methodology segments natural images combining a set of statistical algorithms. First, the image is preprocessed to enhance the edges. Weighted average of the denoised image and its derivatives is the preprocessed output. Thereafter, an energy based super pixelation is applied to over segment the image. Finally, a connectivity graph is built where nodes correspond to super pixels and edges connect the adjacent super pixels. The adjacent super pixels are merged based on the confidence value defined in terms of their textural and colour similarity. Proposed methodology has been applied on the images of BSDS500 dataset. Performance of the proposed work has been compared with that of other works based on detected edge maps. Few works generate ultrametric contour maps (UCM). To compare the performance with those works, UCM is also generated by the proposed methodology. To do so images at multiple scales are considered. It is observed that the output of segmentation is better in case of the proposed methodology. Proposed methodology is much faster than others. Thus, makes it suitable for real time application in robot vision.

Affinity Matrix Learning Via Nonnegative Matrix Factorization for Hyperspectral Imagery Clustering

In this article, we integrate the spatial-spectral information of hyperspectral image (HSI) samples into nonnegative matrix factorization (NMF) for affinity matrix learning to address the issue of HSI clustering. This technique consists of three main components: 1) oversegmentation for computing the spectral-spatial affinity matrix; 2) NMF with the guidance of the obtained affinity matrix; and 3) density-based spectral clustering on the final affinity matrix. First, the HSI is oversegmented into superpixels via the entropy rate superpixel algorithm. The spectral-spatial affinity matrix is defined based on the class-consistency assumption of all the HSI samples in each superpixel and the similar HSI samples between adjacent superpixels. Second, to integrate the spectral-spatial information into NMF, the obtained affinity matrix is used to guide the iterative process of NMF. The spectral-spatial affinity matrix is then weighted by the affinity matrix in the obtained low-dimensional subspace to form the final affinity matrix. Third, density-based spectral clustering is applied to the final affinity matrix to obtain clustering maps. Experimental results on three public benchmark HSIs demonstrate that the proposed method is superior to the considered state-of-the-art baseline methods on both the computational cost and clustering accuracy.

Superpixel-Based Minimum Noise Fraction Feature Extraction for Classification of Hyperspectral Images

In this paper, a novel feature extraction technique called SuperMNF is proposed, which is an extension of the minimum noise fraction (MNF) transformation. In SuperMNF, each superpixel has its own transformation matrix and MNF transformation is performed on each superpixel individually. The basic idea behind the SuperMNF is that each superpixel contains its specific signal and noise covariance matrices which are different from the adjacent superpixels. The extracted features, owning spatial-spectral content and provided in the lower dimension, are classified by maximum likelihood classifier and support vector machines. Experiments that are conducted on two real hyperspectral images, named Indian Pines and Pavia University, demonstrate the efficiency of SuperMNF since it yielded more promising results than some other feature extraction methods (MNF, PCA, SuperPCA, KPCA, and MMP).

Image superpixel segmentation based on hierarchical multi-level LI-SLIC

In computer vision, superpixel segmentation has been widely used as a very important preprocessing to reduce the number of image primitives for subsequent image processing tasks. To improve the segmentation accuracy and the robustness to noise, a hierarchical multi-level segmentation framework is developed in this paper. First, original image is initially segmented by a local information based simple liner iterative clustering (LI-SLIC) method. Then, the initial generated superpixels are further segmented hierarchically by LI-SLIC to ensure that all pixels contained within each superpixel belong to a same object. Finally, to eliminate over-segmentation and reduce the number of superpixels, adjacent superpixels belonging to a same object are merged based on the probability distribution similarity. The proposed method does not require setting the seeds or number of the superpixels to be generated in advance, which can segment image into an appropriate number of superpixels without under- or over- segmentation automatically according to its content. Experiments are conducted on two public datasets Berkeley and 3Dircadb, and the results demonstrate that our method is more effective and accurate than many existing superpixel methods and shows a great advantage in dealing with images corrupted by various noises.

End-to-end trainable network for superpixel and image segmentation

Image segmentation and superpixel generation have been studied for many years, and they are still active research topics in computer vision. Although many advanced computer vision algorithms have been used for image segmentation and superpixel generation, there is no end-to-end trainable algorithm that generates superpixels and segment images simultaneously. We propose an end-to-end trainable network to solve this problem. We train a differentiable clustering algorithm module to produce accurate superpixels. Based on the generated superpixels, the superpixel pooling operation is performed to obtain superpixel features, and then we calculate the similarity of two adjacent superpixels. If the similarity is greater than the preset threshold, we merge the two superpixels. Finally, we get the segmented image. We conduct our experiments in the BSDS500 dataset and get good results.

Structure extraction in urbanized aerial images from a single view using a CNN-based approach

ABSTRACT High-Level Structure (HLS) extraction in aerial images consists of recognizing Three-Dimensional (3D) elements on human-made surfaces (objects, buildings, ground, etc.). There are several approaches to HLS extraction in aerial images. However, most of these approaches are based on processing two or more images captured from different camera views or on processing 3D data in the form of point clouds extracted from the camera images. In general, 3D point cloud and multiple view approaches have good performance for certain scenes with video sequences or image sequences, but they need sufficient parallax in order to guarantee accuracy. To address this problem, an alternative is to process a single image seeking to interpret areas of the images where the human-made structure may be observed, thus removing parallax dependency, but adding the challenge of having to interpret image ambiguities correctly. Motivated by the latter, this work presents the results of a novel method for HLS extraction from a single image. Our interest is the buildings structures extraction in urbanized aerial images. For that, our method has six steps. First, we use a new Convolutional Neural Network (CNN) architecture to recognize the labels (tree, roof, and floor) in the input image. Second, we use a CNN to predict the depth. Third, we divide the input image using a superpixel technique. Fourth, we segment the superpixels with its majority label. Fifth, we recognize the structures using a proposed connection analysis that connects the adjacent superpixels with equal labels (tree, roof, and floor). Finally, we use a geometric analysis with the depth prediction of the labels recognized that extracts the 3D shape of the building structure.

Non-parametric spatially constrained local prior for scene parsing on real-world data

Scene parsing aims to recognize the object category of every pixel in scene images, and it plays a central role in image content understanding and computer vision applications. However, accurate scene parsing from unconstrained real-world data is still a challenging task. In this paper, we present the non-parametric Spatially Constrained Local Prior (SCLP) for scene parsing on realistic data. For a given query image, the non-parametric SCLP is learnt by first retrieving a subset of most similar training images to the query image and then collecting prior information about object co-occurrence statistics between spatial image blocks and between adjacent superpixels from the retrieved subset. The SCLP is powerful in capturing both long- and short-range context about inter-object correlations in the query image and can be effectively integrated with traditional visual features to refine the classification results. Our experiments on the SIFT Flow and PASCAL-Context benchmark datasets show that the non-parametric SCLP used in conjunction with superpixel-level visual features achieves one of the top performance compared with state-of-the-art approaches.

A New Method for Image Superpixel Segmentation

Considering the problem that the existing superpixel methods are usually unable to set an appropriate number of generated superpixels automatically and unable to adhere to image boundaries effectively, a new superpixel method is proposed in this paper, which utilizes local information to perform multi-level simple linear iterative clustering to generate superpixels. First, original image is initially segmented by Simple Liner Iterative Clustering based on Local Information (LI-SLIC). Then, each superpixel is segmented iteratively until its color standard deviation is lower than a predefined threshold. Finally, the over-segmented superpixels are merged based on the color differences between adjacent superpixels. Experiments on Berkeley, Pascal VOC and 3Dircadb databases, as well as comparison with other methods indicate that the proposed method can adhere to image boundaries more accurately, and can prevent over- and under- segmentations more effectively.