Superpixel Features Research Articles

Scale‐adaptive superpixels for medical images

BackgroundSuperpixel segmentation is a powerful preprocessing tool to reduce the complexity of image processing. Traditionally, size uniformity is one of the significant features of superpixels. However, in medical images, in which subjects scale varies greatly and background areas are often flat, size uniformity rarely conforms to the varying content. To obtain the fewest superpixels with retaining important details, the size of superpixel should be chosen carefully.MethodsWe propose a scale‐adaptive superpixel algorithm relaxing the size‐uniformity criterion for medical images, especially pathological images. A new path‐based distance measure and superpixel region growing schema allow our algorithm to generate superpixels with different scales according to the complexity of image content, that is smaller (larger) superpixels in color‐riching areas (flat areas).ResultsThe proposed superpixel algorithm can generate superpixels with boundary adherence, insensitive to noise, and with extremely big sizes and extremely small sizes on one image. The number of superpixels is much smaller than size‐uniformly superpixel algorithms while retaining more details of images.ConclusionWith the proposed algorithm, the choice of superpixel size is automatic, which frees the user from the predicament of setting suitable superpixel size for a given application. The results on the nuclear dataset show that the proposed superpixel algorithm superior to the respective state‐of‐the‐art algorithms on both quantitative and quantitative comparisons.

Efficient kernel fuzzy clustering via random Fourier superpixel and graph prior for color image segmentation

The kernel fuzzy clustering algorithms can explore the non-linear relations of pixels in an image. However, most of kernel-based methods are computationally expensive for color image segmentation and neglect the inherent locality information in images. To alleviate these limitations, this paper proposes a novel kernel fuzzy clustering framework for fast color image segmentation. More specifically, we first design a new superpixel generation method that uses random Fourier maps to approximate Gaussian kernels and explicitly represent high-dimensional features of pixels. Clustering superpixels instead of large-sized pixels speeds up the segmentation of a color image significantly. More importantly, the features of superpixels used by fuzzy clustering are also calculated in the approximated kernel space and the local relationships between superpixels are depicted as a graph prior and appended into the objective function of fuzzy clustering as a Kullback–Leibler divergence term. This results in a new fuzzy clustering model that can further improve the accuracy of the image segmentation. Experiments on synthetic and real-world color image datasets verify the superiority and high efficiency of the proposed approach.

A Superpixel Spatial Intuitionistic Fuzzy C-Means Clustering Algorithm for Unsupervised Classification of High Spatial Resolution Remote Sensing Images

This paper proposes a superpixel spatial intuitionistic fuzzy C-means (SSIFCM) clustering algorithm to address the problems of misclassification, salt and pepper noise, and classification uncertainty arising in the pixel-level unsupervised classification of high spatial resolution remote sensing (HSRRS) images. To reduce information redundancy and ensure noise immunity and image detail preservation, we first use a superpixel segmentation to obtain the local spatial information of the HSRRS image. Secondly, based on the bias-corrected fuzzy C-means (BCFCM) clustering algorithm, the superpixel spatial intuitionistic fuzzy membership matrix is constructed by counting an intuitionistic fuzzy set and spatial function. Finally, to minimize the classification uncertainty, the local relation between adjacent superpixels is used to obtain the classification results according to the spectral features of superpixels. Four HSRRS images of different scenes in the aerial image dataset (AID) are selected to analyze the classification performance, and fifteen main existing unsupervised classification algorithms are used to make inter-comparisons with the proposed SSIFCM algorithm. The results show that the overall accuracy and Kappa coefficients obtained by the proposed SSIFCM algorithm are the best within the inter-comparison of fifteen algorithms, which indicates that the SSIFCM algorithm can effectively improve the classification accuracy of HSRRS image.

AF2GNN: Graph convolution with adaptive filters and aggregator fusion for hyperspectral image classification

Hyperspectral image classification (HSIC) is essential in remote sensing image analysis. Applying a graph neural network (GNN) to hyperspectral image (HSI) classification has attracted increasing attention. However, the available GNNs for HSIC only adopt a kind of graph filter and an aggregator, which cannot well deal with the problems of land cover discrimination, noise impaction, and spatial feature learning. To overcome these problems, a graph convolution with adaptive filters and aggregator fusion (AF2GNN) is developed for HSIC. To reduce the number of graph nodes, a superpixel segment algorithm is employed to refine the local spatial features of the HSI. A two-layer 1D CNN is proposed to transform the spectral features of superpixels. In addition, a linear function is designed to combine the different graph filters, with which the graph filter can be adaptively determined by training different weight matrices. Moreover, degree-scalers are defined to combine the multiple filters and present the graph structure. Finally, the AF2GNN is proposed to realize the adaptive filters and aggregator fusion mechanism within a single network. In the proposed network, a softMax function is utilized for graph feature interpretation and pixel-label prediction. Compared with state-of-the-art methods, the proposed method achieves superior experimental results.

Deep integrated pipeline of segmentation guided classification of breast cancer from ultrasound images

Breast cancer has become a symbol of tremendous concern in the modern world, as it is one of the major causes of cancer mortality worldwide. In this regard, breast ultrasonography images are frequently utilized by doctors to diagnose breast cancer at an early stage. However, the complex artifacts and heavily noised breast ultrasonography images make diagnosis a great challenge. Furthermore, the ever-increasing number of patients being screened for breast cancer necessitates the use of automated end-to-end technology for highly accurate diagnosis at a low cost and in a short time. In this concern, to develop an end-to-end integrated pipeline for breast ultrasonography image classification, we conducted an exhaustive analysis of image preprocessing methods such as K Means++ and SLIC, as well as four transfer learning models such as VGG16, VGG19, DenseNet121, and ResNet50. With a Dice-coefficient score of 63.4 in the segmentation stage and accuracy and an F1-Score (Benign) of 73.72 percent and 78.92 percent in the classification stage, the combination of SLIC, UNET, and VGG16 outperformed all other integrated combinations. Finally, we have proposed an end to end integrated automated pipelining framework which includes preprocessing with SLIC to capture super-pixel features from the complex artifact of ultrasonography images, complementing semantic segmentation with modified U-Net, leading to breast tumor classification using a transfer learning approach with a pre-trained VGG16 and a densely connected neural network. The proposed automated pipeline can be effectively implemented to assist medical practitioners in making more accurate and timely diagnoses of breast cancer.

DGCC-EB: Deep Global Context Construction With an Enabled Boundary for Land Use Mapping of CSMA

Land use mapping (LUM) of a coal mining subsidence area (CMSA) is a significant task. The application of convolutional neural networks (CNNs) has become prevalent in LUM, which can achieve promising performances. However, CNNs cannot process irregular data; as a result, the boundary information is overlooked. The graph convolutional network (GCN) flexibly operates with irregular regions to capture the contextual relations among neighbors. However, the global context is not considered in the GCN. In this paper, we develop the deep global context construction with enabled boundary (DGCC-EB) for the LUM of the CMSA. An original Google Earth image is partitioned into nonoverlapping processing units. The DGCC-EB extracts preliminary features from the processing unit that are further divided into nonoverlapping superpixels with irregular edges. The superpixel features are generated and then embedded into the GCN and vision transformer (ViT). In the GCN, the graph convolution is applied to superpixel features; therefore, the boundary information of objects can be preserved. In the ViT, the multihead attention blocks and positional encoding build the global context among the superpixel features. The feature constraint is calculated to fuse the advantages of the features extracted from the GCN and ViT. To improve the LUM accuracy, the cross-entropy (CE) loss is calculated. The DGCC-EB integrates all modules into a whole end-to-end framework and is then optimized by a customized algorithm. The results of case studies show that the proposed DGCC-EB obtained acceptable OA (89.06%/88.68%) and Kappa (0.86/0.87) values for Shouzhou city and Zezhou city, respectively.

Polarimetric SAR Image Classification Based on Feature Enhanced Superpixel Hypergraph Neural Network

Synthetic aperture radar (SAR) images can capture abundant spatial and polarimetric information of land cover objects, and thus polarimetric SAR (PolSAR) image classification has been developed for various applications. Combining the advantages of spatial and polarimetric information simultaneously is of great importance for PolSAR image classification. In this paper, a feature enhanced superpixel hypergraph neural network (FESHNN) is proposed for PolSAR image classification, which aims to take full advantage of spatial features and polarimetric features from PolSAR images. In the proposed model, superpixel hypergraph neural network is constructed for feature representation of superpixels, which aims to obtain spatial correlation and polarimetric correlation in a hypergraph. Then, a feature enhancement module is employed to refine the local features of pixels and the spatial features of superpixels, which aims to enhance the discrimination of feature representation. Experimental results on three PolSAR datasets demonstrate that the proposed method yields superior classification performance compared with other related approaches.

Hyperspectral Image Classification Based on Superpixel Feature Subdivision and Adaptive Graph Structure

The graph-based hyperspectral image classification (HSIC) method has attracted wide attention because it can extract information with a non-Euclidean structure. Many graph-based HSIC works have achieved good results, but unresolved technical issues remain. For example, many graph nodes lead to high computational costs, and the mining of non-Euclidean structures is not sufficient. To solve these problems, we propose a graph attention network with an adaptive graph structure mining (GAT-AGSM) approach. Specifically, we first propose an HSIC framework with a superpixel feature subdivision (SFS) mechanism. In this framework, the number of nodes in the graph structure is reduced by using superpixel segmentation algorithms, and the SFS mechanism is designed to generate finer classification results. Second, we design the spatial–spectral attention layer with an adaptive graph structure mining (AGSM) mechanism for the graph attention network. The spatial–spectral attention layer can filter information in both spatial and spectral dimensions. The AGSM mechanism requires less manual intervention to dynamically generate non-Euclidean graph structures that better aggregate information. We conduct excessive experiments to compare the proposed GAT-AGSM with seven nongraph methods and three graph-based methods on widely used datasets. On the Indian Pines, Pavia University, and Salinas datasets, compared to the comparison method, the overall accuracy of GAT-AGSM is improved by at least 4.26%, 2.59%, and 1.41%, respectively. Experimental results show that GAT-AGSM has the best performance compared to the baselines in terms of various metrics.

Semi-Supervised Superpixel-Based Multi-Feature Graph Learning for Hyperspectral Image Data

Graphs naturally lend themselves to model the complexities of Hyperspectral Image (HSI) data as well as to serve as semi-supervised classifiers by propagating given labels among nearest neighbours. In this work, we present a novel framework for the classification of HSI data in light of a very limited amount of labelled data, inspired by multi-view graph learning and graph signal processing. Given an a priori superpixel-segmented hyperspectral image, we seek a robust and efficient graph construction and label propagation method to conduct semi-supervised learning (SSL). Since the graph is paramount to the success of the subsequent classification task, particularly in light of the intrinsic complexity of HSI data, we consider the problem of finding the optimal graph to model such data. Our contribution is two-fold: firstly, we propose a multi-stage edge-efficient semi-supervised graph learning framework for HSI data which exploits given label information through pseudo-label features embedded in the graph construction. Secondly, we examine and enhance the contribution of multiple superpixel features embedded in the graph on the basis of pseudo-labels in an extension of the previous framework, which is less reliant on excessive parameter tuning. Ultimately, we demonstrate the superiority of our approaches in comparison with state-of-the-art methods through extensive numerical experiments.

Extended set of superpixel features

Superpixel-based image processing and analysis methods usually use a small set of superpixel features. Expanding the description of superpixels can improve the quality of processing algorithms. In the paper, a set of 25 basic superpixel features of shape, intensity, geometry, and location is proposed. The features meet the requirements of low computational complexity in the process of image superpixel segmentation and sufficiency for solving a wide class of application tasks. Applying the set, we present a modification of the well-known approach to the superpixel generation. It consists of fast primary superpixel segmentation of the image with a strict homogeneity predicate, which provides superpixels preserving the intensity information of the original image with high accuracy, and the subsequent enlargement of the superpixels with softer homogeneity predicates. The experiments show that the approach can significantly reduce the number of image elements, which helps to reduce the complexity of processing algorithms, meanwhile the expanded superpixels more accurately correspond to the image objects.

Unsupervised Cosegmentation Model based on Saliency Detection and Optimized Hue Saturation Value Features of Superpixels

Unsupervised Cosegmentation Model based on Saliency Detection and Optimized Hue Saturation Value Features of Superpixels

Multiscale Adjacent Superpixel-Based Extended Multi-Attribute Profiles Embedded Multiple Kernel Learning Method for Hyperspectral Classification

In this paper, superpixel features and extended multi-attribute profiles (EMAPs) are embedded in a multiple kernel learning framework to simultaneously exploit the local and multiscale information in both spatial and spectral dimensions for hyperspectral image (HSI) classification. First, the original HSI is reduced to three principal components in the spectral domain using principal component analysis (PCA). Then, a fast and efficient segmentation algorithm named simple linear iterative clustering is utilized to segment the principal components into a certain number of superpixels. By setting different numbers of superpixels, a set of multiscale homogenous regional features is extracted. Based on those extracted superpixels and their first-order adjacent superpixels, EMAPs with multimodal features are extracted and embedded into the multiple kernel framework to generate different spatial and spectral kernels. Finally, a PCA-based kernel learning algorithm is used to learn an optimal kernel that contains multiscale and multimodal information. The experimental results on two well-known datasets validate the effectiveness and efficiency of the proposed method compared with several state-of-the-art HSI classifiers.

Place recognition with deep superpixel features for brain-inspired navigation.

Navigation in primates is generally supported by cognitive maps. Such a map endows an animal with navigational planning capabilities. Numerous methods have been proposed to mimic these natural navigation capabilities in artificial systems. Based on self-navigation and learning strategies in animals, we propose in this work a place recognition strategy for brain-inspired navigation. First, a place recognition algorithm structure based on convolutional neural networks (CNNs) is introduced, which can be applied in the field of intelligent navigation. Second, sufficient images are captured at each landmark and then stored as a reference image library. Simple linear iterative clustering (SLIC) is used to segment each image into superpixels with multi-scale viewpoint-invariant landmarks. Third, highly representative appearance-independent features are extracted from these landmarks through CNNs. In addition, spatial pyramid pooling (SPP) layers are introduced to generate a fixed-length CNN representation, regardless of the image size. This representation boosts the quality of the extracted landmark features. The proposed SLIC-SPP-CNN place recognition algorithm is evaluated on one collected dataset and two public datasets with viewpoint and appearance variations.

Weakly-supervised action localization based on seed superpixels

In this paper, we present action localization based on weak supervision with seed superpixels. In order to benefit from the superpixel segmentation and to learn a priori knowledge we select the seed superpixels from the action and non-action areas of few video frames of an action sequence equally. We compute correlation, joint entropy and joint histogram as the features of the video frame superpixels based on the optical flow magnitudes and intensity information. An SVM is trained with the action and non-action seed superpixels features and is used to classify the video frame superpixels as action and non-action. The superpixels classified as action provide the action localization. The localized action superpixels are used to recognize the action class by the Dendrogram-SVM based on the already extracted features. We evaluate the performance of the proposed approach for action localization and recognition using UCF sports and UCF-101 actions datasets, which demonstrates that the seed superpixels provide effective action localization and in turn facilitates to recognize the action class.

End-to-end trainable network for superpixel and image segmentation

Image segmentation and superpixel generation have been studied for many years, and they are still active research topics in computer vision. Although many advanced computer vision algorithms have been used for image segmentation and superpixel generation, there is no end-to-end trainable algorithm that generates superpixels and segment images simultaneously. We propose an end-to-end trainable network to solve this problem. We train a differentiable clustering algorithm module to produce accurate superpixels. Based on the generated superpixels, the superpixel pooling operation is performed to obtain superpixel features, and then we calculate the similarity of two adjacent superpixels. If the similarity is greater than the preset threshold, we merge the two superpixels. Finally, we get the segmented image. We conduct our experiments in the BSDS500 dataset and get good results.

No-reference stereoscopic images quality assessment method based on monocular superpixel visual features and binocular visual features

No-reference quality assessment of images has received considerable attention. However, the accuracy of such assessment remains questionable because of its weak biological basis. In this paper, we propose a novel quality assessment model based on the superpixel index and biological binocular mechanisms. The technical contributions of our model are the introduction of local monocular superpixel features and three global binocular visual features. We utilize monocular superpixel segmentation to extract two types of entropies as the local visual features for accurate quality-aware feature extraction. In addition, natural scene statistics features are extracted from the binocular visual information to complement the local monocular features and quantify the naturalness of the stereoscopic images. Finally, a regression model is learned to evaluate the quality of the stereoscopic images. Experimental results from three popular databases demonstrate that the proposed model has a more reliable performance than earlier models in terms of prediction accuracy and generalizability.

A semi-supervised recurrent neural network for video salient object detection

A semi-supervised, one-dimensional recurrent neural network (RNN) approach called RVS has been proposed in this paper for video salient object detection. The proposed RVS approach involves the processing of each frame independently without explicitly considering temporal information. The RNN is trained using one-dimensional superpixel features to classify the salient object regions into salient foreground and non-salient background superpixels. Deep learning algorithms generally exhibit heavy dependence on training data size and often take extremely long time for training. On the contrary, the proposed RVS approach involves the training of an RNN using a small data which results in significant reduction in training time. The RVS approach has been extensively evaluated and its results are compared with those of several state-of-the-art methods using the public-domain VideoSeg, SegTrack v1 and SegTrack v2 benchmark video datasets. Further, the RVS approach has been tested using the authors’ own video dataset and the complex DAVIS and video object segmentation datasets to evaluate the impact of motion and blur on its performance. The RVS approach delivers results superior to those of several approaches that strongly rely upon spatio-temporal features in detecting the salient objects from the video sequences.

Texture based localization of a brain tumor from MR-images by using a machine learning approach

In this paper, a machine learning approach was used for brain tumour localization on FLAIR scans of magnetic resonance images (MRI). The multi-modal brain images dataset (BraTs 2012) was used, that is a skull stripped and co-registered. In order to remove the noise, bilateral filtering is applied and then texton-map images are created by using the Gabor filter bank. From the texton-map, the image is segmented out into superpixel and then the low-level features are extracted: the first order intensity statistical features and also calculates the histogram level of texton-map at each superpixel level. There is a significant contribution here that the low features are not too much significant for the localization of brain tumour from MR images, but we have to make them meaningful by integrating features with the texton-map images at the region level approach. Then these features which are provided later to classifier for the prediction of three classes: background, tumour and non-tumour region, and used the labels for computation of two different areas (i.e. complete tumour and non-tumour). A Leave-one-out (LOOCV) cross validation technique is applied and achieves the dice overlap score of 88% for the whole tumour area localization, which is similar to the declared score in MICCAI BraTS challenge. This brain tumour localization approach using the textonmap image based on superpixel features illustrates the equivalent performance with other contemporary techniques. Recently, medical hypothesis generation by using autonomous computer based systems in disease diagnosis have given the great contribution in medical diagnosis.

Superpixel-based principal component analysis for high resolution remote sensing image classification

In object-based image analysis (OBIA), it is often difficult to select the most useful features from a large number of segment-based information. The problem of choosing superpixel-based features is also very challenging. In order to solve this issue, this paper proposes a principal component analysis (PCA)-based method for superpixel-based classification of high resolution remote sensing imagery. This technique transforms the spectral features of superpixels, and the resulted feature variables are used to train a support vector machine classifier. Experiments based on 4 high resolution multispectral images indicated that although the performance is sensitive to the two parameters, the proposed method can increase classification accuracy effectively.

Superpixel-Driven Optimized Wishart Network for Fast PolSAR Image Classification Using Global ${k}$ -Means Algorithm

Limitation of optical remote sensing technology gave rise to synthetic aperture radar (SAR) imaging. SAR is a microwave imaging technique, which promises to have a long-range propagation characteristic allowing imaging under harsh weather conditions or in hostile lighting situation. This has opened up a domain of classification using polarimetric SAR (PolSAR) images. In this article, we propose a fast PolSAR image classification algorithm, which uses not only pixel-based feature but also spatial features around each pixel. This is achieved by introducing superpixel-driven optimized Wishart network. The first improvement suggested in this article is to take advantage of a fast global $k$ -means algorithm for obtaining optimal cluster centers within each class. It uses real-valued vector representation of PolSAR coherency matrix along with fast matrix inverse and determinant algorithms to reduce computational overhead. Our method then exploits the information of neighboring pixels by forming a superpixel so that even a noisy pixel may not be assigned a wrong class label. The proposed network uses dual-branch architecture to efficiently combine pixel and superpixel features. We concluded that our proposed method has better efficiency in terms of classification accuracy and computational overhead compared with other deep learning-based methods available in the literature.