Homogeneous Superpixels Research Articles

Building Detection in High-Resolution Remote Sensing Images by Enhancing Superpixel Segmentation and Classification Using Deep Learning Approaches

Accurate building detection is a critical task in urban development and digital city mapping. However, current building detection models for high-resolution remote sensing images are still facing challenges due to complex object characteristics and similarities in appearance. To address this issue, this paper proposes a novel algorithm for building detection based on in-depth feature extraction and classification of adaptive superpixel shredding. The proposed approach consists of four main steps: image segmentation into homogeneous superpixels using a modified Simple Linear Iterative Clustering (SLIC), in-depth feature extraction using an variational auto-encoder (VAE) scale on the superpixels for training and testing data collection, identification of four classes (buildings, roads, trees, and shadows) using extracted feature data as input to an Convolutional Neural Network (CNN), and extraction of building shapes through regional growth and morphological operations. The proposed approach offers more stability in identifying buildings with unclear boundaries, eliminating the requirement for extensive prior segmentation. It has been tested on two datasets of high-resolution aerial images from the New Zealand region, demonstrating superior accuracy compared to previous works with an average F1 score of 98.83%. The proposed approach shows potential for fast and accurate urban monitoring and city planning, particularly in urban areas.

Semantic-Aware Region Loss for Land-Cover Classification

Integrating superpixel segmentation into convolutional neural networks is known to be effective in enhancing the accuracy of land-cover classification. However, most of existing methods accomplish such integration by focusing on the development of new network architectures, which suffer from several flaws: 1) conflicts between general superpixels and semantic labels introduce noise into the training, especially at object boundaries; 2) absence of training guidance for superpixels leads to ineffective regional feature learning; 3) unnecessary superpixel segmentation in the testing stage not only increases the computational burden but also incurs jagged edges. In this study, we propose a novel semantic-aware region (SARI) loss to guide the effective learning of regional features with superpixels for accurate land-cover classification. The key idea of the proposed method is to reduce the feature variance inside and between homogeneous superpixels while enlarging feature discrepancy between heterogeneous ones. The SARI loss is thus designed with three sub-parts, including superpixel variance loss, intra-class similarity loss and inter-class distance loss. We also develop semantic superpixels to assist in the network training with SARI loss while overcoming the limitations of general superpixels. Extensive experiments on two challenging datasets demonstrate that the SARI loss can facilitate regional feature learning, achieving state-of-the-art performance with mIoU scores of around 97.11% and 73.99% on GID and DeepGlobe datasets, respectively. Code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/geovsion/SARI</uri> .

Graph Cardinality Preserved Attention Network for Fault Diagnosis of Induction Motor Under Varying Speed and Load Condition

During the long-term operation of motors, their working conditions are changing due to the industrial demands or declining health status, and traditional diagnosis methods perform poorly in that case. This article proposes a fault diagnosis method based on graph cardinality preserved attention network (GCPAT), which can work under varying working conditions, and can be generalized to the transient state. Diagnosis results are obtained by analyzing signal-converting graphs, which are composed of nodes and edges. First, the vibration signals are converted into symmetrical snowflake images by symmetrized dot pattern (SDP) method. Second, SLIC is developed to make homogeneous super-pixels in SDP images as nodes, and form graphs according to color, texture, and distance features. Finally, the GCPAT is utilized to distinguish motor status. Compared with other state-of-art methods, the results show the out-performance of GCPAT under varying working conditions both in steady and transient state.

Automatic liver tumor segmentation from CT images using hierarchical iterative superpixels and local statistical features

Liver tumor segmentation from CT images plays an important role in disease diagnosis and treatment planning. In this paper, we propose an automatic segmentation framework dedicated to accurate liver tumor extraction from CT images. To reduce the segmentation complexity, 3D U-Net is first employed to extract liver region. Then, the liver region is divided into homogeneous superpixels by applying a LI-SLIC based hierarchical iterative segmentation strategy, in which the superpixels are decomposed recursively according to their intensity standard deviation in order to adhere to tumor boundaries precisely. Meanwhile, each pixel in the liver region is roughly classified into tumor or non-tumor by SVM using its local intensity and texture features. Finally, a voting model is developed to identify tumor regions from superpixels based on the pixel-wise classification results. Extensive experiments on two public clinical datasets and comparisons with many state-of-the-art methods demonstrate the superiority of our method on liver tumor segmentation especially for the images with noises, ambiguous boundaries, and low contrast.

Superpixel-Guided Local Sparsity Prior for Hyperspectral Sparse Regression Unmixing

Sparse regression relaxes the difficulties of blind unmixing of hyperspectral data thanks to the spectral library. Many investigations, however, attach importance to global priors such as sparsity and low-rankness. This letter proposes a local-global-based sparse regression unmixing method, called LGSU, by introducing a local sparsity regularization to help boost the unmixing performance that only considers global sparsity. The proposed LGSU first uses a superpixel-based technique to yield a set of homogeneous superpixels for guiding local sparse regularization purposes. LGSU then considers a traditional ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization to enhance global sparsity. Coupling with local and global sparsity constraints, the proposed LGSU can effectively estimate the abundance of a given image via the alternating direction method of multipliers. Experimental results obtained from synthetic and real hyperspectral images demonstrate the effectiveness of the proposed algorithm.

Segmentation of sonar images with intensity inhomogeneity based on improved MRF

Sonar is one of the most important tools for underwater object detection and submarine topography reconstruction. To classify sonar images automatically and accurately is essential for the navigation and path planning of autonomous underwater vehicles (AUV). However, for the intensity inhomogeneity and speckle noise in sonar images, it is difficult to obtain segmentation results of high accurate rate. To address these issues, in this paper, we advocate a segmentation method incorporating simple linear iterative clustering (SLIC) and adaptive intensity constraint into Markov random field (MRF), to segment sonar images with intensity inhomogeneity into the object highlight, the object shadow and the background areas. The main procedures of the proposed work are as follows: first, SLIC is used to separate sonar images into homogeneous super pixels, and second the homogeneity patches, with a novel intensity constraint strategy, is utilized to optimize the segmentation result of MRF at each iteration. Experimental results reveal that the proposed method performs well and fast on real sonar images which have intensity inhomogeneity problem.

Automatic detection of hard and soft exudates from retinal fundus images

Abstract According to WHO estimates, 400 million people suffer from diabetes, and this number is likely to double by year 2030. Unfortunately, diabetes can have severe complications like glaucoma or retinopathy, which both can cause blindness. The main goal of our research is to provide an automated procedure that can detect retinopathy-related lesions of the retina from fundus images. This paper focuses on the segmentation of so-called white lesions of the retina that include hard and soft exudates. The established procedure consists of three main phases. The preprocessing step compensates the various luminosity patterns found in retinal images, using background and foreground pixel extraction and a data normalization operator similar to Z-transform. This is followed by a modified SLIC algorithm that provides homogeneous superpixels in the image. The final step is an ANN-based classification of pixels using fifteen features extracted from the neighborhood of the pixels taken from the equalized images and from the properties of the superpixel where the pixel belongs. The proposed methodology was tested using high-resolution fundus images originating from the IDRiD database. Pixelwise accuracy is characterized by a 54% Dice score in average, but the presence of exudates is detected with 94% precision.

Unifying the seeds auto-generation (SAGE) with knee cartilage segmentation framework: data from the osteoarthritis initiative.

Manual segmentation is sensitive to operator bias, while semiautomatic random walks segmentation offers an intuitive approach to understand the user knowledge at the expense of large amount of user input. In this paper, we propose a novel random walks seed auto-generation (SAGE) hybrid model that is robust to interobserver error and intensive user intervention. Knee image is first oversegmented to produce homogeneous superpixels. Then, a ranking model is developed to rank the superpixels according to their affinities to standard priors, wherein background superpixels would have lower ranking values. Finally, seed labels are generated on the background superpixel using Fuzzy C-Means method. SAGE has achieved better interobserver DSCs of 0.94 ± 0.029 and 0.93 ± 0.035 in healthy and OA knee segmentation, respectively. Good segmentation performance has been reported in femoral (Healthy: 0.94 ± 0.036 and OA: 0.93 ± 0.034), tibial (Healthy: 0.91 ± 0.079 and OA: 0.88 ± 0.095) and patellar (Healthy: 0.88 ± 0.10 and OA: 0.84 ± 0.094) cartilage segmentation. Besides, SAGE has demonstrated greater mean readers' time of 80 ± 19s and 80 ± 27s in healthy and OA knee segmentation, respectively. SAGE enhances the efficiency of segmentation process and attains satisfactory segmentation performance compared to manual and random walks segmentation. Future works should validate SAGE on progressive image data cohort using OA biomarkers.

User tailored colorization using automatic scribbles and hierarchical features

Fully automatic image colorization is extensively developed using a large number of examples. However, this kind of methods need further detailed revision if users require fine colorization for delicate objects. This paper proposes a novel scribble-based colorization scheme which automatically generates scribbles and propagates color information from representative pixels within homogeneous superpixels. The scribbles are placed in the regions with minimal entropy to ensure color propagation from representative pixels with high confidence. To avoid color bleeding and under colorization, colors are propagated under a robust similarity metric using a set of hierarchical features, where the low level features consist of quaternion Gabor phases and the high level features are extracted using VGG-19 pre-trained deep model. Experimental results on open dataset demonstrate that the proposed colorization scheme can achieve more consistent results with the users' purposes.

Very high resolution remote sensing image classification with SEEDS-CNN and scale effect analysis for superpixel CNN classification

ABSTRACTPixel-based convolutional neural network (CNN) has demonstrated good performance in the classification of very high resolution images (VHRI) from which abstract deep features are extracted. However, conventional pixel-based CNN demands large resources in terms of processing time and disk space. Therefore, superpixel CNN classification has recently become a focus of attention. We therefore propose a CNN based deep learning method combining superpixels extracted via energy-driven sampling (SEEDS) for VHRI classification. The approach consists of three main steps. First, based on the concept of geographic object-based image analysis (GEOBIA), the image is segmented into homogeneous superpixels using the SEEDS based superpixel segmentation method thereby decreasing the number of processing units. Second, the training data and testing data are extracted from the image and concatenated on a superpixel level at a variety of scales for CNN. Third, the training data are input to train the parameters of CNN and abstract deep features are extracted from the VHRI. Using these extracted deep features, we classify two VHRI data sets at single scales and multiple scales. To verify the effectiveness of SEEDS based CNN classification, the performance of SEEDS and three others superpixel segmentation algorithms are compared, and the superpixel extraction via SEEDS method was found to be the optimal superpixel segmentation approach for CNN classification. The scale effect on CNN classification accuracy was investigated by comparing the four superpixel segmentation methods. We found that (1) There is no strong evidence that using scales combinations is better than a single scale in some specific situations; (2) Natural objects with low complexity are not as sensitive to scale as artificial objects; (3) For a simple VHRI that contains clear artificial objects and simple texture, the classification result with multiple scales performs better a the single scale; (4) In contrast, for the complex VHRI containing a large number of complex objects, the classification result with a single small-scale best.

Estimation of signal-dependent and -independent noise from hyperspectral images using a wavelet-based superpixel model

ABSTRACTNoise estimation is crucial for many hyperspectral (HS) image processing algorithms. In real HS images, the random noise is mainly composed of a signal-dependent (SD) photon noise component and a signal-independent (SI) electronic noise component. Based on a parametric model that accounts for the dependence of noise variance on the useful image signal, a novel method is proposed to estimate SD and SI noise variances in this paper. In order to accurately detect the homogeneous regions in noisy images, a new wavelet-based superpixel model is designed to segment a HS images into small patches that adhere to the local textures and hence persist in homogeneous characteristic. Then, the relevance vector machine (RVM) is exploited to split the noise and useful image signal in homogeneous superpixels. Finally, the SD and SI noise variances are obtained by fitting the scatter points of local means versus local total noise variances. Experiments on synthetic and real airborne visible/infrared imaging spectrometer (AVIRIS) HS images demonstrate the effectiveness of the proposed method.

Saliency detection via PageRank and local spline regression

A saliency detection method based on the PageRank algorithm and local spline regression (LSR) is proposed. Unlike the principles of most existing bottom-up methods, in the proposed method, saliency detection is considered a label propagation and regression problem. The input image is represented as two-scale graphs with homogeneous superpixels. Multiple color features and spatial information are effectively captured to define the relevance of each node to its surroundings. PageRank is used to assign the saliency value to each region depending on the similarity of the image elements with boundary cues. Furthermore, to comprehensively consider both foreground and background cues, a robust classifier based on LSR is employed to highlight more complete foreground regions. The integrated and refined pixel-level saliency map provides a significant performance boost in the final result. Extensive experiments on three large public datasets demonstrate that the proposed algorithm consistently achieves superior performance compared with state-of-the-art saliency models.

Coarse-to-fine salient object detection based on deep convolutional neural networks

With explosive growth of image data, automatic image interpretation becomes more and more important. Saliency detection is one of the fundamental problems. To predict the saliency map, traditional saliency detection approaches use handcrafted features, which are not robust for complex scene. Recently, convolutional neural network (CNN) have shown good performance in computer vision problems. In this paper, we propose a coarse-to-fine approach combining pixel-wise FCN with superpixel-based CNN for detecting salient objects with precise boundaries. Firstly, the fully convolutional network (FCN) model is used to produce a coarse saliency map. Instead of patch-based CNN taking in overlapping patches as samples, the FCN model adopts the pixel-wise structure which can predict the location of the salient objects from the global aspect. Then, superpixel clustering is presented to decompose the image into homogeneous superpixels. For each superpixel, the local superpixel-based CNN model is created to integrate the coarse saliency map with the original image information for refining the detected salient objects with precise boundaries. Experimental results on large benchmark databases demonstrate the proposed method perform well when tested against the state-of-the-art methods.

Saliency detection for panoramic landscape images of outdoor scenes

Saliency detection has been researched for conventional images with standard aspect ratios, however, it is a challenging problem for panoramic images with wide fields of view. In this paper, we propose a saliency detection algorithm for panoramic landscape images of outdoor scenes. We observe that a typical panoramic image includes several homogeneous background regions yielding horizontally elongated distributions, as well as multiple foreground objects with arbitrary locations. We first estimate the background of panoramic images by selecting homogeneous superpixels using geodesic similarity and analyzing their spatial distributions. Then we iteratively refine an initial saliency map derived from background estimation by computing the feature contrast only within local surrounding area whose range and shape are changed adaptively. Experimental results demonstrate that the proposed algorithm detects multiple salient objects faithfully while suppressing the background successfully, and it yields a significantly better performance of panorama saliency detection compared with the recent state-of-the-art techniques.

Text segmentation using superpixel clustering

Text segmentation is important for text image analysis and recognition; however, it is challenging due to noise and complex background in natural scenes. Superpixel-based image representation can enhance robustness to noise and local disturbances, but conventional superpixel algorithms are difficult to obtain the complete stroke regions and accurate boundaries for text images. In this study, a text segmentation method based on superpixel clustering is proposed. First, to generate accurate superpixels for text images, an adaptive simple linear iterative clustering-based text superpixel generation algorithm is proposed. The adaptive superpixel size and compactness are calculated to enhance boundary adherence. Second, to increase the complete coverage of strokes from superpixels, superpixel clustering merges homogeneous superpixels into larger regions for both strokes and the background. A modified density-based spatial clustering of applications with noise is proposed. Finally, stroke superpixel verification assigns each region to a stroke or to the background and the text segmentation result is obtained. The proposed method shows promising robustness to noise and complex background textures. Experimental results on the Korea Advanced Institute of Science and Technology (KAIST) scene text dataset, International Conference on Document Analysis and Recognition (ICDAR) 2003 natural scene text image dataset and Street View Text dataset verify that this method is effective and significantly outperforms existing methods.

Texture Image Optimization Segmentation Based on the SLIC Algorithm

The superpixels gathered by high similar pixels can effectively reduce the redundancy of the image and the complexity of the subsequent image processing. In this paper, a new method combined with SLIC (simple linear inerative clustering) algorithm and the energy optimization function to segment texture image is proposed. SLIC approach is used to obtain the initial superpixels, and optimization function can relocate new center points to creat more homogeneous superpixels. It has been shown that this method performs better adaptability in the complex texture image.

Local Pattern Collocations Using Regional Co-occurrence Factorization

Human vision benefits a lot from pattern collocations in visual activities such as object detection and recognition. Usually, pattern collocations display as the co-occurrences of visual primitives, e.g., colors, gradients, or textons, in neighboring regions. In the past two decades, many sophisticated local feature descriptors have been developed to describe visual primitives, and some of them even take into account the co-occurrence information for improving their discriminative power. However, most of these descriptors only consider feature co-occurrence within a very small neighborhood, e.g., 8-connected or 16-connected area, which would fall short in describing pattern collocations built up by feature co-occurrences in a wider neighborhood. In this paper, we propose to describe local pattern collocations by using a new and general regional co-occurrence approach. In this approach, an input image is first partitioned into a set of homogeneous superpixels. Then, features in each superpixel are extracted by a variety of local feature descriptors, based on which a number of patterns are computed. Finally, pattern co-occurrences within the superpixel and between the neighboring superpixels are calculated and factorized into a final descriptor for local pattern collocation. The proposed regional co-occurrence framework is extensively tested on a wide range of popular shape, color, and texture descriptors in terms of image and object categorizations. The experimental results have shown significant performance improvements by using the proposed framework over the existing popular descriptors.

Objectness Supervised Merging Algorithm for Color Image Segmentation

Ideal color image segmentation needs both low-level cues and high-level semantic features. This paper proposes a two-hierarchy segmentation model based on merging homogeneous superpixels. First, a region growing strategy is designed for producing homogenous and compact superpixels in different partitions. Total variation smoothing features are adopted in the growing procedure for locating real boundaries. Before merging, we define a combined color-texture histogram feature for superpixels description and, meanwhile, a novel objectness feature is proposed to supervise the region merging procedure for reliable segmentation. Both color-texture histograms and objectness are computed to measure regional similarities between region pairs, and the mixed standard deviation of the union features is exploited to make stop criteria for merging process. Experimental results on the popular benchmark dataset demonstrate the better segmentation performance of the proposed model compared to other well-known segmentation algorithms.

Abstract LB-285: Computational pathology for predicting prostate cancer recurrence

Abstract Background: Conventional methods for predicting prostate cancer (PCa) aggressiveness, which rely heavily on the Gleason score of architectural pattern, remain imperfect. We have developed an approach that combines machine vision and machine learning analysis of routine H&amp;E stained tumors to discriminate recurrent from non-recurrent patients after prostatectomy, based on architectural, color, and nuclear patterns. Methods: We used a set of H&amp;E-stained TMAs containing quadruplicate cores from 176 recurrent cases of PCa and 154 non-recurrent controls, frequency matched on age, Gleason, pathological stage and race. Using software specifically adaptable for computational pathology with hierarchical object recognition (Definiens Developer XD®), we processed images of each tumor at two spatial scale levels: a larger scale for recognition of tissue compartments (epithelium, stroma, lumen and inflammation), and a smaller scale for recognition of normal and atypical nuclei in each of the compartments. Magnification of 20x was used for both levels. At the tissue level, we used multi-resolution segmentation to segment homogeneous superpixels, which were then classified using machine learning into epithelium, stroma. or ambiguous tissue classified as intermediate between epithelium and stroma. We used relative darkness in a spatial neighborhood, size, and shape criteria to segment typical and atypical nuclei. Relational features among objects within and between the tissue and nuclear levels were also created. The resultant feature library contained 884 base features and 3,551 total variables. We used several stages of iterative L1 penalized logistic regression and 5-fold cross-validation to reduce variables and obtain AUC estimates. Variable stability criteria applied across the folds further minimized model overfitting. Results: PSA, Gleason and CAPRA-s achieved AUCs &amp;lt; 0.60 for recurrence in this matched dataset. A fifth stage L1 model containing 18 histometric features had cross-validation AUC = 0.81 (95% CI: 0.76-0.85). A more permissive 74 feature model had AUC = 0.95 (95% CI: 0.93-0.97). Subgroup analysis restricted to Gleason 7 cases produced AUCs ranging from 0.79-0.93 with varying model sparseness. Including pre-surgical PSA, Gleason grade or CAPRA-s score did not change model performance. Notably, leading features were frequently derived from stromal or ambiguously stromal regions and involved nuclear shape or texture. Their independence from tumor grading reflected subject matching and lack of explicit accounting for nuclear or stromal characteristics in the Gleason criteria. Conclusion: A computational pathology approach, applied to discrimination of recurrent and non-recurrent PCa in routine H&amp;E stains, shows considerable promise and appears capable of adding to the predictive power of Gleason and CAPRA scoring. We will conduct additional validation studies and extend this work to biopsy samples. Citation Format: Amit Sethi, Lingdao Sha, Ryan J. Deaton, Virgilia Macias, Andrew H. Beck, Peter H. Gann. Computational pathology for predicting prostate cancer recurrence. [abstract]. In: Proceedings of the 106th Annual Meeting of the American Association for Cancer Research; 2015 Apr 18-22; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Res 2015;75(15 Suppl):Abstract nr LB-285. doi:10.1158/1538-7445.AM2015-LB-285

Regularity Preserved Superpixels and Supervoxels

Most existing superpixel algorithms ignore the spatial structure and regularity properties, which result in undesirable sizes and location relationships for the subsequent processing. In this paper, we introduce a new method to generate the regularity preserved superpixels. Starting from the lattice seeds, our method relocates them to the pixel with locally maximal edge magnitudes and treats them as the superpixel junctions. Then, the shortest path algorithm is employed to find the local optimal boundary connecting each adjacent junction pair. Thanks to the local constraints, our method obtains homogeneous superpixels with adjacency in lowly textured and uniform regions and simultaneously preserves the boundary adherence in the high contrast contents. Our method preserves the regularity property without significantly sacrificing the segmentation accuracy. Moreover, we extend this regular constraint for generating the supervoxels. Our method obtains the regular supervoxels, which preserves the structural relation on both spatial and temporal spaces of the video. Quantitative and qualitative experimental results on benchmark datasets demonstrate that our simple but effective method outperforms the existing regular superpixel methods.