Multi-scale Superpixel Research Articles

Visual object tracking with multi-scale superpixels and color-feature guided kernelized correlation filters

While discriminative correlation filter based tracking algorithms have achieved competitive results and demonstrated successfully, there still remain some challenges, such as handling the scenarios of scale variation, fast motion, etc. A region of interest with a fixed size, which is usually used by discriminative correlation filter based tracking algorithms to train correlation filter and track object, makes the trackers hard to deal with the challenges of fast motion and scale variation. It also restricts the use of object structure information. In this paper, we propose a Multi-Scale Superpixels and Color Feature Guided Kernelized Correlation Filters (MSSCF-KCF) to deal with the problems mentioned above. Firstly, we treat tracking procedure as optimizing the combination of components of an object, and propose a multi-scale superpixel method to segment object image based on a proposed global confidence mask which determines the center and size of object patches automatically. Then, KCF is embedded into Bayesian filter framework, in which the proposed color guided confidence map is viewed as an observation model, to expand the tracking range and improve the performance when handling fast motion. Finally, we propose a center distance matrix of each object patch to use structure information of object. A novel min–max criterion as well as drop-out strategy is used to search the optimal combination and estimate the state of object, which also helps the tracker to cope with scale variation. Besides, monitoring update strategy is proposed to monitor tracking operation and adjust parameters dynamically. The proposed algorithm is tested on a self-made dataset and some challenging sequences with a novel evaluation criterion. The results demonstrate the feasibility and effectiveness compared with the state-of-the-art methods.

Background Subtraction Based on Random Superpixels Under Multiple Scales for Video Analytics

Background subtraction is a fundamental problem of computer vision, which is usually the first step of video analytics to extract the interesting region. Most previously available region-based background subtraction methods ignore the similarity between the pixels, meaning that the information gained from the pixels that do not contribute, or even contribute negatively to understanding an image, is taken into account. A new background subtraction model based on random superpixel segmentation under multiple scales is proposed. A custom region segmentation area is replaced with a superpixel segmentation area that uses similarity characteristics for pixels in the superpixel area. The compactness of the pixels in the same superpixel area means that the pixels positively contribute to understanding an image compared with when using custom region pixels. Superpixel segmentation is performed using the random simple linear iterative cluster method. Taking random samples during the superpixel segmentation process produces the Matthew effect, thus improving the robustness and efficiency of the model. Multi-scale superpixel segmentation is therefore guaranteed to give more accurate results. Standard benchmark experiments using the proposed approach produced encouraging results compared with the results given by previously available algorithms.

Scalable Learning Framework for Traversable Region Detection Fusing With Appearance and Geometrical Information

In this paper, we present an online learning framework for traversable region detection fusing both appearance and geometry information. Our framework proposes an appearance classifier supervised by the sparse geometric clues to capture the variation in online data, yielding dense detection result in real time. It provides superior detection performance using appearance information with weak geometric prior and can be further improved with more geometry from external sensors. The learning process is divided into three steps: First, we construct features from the super-pixel level, which reduces the computational cost compared with the pixel level processing. Then we classify the multi-scale super-pixels to vote the label of each pixel. Second, we use weighted extreme learning machine as our classifier to deal with the imbalanced data distribution since the weak geometric prior only initializes the labels in a small region. Finally, we employ the online learning process so that our framework can be adaptive to the changing scenes. Experimental results on three different styles of image sequences, i.e., shadow road, rain sequence, and variational sequence, demonstrate the adaptability, stability, and parameter insensitivity of our weak geometry motivated method. We further demonstrate the performance of learning framework on additional five challenging data sets captured by Kinect V2 and stereo camera, validating the method’s effectiveness and efficiency.

Automatic psoriasis lesion segmentation in two-dimensional skin images using multiscale superpixel clustering.

Psoriasis is a chronic skin disease that is assessed visually by dermatologists. The Psoriasis Area and Severity Index (PASI) is the current gold standard used to measure lesion severity by evaluating four parameters, namely, area, erythema, scaliness, and thickness. In this context, psoriasis skin lesion segmentation is required as the basis for PASI scoring. An automatic lesion segmentation method by leveraging multiscale superpixels and [Formula: see text]-means clustering is outlined. Specifically, we apply a superpixel segmentation strategy on CIE-[Formula: see text] color space using different scales. Also, we suppress the superpixels that belong to nonskin areas. Once similar regions on different scales are obtained, the [Formula: see text]-means algorithm is used to cluster each superpixel scale separately into normal and lesion skin areas. Features from both [Formula: see text] and [Formula: see text] color bands are used in the clustering process. Furthermore, majority voting is performed to fuse the segmentation results from different scales to obtain the final output. The proposed method is extensively evaluated on a set of 457 psoriasis digital images, acquired from the Royal Melbourne Hospital, Melbourne, Australia. Experimental results have shown evidence that the method is very effective and efficient, even when applied to images containing hairy skin and diverse lesion size, shape, and severity. It has also been ascertained that CIE-[Formula: see text] outperforms other color spaces for psoriasis lesion analysis and segmentation. In addition, we use three evaluation metrics, namely, Dice coefficient, Jaccard index, and pixel accuracy where scores of 0.783%, 0.698%, and 86.99% have been achieved by the proposed method for the three metrics, respectively. Finally, compared with existing methods that employ either skin decomposition and support vector machine classifier or Euclidean distance in the hue-chrome plane, our multiscale superpixel-based method achieves markedly better performance with at least 20% accuracy enhancement.

Multiscale Superpixel-Level Subspace-Based Support Vector Machines for Hyperspectral Image Classification

This letter introduces a new spectral–spatial classification method for hyperspectral images. A multiscale superpixel segmentation is first used to model the distribution of classes based on spatial information. In this context, the original hyperspectral image is integrated with segmentation maps via a feature fusion process in different scales such that the pixel-level data can be represented by multiscale superpixel-level (MSP) data sets. Then, a subspace-based support vector machine (SVMsub) is adopted to obtain the classification maps with multiscale inputs. Finally, the classification result is achieved via a decision fusion process. The resulting method, called MSP-SVMsub, makes use of the spatial and spectral coherences, and contributes to better feature characterization. Experimental results based on two real hyperspectral data sets indicate that the MSP-SVMsub exhibits good performance compared with other related methods.

Adaptive strategy for superpixel-based region-growing image segmentation

This work presents a region-growing image segmentation approach based on superpixel decomposition. From an initial contour-constrained over-segmentation of the input image, the image segmentation is achieved by iteratively merging similar superpixels into regions. This approach raises two key issues: (1) how to compute the similarity between superpixels in order to perform accurate merging and (2) in which order those superpixels must be merged together. In this perspective, we firstly introduce a robust adaptive multi-scale superpixel similarity in which region comparisons are made both at content and common border level. Secondly, we propose a global merging strategy to efficiently guide the region merging process. Such strategy uses an adpative merging criterion to ensure that best region aggregations are given highest priorities. This allows to reach a final segmentation into consistent regions with strong boundary adherence. We perform experiments on the BSDS500 image dataset to highlight to which extent our method compares favorably against other well-known image segmentation algorithms. The obtained results demonstrate the promising potential of the proposed approach.

Discriminative feature learning and region consistency activation for robust scene labeling

This paper presents a learned feature based framework for both outdoor and indoor scene labeling. This framework is combined with a discriminative feature learning process to produce the posteriors of every pixel and a novel strategy to improve the global label consistency of a scene. First, we use Convolutional Neural Networks (ConvNets) to learn the most relevant features of a scene at the multi-scale superpixel level. The effect of both trained and general ConvNets features for our scene labeling framework are investigated. Then, based on the predicted posteriors from the learned features, we propose an algorithm called Region Consistency Activation (RCA) to iteratively improve the global label consistency at different levels of the Ultrametric Contour Map (UCM). In addition, we propose a strategy to make the hyper-parameters of RCA adaptive to the test images, which results in a better generalization ability compared with the hyper-parameters tuning based RCA. Our scene labeling framework were rigorously tested on three popular scene labeling datasets: Stanford Background, SIFT Flow and NYU-Depth V2. Experiments show that our proposed method consistently produces better accuracy and visual consistency compared with the state-of-the-art methods for both outdoor and indoor scenes.

Multiscale Superpixel-Based Sparse Representation for Hyperspectral Image Classification

Recently, superpixel segmentation has been proven to be a powerful tool for hyperspectral image (HSI) classification. Nonetheless, the selection of the optimal superpixel size is a nontrivial task. In addition, compared with single-scale superpixel segmentation, the same image segmented on a different scale can obtain different structure information. To overcome such a drawback also utilizing the structural information, a multiscale superpixel-based sparse representation (MSSR) algorithm for the HSI classification is proposed. Specifically, a modified segmentation strategy of multiscale superpixels is firstly applied on the HSI. Once the superpixels on different scales are obtained, the joint sparse representation classification is used to classify the multiscale superpixels. Furthermore, majority voting is utilized to fuse the labels of different scale superpixels and to obtain the final classification result. Two merits are realized by the MSSR. First, multiscale information fusion can more effectively explore the spatial information of HSI. Second, in the multiscale superpixel segmentation, except for the first scale, the superpixel number on a different scale for different HSI datasets can be adaptively changed based on the spatial complexity of the corresponding HSI. Experiments on four real HSI datasets demonstrate the qualitative and quantitative superiority of the proposed MSSR algorithm over several well-known classifiers.

Graph-based semi-supervised classification on very high resolution remote sensing images

Classification of very high resolution (VHR) remote sensing imagery is a rapidly emerging discipline but faces several challenges owing to the huge scale of the pixel data involved, indiscernibility in the traditionally used features to represent various regions, and the lack of available ground truth data. This paper provides a framework which elegantly overcomes these hurdles by providing a novel semi-supervised learning approach which employs multiscale superpixel tessellation representations of VHR imagery. Superpixels are homogeneous and irregularly shaped regions which form the backbone of our approach and are used to derive novel features by learning a decision tree. Our semi-supervised learning approach works on a superpixel graph and seamlessly combines the large margin capability of a support vector machine (SVM) with a graph-based Laplacian label propagation approach to obtain a novel objective function. Further we also provide a self-contained and easily parallelisable linear iterative optimisation approach based on the principle of majorisation-minimisation. We evaluate this approach on four different geographic settings with varying neighbourhood types and draw comparisons with the popular and widely used Gaussian multiple instance learning algorithm. Our results showcase several advantages in accuracy and efficiency, which coupled with the ease of model building and inherently parallelisable optimisation make our framework a great choice for deployment in large scale applications like global human settlement mapping and population distribution, and change detection.

Graph-based semi-supervised classification on very high resolution remote sensing images

Classification of very high resolution (VHR) remote sensing imagery is a rapidly emerging discipline but faces several challenges owing to the huge scale of the pixel data involved, indiscernibility in the traditionally used features to represent various regions, and the lack of available ground truth data. This paper provides a framework which elegantly overcomes these hurdles by providing a novel semi-supervised learning approach which employs multiscale superpixel tessellation representations of VHR imagery. Superpixels are homogeneous and irregularly shaped regions which form the backbone of our approach and are used to derive novel features by learning a decision tree. Our semi-supervised learning approach works on a superpixel graph and seamlessly combines the large margin capability of a support vector machine (SVM) with a graph-based Laplacian label propagation approach to obtain a novel objective function. Further we also provide a self-contained and easily parallelisable linear iterative optimisation approach based on the principle of majorisation-minimisation. We evaluate this approach on four different geographic settings with varying neighbourhood types and draw comparisons with the popular and widely used Gaussian multiple instance learning algorithm. Our results showcase several advantages in accuracy and efficiency, which coupled with the ease of model building and inherently parallelisable optimisation make our framework a great choice for deployment in large scale applications like global human settlement mapping and population distribution, and change detection.

Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images.

This paper presents and evaluates a fully automatic method for detection of ductal carcinoma in situ (DCIS) in digitized hematoxylin and eosin (H&E) stained histopathological slides of breast tissue. The proposed method applies multi-scale superpixel classification to detect epithelial regions in whole-slide images (WSIs). Subsequently, spatial clustering is utilized to delineate regions representing meaningful structures within the tissue such as ducts and lobules. A region-based classifier employing a large set of features including statistical and structural texture features and architectural features is then trained to discriminate between DCIS and benign/normal structures. The system is evaluated on two datasets containing a total of 205 WSIs of breast tissue. Evaluation was conducted both on the slide and the lesion level using FROC analysis. The results show that to detect at least one true positive in every DCIS containing slide, the system finds 2.6 false positives per WSI. The results of the per-lesion evaluation show that it is possible to detect 80% and 83% of the DCIS lesions in an abnormal slide, at an average of 2.0 and 3.0 false positives per WSI, respectively. Collectively, the result of the experiments demonstrate the efficacy and accuracy of the proposed method as well as its potential for application in routine pathological diagnostics. To the best of our knowledge, this is the first DCIS detection algorithm working fully automatically on WSIs.

Beyond Holistic Descriptors, Keypoints, and Fixed Patches: Multiscale Superpixel Grids for Place Recognition in Changing Environments

Vision-based place recognition in environments subject to severe appearance changes due to day–night cycles, changing weather or seasons is a challenging task. Existing methods typically exploit image sequences, holistic descriptors and/or training data. Each of these approaches limits the practical applicability, e.g., to constant viewpoints for usage of holistic image descriptors. Recently, the combination of local region detectors and descriptors based on convolutional neural networks showed to be a promising approach to overcome these limitations. However, established region detectors, e.g., keypoint detectors, showed severe problems to provide repetitive landmarks despite dramatically changed appearance of the environment. Thus, they are typically replaced by holistic image descriptors or fixedly arranged patches—both known to be sensitive towards viewpoint changes. In this letter, we present a novel local region detector, SP-Grid, that is particularly suited for the combination of severe appearance and viewpoint changes. It is based on multiscale image oversegmentations and is designed to combine the advantages of keypoints and fixed image patches by starting from an initial grid-like arrangement and subsequently adapting to the image content. The grid-like arrangement showed to be beneficial in the presence of severe appearance changes and the adaptation to the image content increases the robustness toward viewpoint changes. The experimental evaluation will show the benefit compared to existing local region detectors and holistic image descriptors.

On developing and enhancing plant-level disease rating systems in real fields

Cercospora leaf spot (CLS) is one of the most serious diseases of sugar beet worldwide, and if uncontrolled, causes nearly complete defoliation and loss of revenue for beet growers. The beet sugar industry continuously seeks CLS-resistant sugar beet cultivars as one strategy to combat this disease. Normally human experts manually observe and rate the resistance of a large variety of sugar beet plants over a period of a few months. Unfortunately, this procedure is laborious and the labels vary from one expert to another resulting in disagreements on the level of resistance. Therefore, we propose a novel computer vision system, CLS Rater, to automatically and accurately rate plant images in the real field to the “USDA scale” of 0–10. Given a set of plant images captured by a tractor-mounted camera, CLS Rater extracts multi-scale superpixels, where in each scale a novel Histogram of Importances feature encodes both the within-superpixel local and across-superpixel global appearance variations. These features at different superpixel scales are then fused for learning a regressor that estimates the rating for each plant image. We further address the issue of the noisy labels by experts in the field, and propose a method to enhance the performance of the CLS Rater by automatically calibrating the experts ratings to ensure consistency. We test our system on the field data collected from two years over a two-month period for each year, under different lighting and weather conditions. Experimental results show that both the CLS Rater and the enhanced CLS Rater to be highly consistent with the rating errors of 0.65 and 0.59 respectively, which demonstrates a higher consistency than the rating standard deviation of 1.31 by human experts.

PolSAR Ship Detection Based on Superpixel-Level Scattering Mechanism Distribution Features

To improve the target detection performance under a low signal-to-clutter ratio, this letter presents a new polarimetric synthetic aperture radar (PolSAR) ship detector based on superpixel-level scattering mechanism (SM) distribution features. The proposed method is based on the observation that the SMs of targets and clutter have different distributions in the classical $H/\alpha$ plane. To make use of this difference in ship detection, multiscale superpixels are first generated for PolSAR images. Then, two features describing the SM distribution in the superpixel are proposed. Based on these features, a test statistic independent of the scattering intensity is finally defined. The performance improvement of the proposed method is verified using a synthetic data set and real PolSAR images obtained from a RADARSAT-2 data set.

Saliency Detection Based on Multi-Scale Superpixel and Dictionary Learning

In recent years, saliency detection has been gaining increasing attention since it could significantly boost many content-based multimedia applications. In this paper, we propose a visual saliency detection algorithm based on multi-scale superpixel and dictionary learning . Firstly, in each scale space, we extract the boundaries as the training samples to learn a dictionary through sparse coding and dictionary learning methods. Then, according to reconstruction error of each superpixel, the saliency map is generated for each scale of superpixel. Finally, some saliency maps from different scale spaces are fused together to generate the final saliency map. The experimental results show that the proposed algorithm can highlight the salient regions uniformly and performs better compared with the other five methods.

Saliency Detection with Multi-Scale Superpixels

We propose a salient object detection algorithm via multi-scale analysis on superpixels. First, multi-scale segmentations of an input image are computed and represented by superpixels. In contrast to prior work, we utilize various Gaussian smoothing parameters to generate coarse or fine results, thereby facilitating the analysis of salient regions. At each scale, three essential cues from local contrast, integrity and center bias are considered within the Bayesian framework. Next, we compute saliency maps by weighted summation and normalization. The final saliency map is optimized by a guided filter which further improves the detection results. Extensive experiments on two large benchmark datasets demonstrate the proposed algorithm performs favorably against state-of-the-art methods. The proposed method achieves the highest precision value of 97.39% when evaluated on one of the most popular datasets, the ASD dataset.

HUMAN BODY SEGMENTATION IN A STATIC IMAGE WITH ON-LINE ADABOOST AT MULTISCALE SUPERPIXELS

In this work, we propose a new method for accurate human body detection in a static image with multi-scale superpixels based on two models. First, based on the face detection, we use designed torso part model to estimate the torso part region to provide the positive samples of on-line AdaBoost. Then, according to segmented torso, we estimate the hip region using the designed torso model to provide the positive samples of on-line AdaBoost. Finally, we combine the segmented part and the skin region for body. The main contributions of this work are as follows: (1) A new framework for human body detection using multi-scale superpixels and classifier with auto-threshold is proposed. (2) Two models have been designed for the human body segmentation. Experimental results show that the framework can accurately detect the human body effectively and efficiently.