Superpixel Classification Research Articles

An Algorithm for Plant Disease Visual Symptom Detection in Digital Images Based on Superpixels

Quantifying diseased areas in plant leaves is an important procedure in agriculture, as it contributes to crop monitoring and decision-making for crop protection. It is, however, a time-consuming and very subjective manual procedure whose automation is, therefore, highly expected. This work proposes a new method for the automatic segmentation of diseased leaf areas. The method used the Simple Linear Iterative Clustering (SLIC) algorithm to group similar-color pixels together into regions called superpixels. The color features of superpixel clusters were used to train artificial neural networks (ANNs) for the classification of superpixels as healthy or not healthy. These network parameters were heuristically tuned by choosing the network with the best classification performance to obtain the automatic segmentation of the diseased areas. The performance of the classifier was measured by comparing its automatic segmentations with those manually made from a database with public and private images divided into nine groups by visual symptom and plant. The mean error of the area obtained was always below 11%, and the average F-score was 0.67, which is higher than that found by the other two approaches reported in the literature (0.57 and 0.58) and used here for comparison.

Improving the performance of convolutional neural network for skin image classification using the response of image analysis filters

In this work, we focus in the analysis of dermoscopy images using convolutional neural networks (CNNs). More specifically, we investigate the value of augmenting CNN inputs with the response of mid-level computer vision filters, using the traditional inputting of simple RGB pixel values as baseline. The proposed methodology is applied on two pattern recognition problems with clinical significance: the binary classification of skin lesions in dermoscopy images into “malignant” and “non-malignant” (nevus skin lesions) cases and the four-class, superpixel classification into differential structures that appear in skin lesions. The transfer learning technique is also utilized to compensate for the limited size of the available training image datasets. Results show that filter-based input augmentation using the response of mid-level computer vision filters significantly improves the classification accuracy achieved by the CNN architectures and simplifies the weights of the receptive fields.

High-Resolution Remote Sensing Image Classification Method Based on Convolutional Neural Network and Restricted Conditional Random Field

Convolutional neural networks (CNNs) can adapt to more complex data, extract deeper characteristics from images, and achieve higher classification accuracy in remote sensing image scene classification and object detection compared to traditional shallow-model methods. However, directly applying common-structure CNNs to pixel-based remote sensing image classification will lead to boundary or outline distortions of the land cover and consumes enormous computation time in the image classification stage. To solve this problem, we propose a high-resolution remote sensing image classification method based on CNN and the restricted conditional random field algorithm (CNN-RCRF). CNN-RCRF adopts CNN superpixel classification instead of pixel-based classification and uses the restricted conditional random field algorithm (RCRF) to refine the superpixel result image into a pixel-based result. The proposed method not only takes advantage of the classification ability of CNNs but can also avoid boundary or outline distortions of the land cover and greatly reduce computation time in classifying images. The effectiveness of the proposed method is tested with two high-resolution remote sensing images, and the experimental results show that the CNN-RCRF outperforms the existing traditional methods in terms of overall accuracy, and CNN-RCRF’s computation time is much less than that of traditional pixel-based deep-model methods.

A Cloud Detection Method for Landsat 8 Images Based on PCANet

Cloud detection for remote sensing images is often a necessary process, because cloud is widespread in optical remote sensing images and causes a lot of difficulty to many remote sensing activities, such as land cover monitoring, environmental monitoring and target recognizing. In this paper, a novel cloud detection method is proposed for multispectral remote sensing images from Landsat 8. Firstly, the color composite image of Bands 6, 3 and 2 is divided into superpixel sub-regions through Simple Linear Iterative Cluster (SLIC) method. Then, a two-step superpixel classification strategy is used to predict each superpixel as cloud or non-cloud. Thirdly, a fully connected Conditional Random Field (CRF) model is used to refine the cloud detection result, and accurate cloud borders are obtained. In the two-step superpixel classification strategy, the bright and thick cloud superpixels, as well as the obvious non-cloud superpixels, are firstly separated from potential cloud superpixels through a threshold function, which greatly speeds up the detection. The designed double-branch PCA Network (PCANet) architecture can extract the high-level information of cloud, then combined with a Support Vector Machine (SVM) classifier, the potential superpixels are correctly classified. Visual and quantitative comparison experiments are conducted on the Landsat 8 Cloud Cover Assessment (L8 CCA) dataset; the results indicate that our proposed method can accurately detect clouds under different conditions, which is more effective and robust than the compared state-of-the-art methods.

Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy.

Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical structure classification and image tagging that features an intrinsic measure of confidence to estimate its own performance with high reliability and which can be applied to both RGB and multispectral imaging (MI) data. Organ recognition is performed using a superpixel classification strategy based on textural and reflectance information. Classification confidence is estimated by analyzing the dispersion of class probabilities. Assessment of the proposed technology is performed through a comprehensive in vivo study with seven pigs. When applied to image tagging, mean accuracy in our experiments increased from 65% (RGB) and 80% (MI) to 90% (RGB) and 96% (MI) with the confidence measure. Results showed that the confidence measure had a significant influence on the classification accuracy, and MI data are better suited for anatomical structure labeling than RGB data. This paper significantly enhances the state of art in automatic labeling of endoscopic videos by introducing the use of the confidence metric, and by being the first study to use MI data for in vivo laparoscopic tissue classification. The data of our experiments will be released as the first in vivo MI dataset upon publication of this paper.

Supervised and unsupervised segmentation using superpixels, model estimation, and graph cut

Image segmentation is widely used as an initial phase of many image analysis tasks. It is often advantageous to first group pixels into compact, edge-respecting superpixels, because these reduce the size of the segmentation problem and thus the segmentation time by an order of magnitudes. In addition, features calculated from superpixel regions are more robust than features calculated from fixed pixel neighborhoods. We present a fast and general multiclass image segmentation method consisting of the following steps: (i) computation of superpixels; (ii) extraction of superpixel-based descriptors; (iii) calculating image-based class probabilities in a supervised or unsupervised manner; and (iv) regularized superpixel classification using graph cut. We apply this segmentation pipeline to five real-world medical imaging applications and compare the results with three baseline methods: pixelwise graph cut segmentation, supertexton-based segmentation, and classical superpixel-based segmentation. On all datasets, we outperform the baseline results. We also show that unsupervised segmentation is surprisingly efficient in many situations. Unsupervised segmentation provides similar results to the supervised method but does not require manually annotated training data, which is often expensive to obtain.

VARIABLE SELECTION FOR ROAD SEGMENTATION IN AERIAL IMAGES

Abstract. For extraction of road pixels from combined image and elevation data, Wegner et al. (2015) proposed classification of superpixels into road and non-road, after which a refinement of the classification results using minimum cost paths and non-local optimization methods took place. We believed that the variable set used for classification was to a certain extent suboptimal, because many variables were redundant while several features known as useful in Photogrammetry and Remote Sensing are missed. This motivated us to implement a variable selection approach which builds a model for classification using portions of training data and subsets of features, evaluates this model, updates the feature set, and terminates when a stopping criterion is satisfied. The choice of classifier is flexible; however, we tested the approach with Logistic Regression and Random Forests, and taylored the evaluation module to the chosen classifier. To guarantee a fair comparison, we kept the segment-based approach and most of the variables from the related work, but we extended them by additional, mostly higher-level features. Applying these superior features, removing the redundant ones, as well as using more accurately acquired 3D data allowed to keep stable or even to reduce the misclassification error in a challenging dataset.

Semi-supervised superpixel classification for medical images segmentation: application to detection of glaucoma disease

Glaucoma is a disease characterized by damaging the optic nerve head, this can result in severe vision loss. An early detection and a good treatment provided by the ophthalmologist are the keys to preventing optic nerve damage and vision loss from glaucoma. Its screening is based on the manual optic cup and disc segmentation to measure the vertical cup to disc ratio (CDR). However, obtaining the regions of interest by the expert ophthalmologist can be difficult and is often a tedious task. In most cases, the unlabeled images are more numerous than the labeled ones.We propose an automatic glaucoma screening approach named Super Pixels for Semi-Supervised Segmentation “SP3S”, which is a semi-supervised superpixel-by-superpixel classification method, consisting of three main steps. The first step has to prepare the labeled and unlabeled data, applying the superpixel method and bringing in an expert for the labeling of superpixels. In the second step, We incorporate prior knowledge of the optic cup and disc by including color and spatial information. In the final step, semi-supervised learning by the Co-forest classifier is trained only with a few number of labeled superpixels and a large number of unlabeled superpixels to generate a robust classifier. For the estimation of the optic cup and disc regions, the active geometric shape model is used to smooth the disc and cup boundary for the calculation of the CDR. The obtained results for glaucoma detection, via an automatic cup and disc segmentation, established a potential solution for glaucoma screening. The SP3S performance shows quantitatively and qualitatively similar correspondence with the expert segmentation, providing an interesting tool for semi-automatic recognition of the optic cup and disc in order to achieve a medical progress of glaucoma disease.

A Bottom-Up Approach for Pancreas Segmentation Using Cascaded Superpixels and (Deep) Image Patch Labeling.

Robust organ segmentation is a prerequisite for computer-aided diagnosis, quantitative imaging analysis, pathology detection, and surgical assistance. For organs with high anatomical variability (e.g., the pancreas), previous segmentation approaches report low accuracies, compared with well-studied organs, such as the liver or heart. We present an automated bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans. The method generates a hierarchical cascade of information propagation by classifying image patches at different resolutions and cascading (segments) superpixels. The system contains four steps: 1) decomposition of CT slice images into a set of disjoint boundary-preserving superpixels; 2) computation of pancreas class probability maps via dense patch labeling; 3) superpixel classification by pooling both intensity and probability features to form empirical statistics in cascaded random forest frameworks; and 4) simple connectivity based post-processing. Dense image patch labeling is conducted using two methods: efficient random forest classification on image histogram, location and texture features; and more expensive (but more accurate) deep convolutional neural network classification, on larger image windows (i.e., with more spatial contexts). Over-segmented 2-D CT slices by the simple linear iterative clustering approach are adopted through model/parameter calibration and labeled at the superpixel level for positive (pancreas) or negative (non-pancreas or background) classes. The proposed method is evaluated on a data set of 80 manually segmented CT volumes, using six-fold cross-validation. Its performance equals or surpasses other state-of-the-art methods (evaluated by "leave-one-patient-out"), with a dice coefficient of 70.7% and Jaccard index of 57.9%. In addition, the computational efficiency has improved significantly, requiring a mere 6 ~ 8 min per testing case, versus ≥ 10 h for other methods. The segmentation framework using deep patch labeling confidences is also more numerically stable, as reflected in the smaller performance metric standard deviations. Finally, we implement a multi-atlas label fusion (MALF) approach for pancreas segmentation using the same data set. Under six-fold cross-validation, our bottom-up segmentation method significantly outperforms its MALF counterpart: 70.7±13.0% versus 52.51±20.84% in dice coefficients.

Early Detection of Glaucoma Disease using Image Processing

Based on the survey of various Image processing techniques, for increasing the accuracy, processing speed as well as the efficiency and reliability of the system, one will be able to use an efficient technique for the Glaucoma which is a major eye disease globally. A new and comprehensive method for an efficient detection of the disease which is a combination of techniques that exits in CDR and Blood vessel calculation with the help of SLIC superpixel classification and Hue transform for the non evasive contribution to the study and research for Glaucoma Disease is proposed. The key image handling strategies incorporate image enlistment, image combination, image segregation, highlight extraction, image improvement, morphology, design coordinating, image order, examination and factual estimation. These techniques are altogether computed using MATLAB software tool with the help of GUI to allow medical practitioners to determine whether a person is suffering from Glaucoma or not. Thus the Brightness Information of the optic cup can be directly used to determine and classify the said disease depending upon a specific and standardized threshold value.

Machine vision for counting fruit on mango tree canopies

Machine vision technologies hold the promise of enabling rapid and accurate fruit crop yield predictions in the field. The key to fulfilling this promise is accurate segmentation and detection of fruit in images of tree canopies. This paper proposes two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, and compares the use of these methods relative to existing techniques:—(i) a method based on K-nearest neighbour pixel classification and contour segmentation, and (ii) a method based on super-pixel over-segmentation and classification using support vector machines. The robustness of each algorithm was tested on multiple sets of images of mango trees acquired over a period of 3 years. These image sets were acquired under varying conditions (light and exposure), distance to the tree, average number of fruit on the tree, orchard and season. For images collected under the same conditions as the calibration images, estimated fruit numbers were within 16 % of actual fruit numbers, and the F1 measure of detection performance was above 0.68 for these methods. Results were poorer when models were used for estimating fruit numbers in trees of different canopy shape and when different imaging conditions were used. For fruit-background segmentation, K-nearest neighbour pixel classification based on colour and smoothness or pixel classification based on super-pixel over-segmentation, clustering of dense scale invariant feature transform features into visual words and bag-of-visual-word super-pixel classification using support vector machines was more effective than simple contrast and colour based segmentation. Pixel classification was best followed by fruit detection using an elliptical shape model or blob detection using colour filtering and morphological image processing techniques. Method results were also compared using precision–recall plots. Imaging at night under artificial illumination with careful attention to maintaining constant illumination conditions is highly recommended.

Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images.

This paper presents and evaluates a fully automatic method for detection of ductal carcinoma in situ (DCIS) in digitized hematoxylin and eosin (H&E) stained histopathological slides of breast tissue. The proposed method applies multi-scale superpixel classification to detect epithelial regions in whole-slide images (WSIs). Subsequently, spatial clustering is utilized to delineate regions representing meaningful structures within the tissue such as ducts and lobules. A region-based classifier employing a large set of features including statistical and structural texture features and architectural features is then trained to discriminate between DCIS and benign/normal structures. The system is evaluated on two datasets containing a total of 205 WSIs of breast tissue. Evaluation was conducted both on the slide and the lesion level using FROC analysis. The results show that to detect at least one true positive in every DCIS containing slide, the system finds 2.6 false positives per WSI. The results of the per-lesion evaluation show that it is possible to detect 80% and 83% of the DCIS lesions in an abnormal slide, at an average of 2.0 and 3.0 false positives per WSI, respectively. Collectively, the result of the experiments demonstrate the efficacy and accuracy of the proposed method as well as its potential for application in routine pathological diagnostics. To the best of our knowledge, this is the first DCIS detection algorithm working fully automatically on WSIs.

Empirical comparison of color normalization methods for epithelial-stromal classification in H and E images

Context: Color normalization techniques for histology have not been empirically tested for their utility for computational pathology pipelines. Aims: We compared two contemporary techniques for achieving a common intermediate goal – epithelial-stromal classification. Settings and Design: Expert-annotated regions of epithelium and stroma were treated as ground truth for comparing classifiers on original and color-normalized images. Materials and Methods: Epithelial and stromal regions were annotated on thirty diverse-appearing H and E stained prostate cancer tissue microarray cores. Corresponding sets of thirty images each were generated using the two color normalization techniques. Color metrics were compared for original and color-normalized images. Separate epithelial-stromal classifiers were trained and compared on test images. Main analyses were conducted using a multiresolution segmentation (MRS) approach; comparative analyses using two other classification approaches (convolutional neural network [CNN], Wndchrm) were also performed. Statistical Analysis: For the main MRS method, which relied on classification of super-pixels, the number of variables used was reduced using backward elimination without compromising accuracy, and test - area under the curves (AUCs) were compared for original and normalized images. For CNN and Wndchrm, pixel classification test-AUCs were compared. Results: Khan method reduced color saturation while Vahadane reduced hue variance. Super-pixel-level test-AUC for MRS was 0.010–0.025 (95% confidence interval limits ± 0.004) higher for the two normalized image sets compared to the original in the 10–80 variable range. Improvement in pixel classification accuracy was also observed for CNN and Wndchrm for color-normalized images. Conclusions: Color normalization can give a small incremental benefit when a super-pixel-based classification method is used with features that perform implicit color normalization while the gain is higher for patch-based classification methods for classifying epithelium versus stroma.

Superpixel-based image noise variance estimation with local statistical assessment

Noise estimation is fundamental and essential in a wide variety of computer vision, image, and video processing applications. It provides an adaptive mechanism for many restoration algorithms instead of using fixed values for the setting of noise levels. This paper proposes a new superpixel-based framework associated with statistical analysis for estimating the variance of additive Gaussian noise in digital images. The proposed approach consists of three major phases: superpixel classification, local variance computation, and statistical determination. The normalized cut algorithm is first adopted to effectively divide the image into a set of superpixel regions, from which the noise variance is computed and estimated. Subsequently, the Jarque–Bera test is used to exclude regions that are not normally distributed. The smallest standard deviation in the remaining regions is finally selected as the estimation result. A wide variety of noisy images with various scenarios were used to evaluate this new noise estimation algorithm. Experimental results indicated that the proposed framework provides accurate estimations across various noise levels. Comparing with many state-of-the-art methods, our algorithm strikes a good compromise between low-level and high-level noise estimations. It is suggested that the proposed method is of potential in many computer vision, image, and video processing applications that require automation.

Chronological Advancement in Image Processing from Lime Stone Mofits to Superpixel Classification

Image processing is widely used method for representation and extraction of information. To get the information from the areas and objects which are not possible to be physically contact directly remote sensing image processing is used. While extracting the information from the remote sensed images the major issues that affect the accuracy of the classification is the presence of mixed pixels (reflecting more than one spectral signature) in image. In this study, we have summarized the various eras of image processing from its origin and also give the various techniques and methods used to solve the problem of mixed pixels. In this study, a new algorithm is proposed to solve the mixed pixel problem by using PSO based Fuzzy C-mean and Biogeographical Based Optimization (BBO). This proposed algorithm will improve the classification of mixed pixels as compared to the literature.

Robust multi-scale superpixel classification for optic cup localization

This paper presents an optimal model integration framework to robustly localize the optic cup in fundus images for glaucoma detection. This work is based on the existing superpixel classification approach and makes two major contributions. First, it addresses the issues of classification performance variations due to repeated random selection of training samples, and offers a better localization solution. Second, multiple superpixel resolutions are integrated and unified for better cup boundary adherence. Compared to the state-of-the-art intra-image learning approach, we demonstrate improvements in optic cup localization accuracy with full cup-to-disc ratio range, while incurring only minor increase in computing cost.

Segmentation of anterior cruciate ligament in knee MR images using graph cuts with patient-specific shape constraints and label refinement

We propose a graph-cut-based segmentation method for the anterior cruciate ligament (ACL) in knee MRI with a novel shape prior and label refinement. As the initial seeds for graph cuts, candidates for the ACL and the background are extracted from knee MRI roughly by means of adaptive thresholding with Gaussian mixture model fitting. The extracted ACL candidate is segmented iteratively by graph cuts with patient-specific shape constraints. Two shape constraints termed fence and neighbor costs are suggested such that the graph cuts prevent any leakage into adjacent regions with similar intensity. The segmented ACL label is refined by means of superpixel classification. Superpixel classification makes the segmented label propagate into missing inhomogeneous regions inside the ACL. In the experiments, the proposed method segmented the ACL with Dice similarity coefficient of 66.47±7.97%, average surface distance of 2.247±0.869, and root mean squared error of 3.538±1.633, which increased the accuracy by 14.8%, 40.3%, and 37.6% from the Boykov model, respectively.

CUP and DISC OPTIC Segmentation Using Optimized Superpixel Classification for Glaucoma Screening

Glaucoma, an incurable disease related to eyes which results in loss of the vision. Identifying this disease within in a proper period of time is most important, since it cannot be cured. The important aspect of this paper is to detect glaucoma at initial stages. Segmentation in the optic disc necessitates the differentiation of each super pixel by employing Histograms, centre surround statistics. Information location in merged with the above methods in increasing the performance of optic cup segmentation. Optic disc and optic cups are employed to evaluate cup to disc ratio of the disease identified. Neural network is used to extract the patterns and also to detect glaucomatous cells that are too complex to be noticed by either humans or other computer techniques.

A novel outdoor scene-understanding framework for unmanned ground vehicles with 3D laser scanners

Outdoor scene understanding plays a key role for unmanned ground vehicles (UGVs) to navigate in complex urban environments. This paper presents a novel 3D scene-understanding framework for UGVs to handle uncertain and changing lighting conditions outdoors. A 2D bearing angle (BA) image is deployed to perform scene understanding so that the computational burden in the process of segmentation and classification of the 3D laser point cloud can be reduced. An improved super-pixel algorithm is used for fast 3D scene segmentation, and then the Gentle–Adaboost algorithm is utilized to perform super-pixel patch classification using the texture features of the Gray Level Co-occurrence Matrix. All false classification results in the uncertain super-pixel patches of BA images are transformed back to raw 3D laser points and a re-classification is conducted to refine the 3D scene understanding for UGVs. The results from a real laser dataset taken from a large-scale campus environment show the validity and robust performance of the proposed approach, in comparison with the results from Korea Advanced Institute of Science and Technology dataset.

Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening

Glaucoma is a chronic eye disease that leads to vision loss. As it cannot be cured, detecting the disease in time is important. Current tests using intraocular pressure (IOP) are not sensitive enough for population based glaucoma screening. Optic nerve head assessment in retinal fundus images is both more promising and superior. This paper proposes optic disc and optic cup segmentation using superpixel classification for glaucoma screening. In optic disc segmentation, histograms, and center surround statistics are used to classify each superpixel as disc or non-disc. A self-assessment reliability score is computed to evaluate the quality of the automated optic disc segmentation. For optic cup segmentation, in addition to the histograms and center surround statistics, the location information is also included into the feature space to boost the performance. The proposed segmentation methods have been evaluated in a database of 650 images with optic disc and optic cup boundaries manually marked by trained professionals. Experimental results show an average overlapping error of 9.5% and 24.1% in optic disc and optic cup segmentation, respectively. The results also show an increase in overlapping error as the reliability score is reduced, which justifies the effectiveness of the self-assessment. The segmented optic disc and optic cup are then used to compute the cup to disc ratio for glaucoma screening. Our proposed method achieves areas under curve of 0.800 and 0.822 in two data sets, which is higher than other methods. The methods can be used for segmentation and glaucoma screening. The self-assessment will be used as an indicator of cases with large errors and enhance the clinical deployment of the automatic segmentation and screening.