Superpixel Algorithm Research Articles

Polarimetric SAR Image Classification Based on Ensemble Dual-Branch CNN and Superpixel Algorithm

Recently, convolutional neural networks (CNNs) have been successfully utilized in polarimetric synthetic aperture radar (PolSAR) image classification and obtained promising results. However, most CNN-based classification methods require a large number of labeled samples and it is difficult to obtain sufficient labeled samples. For this reason, an ensemble dual-branch CNN (EDb-CNN) is proposed for PolSAR image classification with small samples. First, to solve the problem of the small sample in PolSAR image classification, a new data enhancement method based on the superpixel algorithm is proposed to expand the number of labeled samples. Second, to obtain different scales of features from PolSAR images, a Db-CNN model is proposed. This model contains two parallel CNN structures. One CNN branch is used to extract the polarization features from the complex coherency matrix. The other branch is utilized to extract the spatial features based on weighted spatial neighborhood. On the top of these two branches, a feature fusion model is adopted to combine these two deep features, and a weighted loss function is employed to improve the learning procedure. Then, the ensemble learning algorithm is used for each CNN branch and Db-CNN network to obtain the better classification results. Finally, a postprocess algorithm based on the superpixel algorithm is proposed to improve the consistency of classification results. Experiments on two PolSAR datasets show that the proposed method achieves a much better performance than other classification methods, especially when only a few labeled samples are available.

PolSAR-SSN: An End-to-End Superpixel Sampling Network for PolSAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) image classification is one of the fundamental research areas in remote sensing. Superpixels can provide boundary constraint information and are widely used in PolSAR image interpretation. However, traditional machine learning superpixel algorithms have many limitations for PolSAR image interpretation. Pseudo-color images are usually used as the superpixel algorithm inputs, and the loss of polarimetric information will decrease the performance. In addition, the superpixel algorithms are difficult to incorporate into state-of-the-art deep learning models and cannot be trained in an end-to-end manner. In this letter, a trainable end-to-end deep superpixel network is proposed for PolSAR image classification. The inputs of the proposed method can be any low/middle-level polarimetric features of a PolSAR image and the rich polarimetric feature representation can be learned. The produced superpixels of the proposed method are more concentrated near the land cover boundaries and can significantly improve the performance of PolSAR image classification. Experimental results show that the overall accuracies of the proposed method are approximately 2.57% and 1.44% higher than traditional superpixel algorithms on two PolSAR datasets and surpass some well-known deep learning methods.

Multi-scale super-pixels-based passive forensics for copy-move forgery using local characteristics

Copy-move forgery is the most common kind of tampering technique for digital images. This paper presents a novel hybrid approach, which uses the Speeded-Up Robust Features (SURF) point and characteristics of Local Feature Regions (LFRs) matching. First, the multi-scale super-pixels algorithm adaptively divides the suspect image into irregular blocks according to the texture level of host images. Then, the improved SURF detector is adopted in extracting feature points from each super-pixel and the feature point threshold is related to the entropy of each super-pixel block. Next, LFRs are defined, and a robust feature descriptor is extracted from each LFR as a vector field. Last, the matching LFRs are found by using Euclidean locality sensitive hashing; the removal of falsely matched pairs is realised by using the Random Sample Consensus (RANSAC) algorithm. Comparing with the leading-edge block-matching methods and point-based methods, our method can produce far better detection results.

Superpixel Sizes using Topology Preserved Regular Superpixel Algorithm and their Impacts on Interactive Segmentation

Interactive Image Segmentation is a type of semi-automated segmentation that uses user input to extract the object of interest. It is possible to speed up and improve the end result of segmentation by using pre-processing steps. The use of superpixels is an example of a pre-processing step. A superpixel is a collection of pixels with similar properties such as texture and colour. Previous research was conducted to assess the impact of the number of superpixels (based on SEEDS superpixel aglorithms) required to achieve the best segmentation results. The study, however, only examined one type of input (strokes) and a small number of images. As a result, the goal of this study is to extend previous work by performing interactive segmentation with input strokes and a combination of bounding box and strokes on images from Grabcut image data sets generated by Topology preserved regular superpixel (TPRS). Based on our findings, an image with 1000 to 2500 superpixels and a combination of bounding box and strokes will help the interactive segmentation algorithm produce a good segmentation result. Finally, the size of the superpixels would influence the final segmentation results as well as the input type.

Improved Superpixels Generation Algorithm for Qualified Graph-Based Technique

Hyperspectral images (HSIs) represent an important source of information in the remote sensing field. Indeed, HSIs, which collect data in many spectral bands, are more simple interpretable and provide a detailed information about interest areas. However, hyperspectral imaging systems generate a huge amount of redundant data and an important level of noise. Dimensionality reduction is an important task that attempts to reduce dimensionality and remove noise so as to enhance the accuracy of remote sensing applications. The first dimensionality reduction approaches date back to 1970s, and various model-based methods have been proposed since these years. This field has known an increasing attention by the suggestion of graph based models that have yielded promising results. While graph based approaches generate considerable outputs, these models require often an important processing time to handle data. In this work, we aim to reduce the computational burden of a promising graph based method called the Modified Schroedinger Eigenmap Projections (MSEP). In this respect, we suggest an efficient superpixel algorithm, called Improved Simple Linear Iterative Clustering (Improved SLIC), to lessen the heavy computational load of the MSEP method. The proposed approach exploits the superpixels as inputs instead of pixels; and then runs the MSEP algorithm. While respecting the HSIs properties, the proposed scheme illustrates that the MSEP method can be performed with computational efficiency.

Fast SAR Image Segmentation With Deep Task-Specific Superpixel Sampling and Soft Graph Convolution

Since the number of superpixels is lower than that of pixels, superpixels can substantially speed up subsequent processing steps and have been widely used in synthetic aperture radar (SAR) image segmentation. However, in most of the existing superpixel-wise segmentation algorithms, superpixel prediction is an isolated preprocessing step and is independent of the segmentation task. The performance of the segmentation results is determined by the accuracy of superpixels. Once superpixels are generated, their shape cannot be changed in the following segmentation stage, even if the same superpixels contain pixels of different landcovers. To address this, we propose an end-to-end trainable superpixel-wise segmentation method for single-polarization SAR images. First, we design a differentiable boundary-ware clustering method for estimating task-specific superpixels. Instead of the hard association between pixels and superpixels in the existing superpixel algorithms, this method introduces the soft association map to make the clustering differentiable. Hence, it can be implemented using a simple deep fully convolutional network. In the segmentation part, we propose a novel soft graph convolution network (Soft-GCN), which takes the association map as input and performs superpixel-wise segmentation. The advantage of our method is that superpixel generation and graph convolution parts can be trained under a unified framework, until two parts obtain the optimum parameters. In the training process, it can adaptively adjust the shape of the superpixels according to the segmentation results, ensuring the superpixels correctly adhere the boundaries. Experimental results with simulated and real SAR images demonstrate that our method outperforms other state-of-the-art segmentation algorithms, while also being faster.

Automatic stomata recognition and measurement based on improved YOLO deep learning model and entropy rate superpixel algorithm

The traditional methods of analyzing stomatal traits are mostly manual observation and measurement. These methods are time-consuming, labor-intensive, and inefficient. Some methods have been proposed for the automatic recognition and counting of stomata, however most of those methods could not complete the automatic measurement of stomata parameters at the same time. Some non-deep learning methods could automatically measure the parameters of stomata, but they could not complete the automatic recognition and detection of stomata. In this paper, a deep learning-based method was proposed for automatically identifying, counting and measuring stomata of maize (Zea mays L.) leaves at the same time. An improved YOLO (You Only Look Once) deep learning model was proposed to identify stomata of maize leaves automatically, and an entropy rate superpixel algorithm was used for the accurate measurement of stomatal parameters. According to the characteristics of the stomata images data set, the network structure of YOLOv5 was modified, which greatly reduced the training time without affecting the recognition performance. The predictor in YOLO deep learning model was optimized, which reduced the false detection rate. At the same time, the 16-fold and 32-fold down-sampling layers were simplified according to the characteristics of stomatal objects, which improved the recognition efficiency. Experimental results showed that the recognition precision of the improved YOLO deep learning model reached 95.3% on the maize leaves stomatal data set, and the average accuracy of parameter measurement reached 90%. The proposed method could fully automatically complete the recognition, counting and measurement of stomata of plants, which can help agricultural scientists and botanists to conduct large-scale researches of stomatal morphology, structure and physiology, as well as the researches combined with genetic analysis or molecular-level analysis.

Deep Learning-Based Maritime Environment Segmentation for Unmanned Surface Vehicles Using Superpixel Algorithms

Unmanned surface vehicles (USVs) are receiving increasing attention in recent years from both academia and industry. To make a high-level autonomy for USVs, the environment situational awareness is a key capability. However, due to the richness of the features in marine environments, as well as the complexity of the environment influenced by sun glare and sea fog, the development of a reliable situational awareness system remains a challenging problem that requires further studies. This paper, therefore, proposes a new deep semantic segmentation model together with a Simple Linear Iterative Clustering (SLIC) algorithm, for an accurate perception for various maritime environments. More specifically, powered by the SLIC algorithm, the new segmentation model can achieve refined results around obstacle edges and improved accuracy for water surface obstacle segmentation. The overall structure of the new model employs an encoder–decoder layout, and a superpixel refinement is embedded before final outputs. Three publicly available maritime image datasets are used in this paper to train and validate the segmentation model. The final output demonstrates that the proposed model can provide accurate results for obstacle segmentation.

An automatic phytopathometry system for chlorosis and necrosis severity evaluation of asian soybean rust infection

The development of soybean cultivars resistant to Phakopsora pachyrhizi (also known as Asian Soybean Rust (ASR)) may be achieved through genetic breeding. In the stage of selecting the resistant genotypes, one factor that can be used is to observe visually the severity of the rust symptoms upon leaflets of sampled soybean plants. By being able to quantify the exact amounts of regions with chlorosis, necrosis, and unaffected of each sample an automatic method is developed to grade rust severity and to classify the samples accordingly. The proposed approach uses input images of soybean leaflets, in a range from healthy to different levels of ASR severity, sampled from controlled experiments in a farm in southeast Brazil. Images are then processed by channel transformation (RGB to CIELab), channel distribution analysis, and a clustering algorithm in order to segment the regions in three main areas. Those regions represent the main visual symptoms of the ASR severity, but still with ill-defined transitions between them. A post-processing superpixel algorithm is leveraged upon those results to arrive at a first rust intervals classification. A necrosis’ neighborhood analysis is proposed to compute the intermediate areas and to include them in a final rust index. Results are given upon four different experiments: 1) 200 acquired image samples taken directly from the field; 2) 67 detached leaflets at different evolved periods of infection; 3) 50 images from a previous open database, and 4) 10 images with the standards for diagrammatic scales. By grading rust severity, and separating chlorosis, and necrosis indexes this solution contributes to efforts in managing ASR in crops, and in breeding programs.

Machine learning and SLIC for Tree Canopies segmentation in urban areas

This paper presents a novel approach combining the Simple Linear Iterative Clustering (SLIC) superpixel algorithm with a Convolutional Neural Network (CNN) over high-resolution imagery to detect trees in a typical urban environment of the Brazilian Cerrado biome. Our analysis approach for better results uses the deep learning classifier ResNet-50, with a variation in the batch size and five traditional shallow learning methods. The results were processed and avaliated using mainly accuracy as a metric, but we show that the accuracy poorly represent the overlap between the manual annotation and the resulting map, so we bring the IoU metric results to better show the Network learning classification maps results. The combined SLIC algorithm and the best CNN resulted in an accuracy of 93.20%, IoU of 0.700 and a variation of 1% for difference in the area of tree canopies if compared to our labels, while the best shallow presented an accuracy of 91.70%, IoU of 0.200 and a variation in area of 12.52%. Demonstrating that the proposed CNN method is suitable for segmenting trees from high-resolution images acquired over urban environments. The segmentation with SLIC and CNN can provide very useful results for urban management using low cost RGB images. Such outcomes are of great interest for local managers since reliable maps showing the spatial distribution of trees in urban areas are often required for many applications.

Automatic mapping of national surface water with OpenStreetMap and Sentinel-2 MSI data using deep learning

Large-scale mapping activities can benefit from the vastly increasing availability of earth observation (EO) data, especially when combined with volunteered geographical information (VGI) using machine learning (ML). High-resolution maps of inland surface water bodies are important for water supply and natural disaster mitigation as well as for monitoring, managing, and preserving landscapes and ecosystems. In this paper, we propose an automatic surface water mapping workflow by training a deep residual neural network (ResNet) based on OpenStreetMap (OSM) data and Sentinel-2 multispectral data, where the Simple Non-Iterative Clustering (SNIC) superpixel algorithm was employed for generating object-based training samples. As a case study, we produced an open surface water layer for Germany using a national ResNet model at a 10 m spatial resolution, which was then harmonized with OSM data for final surface water products. Moreover, we evaluated the mapping accuracy of our open water products via conducting expert validation campaigns, and comparing to existing water products, namely the WasserBLIcK and Global Surface Water Layer (GSWL). Using 4,600 validation samples in Germany, the proposed model (ResNet+SNIC) achieved an overall accuracy of 86.32% and competitive detection rates over the WasserBLIcK (87.47%) and GSWL (98.61%). This study provides comprehensive insights into how to best explore the synergy of VGI and ML of EO data in a large-scale surface water mapping task.

Automatic Superpixel-Based Clustering for Color Image Segmentation Using q-Generalized Pareto Distribution under Linear Normalization and Hunger Games Search

Superixel is one of the most efficient of the image segmentation approaches that are widely used for different applications. In this paper, we developed an image segmentation based on superpixel and an automatic clustering using q-Generalized Pareto distribution under linear normalization (q-GPDL), called ASCQPHGS. The proposed method uses the superpixel algorithm to segment the given image, then the Density Peaks clustering (DPC) is employed to the results obtained from the superpixel algorithm to produce a decision graph. The Hunger games search (HGS) algorithm is employed as a clustering method to segment the image. The proposed method is evaluated using two different datasets, collected form Berkeley segmentation dataset and benchmark (BSDS500) and standford background dataset (SBD). More so, the proposed method is compared to several methods to verify its performance and efficiency. Overall, the proposed method showed significant performance and it outperformed all compared methods using well-known performance metrics.

Region extraction method for skin grafts via image analysis

ABSTRACT Dermal grafts are used for patients with severe burn injuries whose natural healing abilities are limited or exhausted. This paper proposes a method to segment burnt and grafted areas from images with meshed grafts. First, RGB colour input images are converted to images in the L*a*b* colour space. The transformed images are then segmented with the simple linear iterative clustering superpixel algorithm. Subsequently, the burnt and grafted areas are classified with a support vector machine, and the output area size is modified for all burnt areas. The segmentation results are compared with the areas identified by two dermatologists based on the concordance rate (CR), positive predictive value (PPV), sensitivity, and F-measure values. The proposed segmentation method yielded CRs equal to 84.3 % and 85.9 %, PPVs equal to 87.2 % and 87.9 %, sensitivities equal to 84.7 % and 86.4 %, and F-measure values equal to 85.8 % and 87.0 %. This study developed a segmentation method for classifying burnt and grafted areas from images with meshed grafts. The segmented areas obtained were significantly consistent with those identified by the two dermatologists.

Fast and accurate superpixel algorithms for 360[formula omitted] images

Superpixels are fundamental in many visual computing applications, and most existing algorithms are designed to work with pinhole-based images. However, immersive applications are gaining visibility with the growing number of devices for capturing and visualizing 360∘ media. This paper introduces two fast and accurate superpixel algorithms tailored to the spherical domain. The methods are assessed under common figures of merit, and a benchmark annotated dataset. Additionally, an annotation-free evaluation metric for exploiting larger datasets is introduced. The methods introduced in this paper perform quantitatively close to or better than state-of-the-art approaches and are faster.

Cutset-type Possibilistic C-means Clustering Algorithms Based on Semi-supervised Information

Cutset-type Possibilistic C-Means clustering (C-PCM) algorithm can significantly reduce the coincident clustering phenomenon of the Possibilistic C-Means clustering (PCM) algorithm by introducing the cut-set concept into the PCM. The C-PCM also has strong robustness to noise and outliers. However, the C-PCM still suffers from the center migration problem for datasets with small targets. In order to solve this problem, a Semi-Supervised Cutset-type Possibility C-Means (SS-C-PCM) clustering algorithm is proposed by introducing the semi-supervised learning mechanism into the objective function of the C-PCM and utilizing some prior information to guide the clustering process. Meanwhile, in order to improve the segmentation efficiency and accuracy of color images, a differential evolutionary superpixel-based Semi-Supervised Cutset-type Possibilistic C-Means (desSS-C-PCM) clustering algorithm is proposed. In the desSS-C-PCM, the Differential Evolutionary Superpixel(DES) algorithm is used to obtain the spatial neighborhood information of an image, which is integrated into the objective function of the semi-supervised C-PCM to improve the segmentation quality. Simultaneously, the color histogram is used to reconstruct the new objective function to reduce the computational complexity of the algorithm. Several experiments of artificial data clustering and color image segmentation show that the proposed algorithm can effectively improve the clustering effect of datasets with small targets and the execution efficiency compared with several related algorithms.

Two-level K-nearest neighbors approach for invasive plants detection and classification

Invasive plants (IPs) are considered major threat to biodiversity and ecosystems due to their potential to compete for resources with other plants or crops. Timely and effective detection is important to prevent and control their growth. Recent advancements in unmanned aerial systems (UAS) have brought an effective, inexpensive, and non-invasive approach to monitoring plants. The use of UAS is promising in the accurate and rapid identification of IPs in the field. However, challenges exist when using UAS for IP detection as they generate a huge number of images that must be interpreted by researchers manually. This time-consuming and error-prone process significantly limits the applicability of UAS in a variety of environmental and agriculture monitoring applications. To reduce processing time and increase detection accuracy, we develop an automatic and intelligent detection approach to identify IPs with high levels of efficiency and accuracy from images of the monitoring areas captured by UAS. First, the original field images are segmented into patch images using the simple linear iterative clustering (SLIC) superpixels algorithm. Then a novel two-level K-nearest neighbors (TLKNN) algorithm is proposed to classify the patch images into different IP categories. In it, a deep convolutional neural network (CNN) is used to extract the features from the patch image and a kernel map function is utilized to obtain the kernel features. Two feature sets are fed to train two K-nearest neighbor (KNN) models. Finally, the final classification is determined by the selection results by KNN models. In the experiments, two public UAV image sets are employed to train and verify the proposed method. Several metrics such as accuracy, recall, precision, and F1-score, are used to evaluate the performance. The proposed TLKNN is compared with two newly proposed convolutional neural network (CNN) methods on the same datasets. Experimental results indicate the proposed TLKNN algorithm achieves better classification performance and reaches the highest accuracy, recall rate, precision rate, and F1-score. The improved performance not only means that IPs can be detected more accurately and automatically, but also that there are environmental benefits in developing an intelligent strategy for IP prevention and control.

Swarm Intelligence-Based Methodology for Scanning Electron Microscope Image Segmentation of Solid Oxide Fuel Cell Anode

Segmentation of images from scanning electron microscope, especially multiphase, poses a drawback in their microstructure quantification process. The labeling process must be automatized due to the time consumption and irreproducibility of the manual labeling procedure. Here we show a swarm intelligence-driven filtration methodology performed on raw solid oxide fuel cell anode’s material images to improve the segmentation methods’ performance. The methodology focused on two significant parts of the segmentation process, which are filtering and labeling. During the first one, the images underwent filtering by applying a series of filters, whose operation parameters were determined using Particle Swarm Optimization upon a dedicated cost function. Next, Seeded Region Growing, k-Means Clustering, Multithresholding, and Simple Linear Iterative Clustering Superpixel algorithms were utilized to label the filtered images’ regions into consecutive phases in the microstructure. The improvement was presented for three different metrics: the Misclassification Ratio, Structural Similarity Index Measure, and Mean Squared Error. The obtained distribution of metrics’ performances was based on 200 images, with and without filtering. Results indicate an improvement up to 29%, depending on the metric and method used. The presented work contributes to the ongoing efforts to automatize segmentation processes fully for an increasing number of tomographic measurements, particularly in solid oxide fuel cell research.

PolSAR Data-Based Land Cover Classification Using Dual-Channel Watershed Region-Merging Segmentation and Bagging-ELM

Land cover classification based on spaceborne synthetic aperture radar (SAR) imagery is an emerging technology with many applications. Machine learning and related ensemble models have attracted much attention to this technology. This letter presents a new object-oriented land cover classification based on Polarimetric SAR data that integrate the dual-channel (DC) Watershed region-merging segmentation algorithm and a Bagging-based ensemble extreme learning machine (Bagging-ELM) image object classifier. The advantage of the DC segmentation algorithm is verified by comparing it with the well-known simple linear iterative clustering (SLIC) superpixel algorithm. Following the DC segmentation, the Bagging-ELM is implemented to perform the classification of image objects. Besides, two other classification methods (single ELM and ensembled-tree algorithms) are used for performance comparison. The quantitative analysis with respect to ground-truth information available for the test sites shows that the proposed approach can achieve noticeable improvements in classification accuracy and outperforms the other two algorithms, showing the potential for spaceborne-SAR-based land cover classification task in real applications.

A novel augmented reality for hidden organs visualisation in surgery: enhanced super-pixel with sub sampling and variance adaptive algorithm

In recent years, Augmented Reality (AR) has gained more attention as an effective tool in medical surgeries. The potentials of using AR in the medical field can change conventional medical procedures. However, the technology still facing fundamental challenges, especially hidden organs, for example, the organs behind the bowel and liver. The surgeries in these areas lack accuracy in the visualization of the soft tissues behind the bowel and liver like the uterus and gall bladder. This research aims to improve the accuracy of visualisation and the processing time of the augmented video. The proposed system consists of an enhanced super-pixel algorithm with variance weight adaptation and subsampling method. The simulation studies show significant improvements in visualization accuracy and a reduction in processing time. The results show reduced visualisation error by 0.23 mm. It provides better accuracy of the video in terms of visualization error from 1.58 ~ 1.83 mm to 1.35 ~ 1.60 mm, and the processing time decreases from 50 ~ 58 ms/frames to 40 ~ 48 ms/frames. The proposed system \ focused on the pixel refinement for the 3d reconstruction of the soft tissue, which helps solve the issue of visualising the bowel and liver in an augmented video.

SuperHistopath: A Deep Learning Pipeline for Mapping Tumor Heterogeneity on Low-Resolution Whole-Slide Digital Histopathology Images.

High computational cost associated with digital pathology image analysis approaches is a challenge towards their translation in routine pathology clinic. Here, we propose a computationally efficient framework (SuperHistopath), designed to map global context features reflecting the rich tumor morphological heterogeneity. SuperHistopath efficiently combines i) a segmentation approach using the linear iterative clustering (SLIC) superpixels algorithm applied directly on the whole-slide images at low resolution (5x magnification) to adhere to region boundaries and form homogeneous spatial units at tissue-level, followed by ii) classification of superpixels using a convolution neural network (CNN). To demonstrate how versatile SuperHistopath was in accomplishing histopathology tasks, we classified tumor tissue, stroma, necrosis, lymphocytes clusters, differentiating regions, fat, hemorrhage and normal tissue, in 127 melanomas, 23 triple-negative breast cancers, and 73 samples from transgenic mouse models of high-risk childhood neuroblastoma with high accuracy (98.8%, 93.1% and 98.3% respectively). Furthermore, SuperHistopath enabled discovery of significant differences in tumor phenotype of neuroblastoma mouse models emulating genomic variants of high-risk disease, and stratification of melanoma patients (high ratio of lymphocyte-to-tumor superpixels (p = 0.015) and low stroma-to-tumor ratio (p = 0.028) were associated with a favorable prognosis). Finally, SuperHistopath is efficient for annotation of ground-truth datasets (as there is no need of boundary delineation), training and application (~5 min for classifying a whole-slide image and as low as ~30 min for network training). These attributes make SuperHistopath particularly attractive for research in rich datasets and could also facilitate its adoption in the clinic to accelerate pathologist workflow with the quantification of phenotypes, predictive/prognosis markers.