Superpixel Image Research Articles

Unmanned aerial vehicle image stitching based on multi‐region segmentation

AbstractUnmanned aerial vehicle (UAV) image stitching is a key technology for aerial remote sensing applications. Most existing image stitching methods based on optimal seamline searching algorithms can eliminate defects such as ghosting and distortion in stitched images, which unfortunately suffer from the problem that the seamline may cross those regions with significant geometric misalignment between different images. Therefore, a novel image stitching method based on multi‐region image segmentation is proposed. The algorithm starts by performing a multi‐scale morphological reconstruction in the overlapping regions between UAV images to obtain superpixel images with precise contours. Then, the fast density peaks clustering based on K‐nearest neighbours is applied to automatically determine the clustering centres and the number of clusters. By constructing a cost function, an energy map is generated in the overlapping regions between UAV images. Finally, the optimal seamline can be determined with a graph‐cut method. Compared to several popular image stitching algorithms in real experiments, the proposed method can essentially prevent the seamline from crossing significant ground objects to ensure the integrity of structural objects while achieving satisfactory accuracy and efficiency during the UAV image stitching process.

Magnetic Resonance Images Segmentation of Multifidus based on Dense-unet and Superpixel.

Lumbar disc herniation (LDH) is a common clinical condition causing lower back and leg pain. Accurate segmentation of the lumbar discs is crucial for assessing and diagnosing LDH. Magnetic resonance imaging (MRI) can reveal the condition of articular cartilage. However, manual segmentation of MRI images is burdensome for physicians and needs to be more efficient. In this study, we propose a method that combines UNet and superpixel segmentation to address the problem of loss of detailed information in the feature extraction phase, leading to poor segmentation results at object edges. The aim is to provide a reproducible solution for diagnosing patients with lumbar disc herniation. We suggest using the network structure of UNet. Firstly, dense blocks are inserted into the UNet network, and training is performed using the Swish activation function. The Dense-UNet model extracts semantic features from the images and obtains rough semantic segmentation results. Then, an adaptive-scale superpixel segmentation algorithm is applied to segment the input images into superpixel images. Finally, high-level abstract semantic features are fused with the detailed information of the superpixels to obtain edge-optimized semantic segmentation results. Evaluation of a private dataset of multifidus muscles in magnetic resonance images demonstrates that compared to other segmentation algorithms, this algorithm exhibits better semantic segmentation performance in detailed areas such as object edges. Compared to UNet, it achieves a 9.5% improvement in the Dice Similarity Coefficient (DSC) and an 11.3% improvement in the Jaccard Index (JAC). The experimental results indicate that this algorithm improves segmentation performance while reducing computational complexity.

Saliency Detection Based on Multiple-Level Feature Learning.

Finding the most interesting areas of an image is the aim of saliency detection. Conventional methods based on low-level features rely on biological cues like texture and color. These methods, however, have trouble with processing complicated or low-contrast images. In this paper, we introduce a deep neural network-based saliency detection method. First, using semantic segmentation, we construct a pixel-level model that gives each pixel a saliency value depending on its semantic category. Next, we create a region feature model by combining both hand-crafted and deep features, which extracts and fuses the local and global information of each superpixel region. Third, we combine the results from the previous two steps, along with the over-segmented superpixel images and the original images, to construct a multi-level feature model. We feed the model into a deep convolutional network, which generates the final saliency map by learning to integrate the macro and micro information based on the pixels and superpixels. We assess our method on five benchmark datasets and contrast it against 14 state-of-the-art saliency detection algorithms. According to the experimental results, our method performs better than the other methods in terms of F-measure, precision, recall, and runtime. Additionally, we analyze the limitations of our method and propose potential future developments.

Deep learning-aided 3D proxy-bridged region-growing framework for multi-organ segmentation

Accurate multi-organ segmentation in 3D CT images is imperative for enhancing computer-aided diagnosis and radiotherapy planning. However, current deep learning-based methods for 3D multi-organ segmentation face challenges such as the need for labor-intensive manual pixel-level annotations and high hardware resource demands, especially regarding GPU resources. To address these issues, we propose a 3D proxy-bridged region-growing framework specifically designed for the segmentation of the liver and spleen. Specifically, a key slice is selected from each 3D volume according to the corresponding intensity histogram. Subsequently, a deep learning model is employed to pinpoint the semantic central patch on this key slice, to calculate the growing seed. To counteract the impact of noise, segmentation of the liver and spleen is conducted on superpixel images created through proxy-bridging strategy. The segmentation process is then extended to adjacent slices by applying the same methodology iteratively, culminating in the comprehensive segmentation results. Experimental results demonstrate that the proposed framework accomplishes segmentation of the liver and spleen with an average Dice Similarity Coefficient of approximately 0.93 and a Jaccard Similarity Coefficient of around 0.88. These outcomes substantiate the framework's capability to achieve performance on par with that of deep learning methods, albeit requiring less guidance information and lower GPU resources.

An unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization

ABSTRACT In the field of remote sensing, using a large amount of labeled image data to supervise the training of fully convolutional networks for the semantic segmentation of images is expensive. However, using a small amount of labeled data can lead to reduced network performance. This paper proposes an unsupervised semantic segmentation method that combines the ImSE-Net model with SLICm superpixel optimization. First, the ImSE-Net model is used to extract semantic features from the image to obtain rough semantic segmentation results. Then, the SLICm superpixel segmentation algorithm is used to segment the input image into superpixel images. Finally, an unsupervised semantic segmentation model (UGLS) is used to combine high-level abstract semantic features with detailed information on superpixels to obtain edge-optimized semantic segmentation results. Experimental results show that compared with other semantic segmentation algorithms, our method more effectively handles unbalanced areas, such as object boundaries, and achieves better segmentation results, with higher semantic consistency.

Brain MR Image Classification Using Superpixel-Based Deep Transfer Learning.

Nowadays, brain MR (Magnetic Resonance) images are widely used by clinicians to examine the brain's anatomy to look into various pathological conditions like cerebrovascular incidents and neuro-degenerative diseases. Generally, these diseases can be identified with the MR images as "normal" and "abnormal" brains in a two-class classification problem or as disease-specific classes in a multi-class problem. This article presents an ensemble transfer learning-inspired deep architecture that uses the simple linear iterative clustering (SLIC)-based superpixel algorithm along with convolutional neural network (CNN) to classify the MR images as normal or abnormal. Superpixel algorithm segments the input MR images into clusters of regions defined by similarity measures using perceptual feature space. These superpixel images are beneficial as they can provide a compact and meaningful role in computationally demanding applications. The superpixel images are then fed to the deep convolutional neural network (CNN) to classify the images. Three brain MR image datasets, NITR-DHH, DS-75, and DS-160, are used to conduct the experimentation. Through the use of deep transfer learning, the model achieves performance accuracy of 88.15% (NITR-DHH), 98.15% (DS-160), and 98.33% (DS-75) even with the small-scale medical image dataset. The experimentally obtained results demonstrate that the proposed method is promising and efficient for clinical applications for diagnosing different brain diseases via MR images.

Fast Automatic Fuzzy C-Means Knitting Pattern Color-Separation Algorithm Based on Superpixels.

Patterns entered into knitting CAD have thousands or tens of thousands of different colors, which need to be merged by color-separation algorithms. However, for degraded patterns, the current color-separation algorithms cannot achieve the desired results, and the clustering quantity parameter needs to be managed manually. In this paper, we propose a fast and automatic FCM color-separation algorithm based on superpixels, which first uses the Real-ESRGAN blind super-resolution network to clarify the degraded patterns and obtain high-resolution images with clear boundaries. Then, it uses the improved MMGR-WT superpixel algorithm to pre-separate the high-resolution images and obtain superpixel images with smooth and accurate edges. Subsequently, the number of superpixel clusters is automatically calculated by the improved density peak clustering (DPC) algorithm. Finally, the superpixels are clustered using fast fuzzy c-means (FCM) based on a color histogram. The experimental results show that not only is the algorithm able to automatically determine the number of colors in the pattern and achieve the accurate color separation of degraded patterns, but it also has lower running time. The color-separation results for 30 degraded patterns show that the segmentation accuracy of the color-separation algorithm proposed in this paper reaches 95.78%.

Adaptive fuzzy weighted C-mean image segmentation algorithm combining a new distance metric and prior entropy

The fuzzy C-mean clustering algorithm (FCM) is effective in image segmentation. However, its sensitivity to initialization settings and the limitation of the Euclidean distance metric in measuring similarity lead to unsatisfactory image segmentation. To overcome these shortcomings, we propose an adaptive fuzzy weighted C-mean image segmentation algorithm that combines a new distance metric and prior entropy. This algorithm significantly contributes in three aspects. First, we employ an improved sparse subspace clustering (SSC) algorithm based on the superpixel image for initialization settings. This step aims to obtain the initial number of clusters and the membership matrix, ensuring the algorithm’s convergence to the optimal solution. Second, to better capture differences between image regions, we introduce a novel distance metric that combines Euclidean and non-Euclidean distances. Simultaneously, the fuzzy weight is introduced to mitigate redundant feature interference, making the clustering result more reasonable. Finally, leveraging prior entropy – a maximum entropy under conditional constraints – we refine the algorithm’s adaptability to unknown features, thereby improving clustering accuracy. Comparative experiments with several state-of-the-art algorithms validate the effectiveness and robustness of our proposed algorithm.

Aquila-particle swarm based cooperative search optimizer with superpixel techniques for epithelial layer segmentation

The segmentation of epithelial layers from oral histopathology images plays a crucial role for early detection of oral cancer disease. As a result, more accurate segmentation of this layer is of the utmost importance for a Computer Aided Diagnosis (CAD) system. Therefore, this paper presents a superpixel image-based clustering technique using an improved Nature-Inspired Optimization Algorithm (NIOA), called the Cooperative Search (CS) algorithm. Here, superpixel image is utilized to construct a faster and noise-resistant oral histopathology image clustering method. The designed CS, on the other hand, is based on two popular NIOAs, the Aquila Optimizer (AO) and the Particle Swarm Optimizer (PSO), has better exploration-exploitation abilities. This CS has been used in the image clustering field to circumvent the problem of local optima trapping. On the other hand, a comparative investigation of three popular superpixel strategies with the proposed CS optimizer has been conducted to determine the optimal superpixel strategy for epithelial layer segmentation from clean as well as noisy oral histopathology images. Finally, the optimal superpixel strategy with CS has been tested with other state-of-the-art image segmentation techniques in the epithelial layer segmentation domain. The results obtained by the proposed segmentation technique are 98.82%, 96.70%, 97.60%, and 95.25%, in terms of average Accuracy, MCC, Dice, and Jaccard, respectively, which all are better than the cutting-edge segmentation methods. The proposed segmentation model is also evaluated over the leaf segmentation model and achieved better results visually and numerically. The optimization efficiency of the CS has also been measured over CEC2019 mathematical benchmark test functions and produced competitive results compared to other tested NIOAs.

Automatic Detection Method for Black Smoke Vehicles Considering Motion Shadows.

Various statistical data indicate that mobile source pollutants have become a significant contributor to atmospheric environmental pollution, with vehicle tailpipe emissions being the primary contributor to these mobile source pollutants. The motion shadow generated by motor vehicles bears a visual resemblance to emitted black smoke, making this study primarily focused on the interference of motion shadows in the detection of black smoke vehicles. Initially, the YOLOv5s model is used to locate moving objects, including motor vehicles, motion shadows, and black smoke emissions. The extracted images of these moving objects are then processed using simple linear iterative clustering to obtain superpixel images of the three categories for model training. Finally, these superpixel images are fed into a lightweight MobileNetv3 network to build a black smoke vehicle detection model for recognition and classification. This study breaks away from the traditional approach of "detection first, then removal" to overcome shadow interference and instead employs a "segmentation-classification" approach, ingeniously addressing the coexistence of motion shadows and black smoke emissions. Experimental results show that the Y-MobileNetv3 model, which takes motion shadows into account, achieves an accuracy rate of 95.17%, a 4.73% improvement compared with the N-MobileNetv3 model (which does not consider motion shadows). Moreover, the average single-image inference time is only 7.3 ms. The superpixel segmentation algorithm effectively clusters similar pixels, facilitating the detection of trace amounts of black smoke emissions from motor vehicles. The Y-MobileNetv3 model not only improves the accuracy of black smoke vehicle recognition but also meets the real-time detection requirements.

Fuzzy C‐means clustering algorithm based on superpixel merging and multi‐feature adaptive fusion measurement

AbstractThe fuzzy C‐means clustering (FCM) algorithm is widely used in greyscale and colour image segmentation, especially in real colour images. However, in the process of interested regions extraction, it performs barely satisfactory due to the use of single distance metric in traditional FCM. In order to address the issue, a fuzzy C‐means clustering algorithm based on superpixel merging and multi‐feature adaptive fusion measurement (FCM‐SM) are proposed. First of all, since superpixel image adapts well to irregular image boundaries, the FCM algorithm is elevated to the superpixel level. Moreover, the superpixel merging module based on colour similarity is introduced to improve the traditional watershed transformation by incorporating more gradient and intensity information. Finally, the multi‐feature adaptive fusion measurement is designed in the clustering process to incorporate multiple sources of spatial information, which comprehensively measures the information from different fields to enhance the segmentation capability further. Experiments performed on Berkeley Segmentation Dataset and Brain Tumor Segmentation Dataset demonstrate that the proposed algorithm provides better segmentation results than benchmarks during the object extraction.

Retracted: Electrochemical Intelligent Recognition of Mineral Materials Based on Superpixel Image Segmentation.

[This retracts the article DOI: 10.1155/2022/6755771.].

Multi-view expressive graph neural networks for 3D CAD model classification

The creation of effective content-based retrieval systems for the 3D models of engineering components created by Mechanical Computer Aided Design (MCAD) systems has been a subject of academic investigation since the 1990 s. Recently some of the most promising results have been reported by researchers using Deep Neural Nets (DNN) to classify industrial parts represented by collections of images generated from different viewpoints. Known as Multi-View Convolutional Neural Network (MVCNN) these systems have extended the architectures developed for 2D images analysis to handle 3D data by using a series of pictures rendered from different viewpoints. In 2016 Monti et al. used Graph Neural Networks (GNN) to classify the superpixel images for the first time and reported results which suggested that the approach could produce good classification accuracy (an observation confirmed by the application of GNNs to 2D image datasets of common objects such as MNIST and Cifar). To investigate the potential for GNNs to reason about spatial adjacency relationships this paper reports, for the first time, the application of GNNs to classifications of 3D MCAD models of mechanical components. Viewing GNN as a convolution operator, parallel GNN can be applied to multiple views of a 3D shape in the same way as CNN is structured in a MVCNN. After outlining the architecture implemented to support MVGNNs the impact of various hyperparameters on the expressive power are explored. When optimised these hyperparameters (located in both intra-layer and inter-layer structures) combined with the multi-view architecture produced a classification accuracy superior to previous methods. The results also suggest that GNNs have the potential to produce fast, accurate classification systems for 3D MCAD data using significantly smaller datasets than those used for transfer learning in MVCNNs.

Automatic segmentation of thyroid with the assistance of the devised boundary improvement based on multicomponent small dataset.

Deep learning has been widely considered in medical image segmentation. However, the difficulty of acquiring medical images and labels can affect the accuracy of the segmentation results for deep learning methods. In this paper, an automatic segmentation method is proposed by devising a multicomponent neighborhood extreme learning machine to improve the boundary attention region of the preliminary segmentation results. The neighborhood features are acquired by training U-Nets with the multicomponent small dataset, which consists of original thyroid ultrasound images, Sobel edge images and superpixel images. Afterward, the neighborhood features are selected by min-redundancy and max-relevance filter in the designed extreme learning machine, and the selected features are used to train the extreme learning machine to obtain supplementary segmentation results. Finally, the accuracy of the segmentation results is improved by adjusting the boundary attention region of the preliminary segmentation results with the supplementary segmentation results. This method combines the advantages of deep learning and traditional machine learning, boosting the accuracy of thyroid segmentation accuracy with a small dataset in a multigroup test.

A survey on the utilization of Superpixel image for clustering based image segmentation.

Superpixel become increasingly popular in image segmentation field as it greatly helps image segmentation techniques to segment the region of interest accurately in noisy environment and also reduces the computation effort to a great extent. However, selection of proper superpixel generation techniques and superpixel image segmentation techniques play a very crucial role in the domain of different kinds of image segmentation. Clustering is a well-accepted image segmentation technique and proved their effective performance over various image segmentation field. Therefore, this study presents an up-to-date survey on the employment of superpixel image in combined with clustering techniques for the various image segmentation. The contribution of the survey has four parts namely (i) overview of superpixel image generation techniques, (ii) clustering techniques especially efficient partitional clustering techniques, their issues and overcoming strategies, (iii) Review of superpixel combined with clustering strategies exist in literature for various image segmentation, (iv) lastly, the comparative study among superpixel combined with partitional clustering techniques has been performed over oral pathology and leaf images to find out the efficacy of the combination of superpixel and partitional clustering approaches. Our evaluations and observation provide in-depth understanding of several superpixel generation strategies and how they apply to the partitional clustering method.

Salient Object Detection based on Background Optimization

Aiming at the problems of low detection accuracy and blurred object edges in current salient object detection based on background algorithms, a new algorithm based on boundary prior to estimate background is proposed. Firstly, the super-pixel image segmentation algorithm (SLIC) is used to segment the image into super-pixels. According to the theory that the foreground and background are relative, the background is estimated by using the four-edges boundary prior of the image; At the same time, an optimal fusion algorithm is proposed to fuse the prior background and foreground with optimal weights to obtain the final saliency map. Quantitative and qualitative comparisons with 10 classic salient object detection algorithms, the results show that the proposed algorithm can not only detect salient objects of different sizes more accurately, but also effectively suppress background redundant information, preserving the object boundary. Therefore, the accuracy of our algorithm is better than 10 classical algorithms. The algorithm in this paper which serve as pre-processing for computer vision has great promise in the field of intelligent control of machine vision.

Superpixel-Based Multiscale CNN Approach Toward Multiclass Object Segmentation From UAV-Captured Aerial Images

Unmanned aerial vehicles (UAVs) are promising remote sensors capable of reforming remote sensing applications. However, for artificial intelligence (AI)-guided tasks, like land cover mapping and ground-object mapping, most deep learning-based architectures fail to extract scale-invariant features, resulting in poor performance accuracy. In this context, the article proposes a superpixel-aided multiscale convolutional neural network (CNN) architecture to avoid misclassification in complex urban aerial images.The proposed framework is a two-tier deep learning-based segmentation architecture. In the first stage, a superpixel-based simple linear iterative cluster (SLIC) algorithm produces superpixel images with crucial contextual information. The second stage comprises a multiscale CNN architecture that uses these information-rich superpixel images to extract scale-invariant features for predicting the object class of each pixel. Two UAV-image-based aerial image datasets: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NITRDrone</i> dataset and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">urban drone dataset</i> (UDD) are considered to perform the experimentation. The proposed model outperforms the considered state-of-the-art methods with an intersection of union (IoU) of 76.39% and 86.85% on UDD and NITRDrone datasets, respectively. Experimentally obtained results prove that the proposed architecture performs superior by achieving better performance accuracy in complex and challenging scenarios.

Automatic segmentation algorithm for high-spatial-resolution remote sensing images based on self-learning super-pixel convolutional network

ABSTRACT Super-pixel algorithms based on convolutional neural networks with fuzzy C-means clustering are widely used for high-spatial-resolution remote sensing images segmentation. However, this model requires the number of clusters to be set manually, resulting in a low automation degree due to the complexity of the iterative clustering process. To address this problem, a segmentation method based on a self-learning super-pixel network (SLSP-Net) and modified automatic fuzzy clustering (MAFC) is proposed. SLSP-Net performs feature extraction, non-iterative clustering, and gradient reconstruction. A lightweight feature embedder is adopted for feature extraction, thus expanding the receiving range and generating multi-scale features. Automatic matching is used for non-iterative clustering, and the overfitting of the network model is overcome by adaptively adjusting the gradient weight parameters, providing a better irregular super-pixel neighborhood structure. An optimized density peak algorithm is adopted for MAFC. Based on the obtained super-pixel image, this maximizes the robust decision-making interval, which enhances the automation of regional clustering. Finally, prior entropy fuzzy C-means clustering is applied to optimize the robust decision-making and obtain the final segmentation result. Experimental results show that the proposed model offers reduced experimental complexity and achieves good performance, realizing not only automatic image segmentation, but also good segmentation results.

Shape modeling with spline partitions

Shape modelling (with methods that output shapes) is a new and important task in Bayesian nonparametrics and bioinformatics. In this work, we focus on Bayesian nonparametric methods for capturing shapes by partitioning a space using curves. In related work, the classical Mondrian process is used to partition spaces recursively with axis-aligned cuts, and is widely lied in multi-dimensional and relational data. The Mondrian process outputs hyper-rectangles. Recently, the random tessellation process was introduced as a generalization of the Mondrian process, partitioning a domain with non-axis aligned cuts in an arbitrary dimensional space, and outputting polytopes. Motivated by these processes, in this work, we propose a novel parallelized Bayesian nonparametric approach to partition a domain with curves, enabling complex data-shapes to be acquired. We apply our method to HIV-1-infected human macrophage image dataset, and also simulated datasets sets to illustrate our approach. We compare to support vector machines, random forests and state-of-the-art computer vision methods such as simple linear iterative clustering super pixel image segmentation. We develop an R package that is available at https://github.com/ShufeiGe/Shape-Modeling-with-Spline-Partitions.

Cross-scale sensing of field-level crop residue cover: Integrating field photos, airborne hyperspectral imaging, and satellite data

Conservation tillage practices can bring benefits to agricultural sustainability. Accurate spatial and temporal resolved information of field-scale crop residue cover, which reflects tillage intensity, is highly valuable for evaluating the outcomes of government conservation programs and voluntary ecosystem service markets, as well as facilitating agroecosystem modeling to quantify cropland biogeochemical processes. Remote sensing has the potential to cost-effectively detect crop residue cover, however, existing regional-scale studies were limited by insufficient ground truth data, scale mismatch between coarse satellite pixels and ground data, and the lack of key spectral data for detecting crop residues. Therefore, this study developed an innovative cross-sensing framework to integrate proximal sensing, airborne hyperspectral imaging, and satellite Earth Observation through deep learning to quantify field-level crop residue cover fractions at the regional scale. Specifically, we have collected intensive ground orthographic photos and conducted airborne hyperspectral surveys at corn and soybean fields of Champaign and nearby counties in Illinois, the heartland of the U.S. Corn Belt. Through semi-automatic labeling aided by ResNet-50 and superpixel image segmentation, we obtained 6719 records of ground residue fractions. With these ground data, we developed the 1-dimensional convolution neural network (CNN) model using airborne hyperspectral reflectance, which has 0.5 m spatial resolution and 3–5 nm spectral resolution from 400 to 2400 nm, to predict residue fractions. By applying the CNN model to airborne pixels, we augmented “ground truth” data of crop residues and further combined them with Harmonized Landsat and Sentinel-2 (HLS) satellite data to quantify regional residue fractions at 30 m resolution. Results show that airborne hyperspectral imagery with CNN can accurately detect residue fractions (R2 = 0.82, relative RMSE = 11.73%) to effectively generate quasi “ground truth” data to support satellite upscaling to all fields. With independent ground data for testing, we found that the ground-airborne-satellite integrative framework achieved better predictions in estimating crop residue cover (R2 = 0.67, relative RMSE = 17.53%) than the conventional ground-satellite upscaling (R2 = 0.22, relative RMSE = 32.09%). We also found that the shortwave infrared wavelengths, particularly 2100–2300 nm, are vital for predicting crop residue cover. Sentinel-2 and Landsat-8 data have a comparable capability to track residue fractions due to similar shortwave infrared wavelengths. This study highlights the high accuracy of hyperspectral imaging to detect agroecosystem tillage management practices and the advantages of cross-scale sensing to cost-effectively integrate multi-source data to quantify field-level agroecosystem variables across scales.