Classical Segmentation Methods Research Articles

Surveying image segmentation approaches in astronomy

Image segmentation plays a critical role in unlocking the mysteries of the universe, providing astronomers with a clearer perspective on celestial objects within complex astronomical images and data cubes. Manual segmentation, while traditional, is not only time-consuming but also susceptible to biases introduced by human intervention. As a result, automated segmentation methods have become essential for achieving robust and consistent results in astronomical studies. This review begins by summarizing traditional and classical segmentation methods widely used in astronomical tasks. Despite the significant improvements these methods have brought to segmentation outcomes, they fail to meet astronomers’ expectations, requiring additional human correction, further intensifying the labor-intensive nature of the segmentation process. The review then focuses on the transformative impact of machine learning, particularly deep learning, on segmentation tasks in astronomy. It introduces state-of-the-art machine learning approaches, highlighting their applications and the remarkable advancements they bring to segmentation accuracy in both astronomical images and data cubes. As the field of machine learning continues to evolve rapidly, it is anticipated that astronomers will increasingly leverage these sophisticated techniques to enhance segmentation tasks in their research projects. In essence, this review serves as a comprehensive guide to the evolution of segmentation methods in astronomy, emphasizing the transition from classical approaches to cutting-edge machine learning methodologies. We encourage astronomers to embrace these advancements, fostering a more streamlined and accurate segmentation process that aligns with the ever-expanding frontiers of astronomical exploration.

RANSAC-Based Planar Point Cloud Segmentation Enhanced by Normal Vector and Maximum Principal Curvature Clustering

Abstract. Planar feature segmentation is an essential task for 3D point cloud processing, finding many applications in various fields such as robotics and computer vision. The Random Sample Consensus (RANSAC) is one of the most common algorithms for the segmentation, but its performance, as given by the original form, is usually limited due to the use of a single threshold and interruption of similar planar features presented close to each other. To address these issues, we present a novel point cloud processing workflow which aims at developing an initial segmentation stage before the basic RANSAC is performed. Initially, normal vectors and maximum principal curvatures for each point of a given point cloud are analyzed and integrated. Subsequently, a subset of normal vectors and curvature is utilized to cluster planes with similar geometry based on the region growing algorithm, serving as a coarse but fast segmentation process. The segmentation is therefore refined with the RANSAC algorithm which can be now performed with higher accuracy and speed due to the reduced interference. After the RANSAC process is applied, resultant planar point clouds are built from the sparse ones via a point aggregation process based on geometric constraints. Four datasets (three real and one simulated) were used to verify the method. Compared to the classic segmentation method, our method achieves higher accuracy, with an RMSE from fitting equal to 0.0521 m, along with a higher recall of 93.31% and a higher F1-score of 95.38%.

An Approach for Plant Leaf Image Segmentation Based on YOLOV8 and the Improved DEEPLABV3.

Accurate plant leaf image segmentation provides an effective basis for automatic leaf area estimation, species identification, and plant disease and pest monitoring. In this paper, based on our previous publicly available leaf dataset, an approach that fuses YOLOv8 and improved DeepLabv3+ is proposed for precise image segmentation of individual leaves. First, the leaf object detection algorithm-based YOLOv8 was introduced to reduce the interference of backgrounds on the second stage leaf segmentation task. Then, an improved DeepLabv3+ leaf segmentation method was proposed to more efficiently capture bar leaves and slender petioles. Densely connected atrous spatial pyramid pooling (DenseASPP) was used to replace the ASPP module, and the strip pooling (SP) strategy was simultaneously inserted, which enabled the backbone network to effectively capture long distance dependencies. The experimental results show that our proposed method, which combines YOLOv8 and the improved DeepLabv3+, achieves a 90.8% mean intersection over the union (mIoU) value for leaf segmentation on our public leaf dataset. When compared with the fully convolutional neural network (FCN), lite-reduced atrous spatial pyramid pooling (LR-ASPP), pyramid scene parsing network (PSPnet), U-Net, DeepLabv3, and DeepLabv3+, the proposed method improves the mIoU of leaves by 8.2, 8.4, 3.7, 4.6, 4.4, and 2.5 percentage points, respectively. Experimental results show that the performance of our method is significantly improved compared with the classical segmentation methods. The proposed method can thus effectively support the development of smart agroforestry.

Synthesis of large scale 3D microscopic images of 3D cell cultures for training and benchmarking.

The analysis of 3D microscopic cell culture images plays a vital role in the development of new therapeutics. While 3D cell cultures offer a greater similarity to the human organism than adherent cell cultures, they introduce new challenges for automatic evaluation, like increased heterogeneity. Deep learning algorithms are able to outperform conventional analysis methods in such conditions but require a large amount of training data. Due to data size and complexity, the manual annotation of 3D images to generate large datasets is a nearly impossible task. We therefore propose a pipeline that combines conventional simulation methods with deep-learning-based optimization to generate large 3D synthetic images of 3D cell cultures where the labels are known by design. The hybrid procedure helps to keep the generated image structures consistent with the underlying labels. A new approach and an additional measure are introduced to model and evaluate the reduced brightness and quality in deeper image regions. Our analyses show that the deep learning optimization step consistently improves the quality of the generated images. We could also demonstrate that a deep learning segmentation model trained with our synthetic data outperforms a classical segmentation method on real image data. The presented synthesis method allows selecting a segmentation model most suitable for the user's data, providing an ideal basis for further data analysis.

Positive–negative equal contrastive loss for semantic segmentation

The contextual information is critical for various computer vision tasks, previous works commonly design plug-and-play modules and structural losses to effectively extract and aggregate the global context. These methods utilize fine-label to optimize the model but ignore that fine-trained features are also precious training resources, which can introduce preferable distribution to hard pixels (i.e., misclassified pixels). Inspired by contrastive learning in unsupervised paradigm, we apply the contrastive loss in a supervised manner and re-design the loss function to cast off the stereotype of unsupervised learning (e.g., imbalance of positives and negatives, confusion of anchors computing). To this end, we propose Positive-Negative Equal contrastive loss (PNE loss), which increases the latent impact of positive embedding on the anchor and treats the positive as well as negative sample pairs equally. The PNE loss can be directly plugged right into existing semantic segmentation frameworks and leads to excellent performance with neglectable extra computational costs. We utilize a number of classic segmentation methods (e.g., DeepLabV3, HRNetV2, OCRNet, UperNet) and backbone (e.g., ResNet, HRNet, Swin Transformer) to conduct comprehensive experiments and achieve state-of-the-art performance on three benchmark datasets (e.g., Cityscapes, COCO-Stuff and ADE20K). Our code will be publicly available at https://github.com/jingw193/PNE_Loss.

FEC: Fast Euclidean Clustering for Point Cloud Segmentation

Segmentation from point cloud data is essential in many applications, such as remote sensing, mobile robots, or autonomous cars. However, the point clouds captured by the 3D range sensor are commonly sparse and unstructured, challenging efficient segmentation. A fast solution for point cloud instance segmentation with small computational demands is lacking. To this end, we propose a novel fast Euclidean clustering (FEC) algorithm which applies a point-wise scheme over the cluster-wise scheme used in existing works. The proposed method avoids traversing every point constantly in each nested loop, which is time and memory-consuming. Our approach is conceptually simple, easy to implement (40 lines in C++), and achieves two orders of magnitudes faster against the classical segmentation methods while producing high-quality results.

Deep learning image segmentation reveals patterns of UV reflectance evolution in passerine birds

Ultraviolet colouration is thought to be an important form of signalling in many bird species, yet broad insights regarding the prevalence of ultraviolet plumage colouration and the factors promoting its evolution are currently lacking. In this paper, we develop a image segmentation pipeline based on deep learning that considerably outperforms classical (i.e. non deep learning) segmentation methods, and use this to extract accurate information on whole-body plumage colouration from photographs of >24,000 museum specimens covering >4500 species of passerine birds. Our results demonstrate that ultraviolet reflectance, particularly as a component of other colours, is widespread across the passerine radiation but is strongly phylogenetically conserved. We also find clear evidence in support of the role of light environment in promoting the evolution of ultraviolet plumage colouration, and a weak trend towards higher ultraviolet plumage reflectance among bird species with ultraviolet rather than violet-sensitive visual systems. Overall, our study provides important broad-scale insight into an enigmatic component of avian colouration, as well as demonstrating that deep learning has considerable promise for allowing new data to be brought to bear on long-standing questions in ecology and evolution.

A Discriminative Level Set Method with Deep Supervision for Breast Tumor Segmentation

Background:Breast tumor segmentation in B-mode ultrasound imaging is important for analyzing, identifying, and diagnosing tumors. The level set is an approach most widely used in breast segmentation, and the refinement is still in progress. However, its effectiveness is harmed by a dearth of semantic information. On the other hand, deep networks contain rich semantic information but loss much influential low-level details. Method:This paper proposes a novel deep-feature embedded level set group to exploit semantically enriched features for breast tumor segmentation. First, a UNet-based network is trained to extract different features at different stages. Each stage has unique features depiction. Then, a novel level-set method is integrated at the end of each stage to approach more accurate and precise features maps. A new feature-discriminator is devised in the energy function of the level set method to refine the low confidence pixels at the boundaries. Lastly, the outputs of the level set method at different stages are incorporated into final feature maps to further empower the segmentation process. Two datasets comprising 349 breast ultrasound images from various hospitals have been utilized to assess the proposed approach’s performance. The model’s effectiveness is estimated on different metrics, including Accuracy, Sensitivity or True Positive rate, Specificity or True Negative rate, False Positive rate Dice, and IoU values for both datasets. Furthermore, the efficiency of the model is investigated by performing a comparison with several state-of-the-art classic segmentation methods and deep learning methods. Result:The proposed method outperformed segmenting breast ultrasound tumors in terms of Dice and IoU for datasets A and B (with p-value < 0.005 against compared methods). Additionally, the performance of the proposed approach is evaluated using the Area Under Receiver Operating Characteristics curve (AUC) and Mean Absolute Error (MAE). Our findings indicate that the proposed method seems to gain superiority over other methods by obtaining a lower MAE rate with the highest value of the AUC. Conclusion:Experiments determine that our method has obtained the best cut-off to deal with the noticeable glitches present in other approaches and generates more accurate segmentation results for tumors in complex images. Hence, the results confirm the proposed method’s effectiveness compared to classic segmentation methods over ultrasound images with blurry boundaries, noise, and intensity inhomogeneity. Moreover, our approach gives unprecedented prediction accuracy and similarity for malignant tumors.

Super-Resolved Segmentation of X-ray Images of Carbonate Rocks Using Deep Learning

Reliable quantitative analysis of digital rock images requires precise segmentation and identification of the macroporosity, sub-resolution porosity, and solid\\mineral phases. This is highly emphasized in heterogeneous rocks with complex pore size distributions such as carbonates. Multi-label segmentation of carbonates using classic segmentation methods such as multi-thresholding is highly sensitive to user bias and often fails in identifying low-contrast sub-resolution porosity. In recent years, deep learning has introduced efficient and automated algorithms that are capable of handling hard tasks with precision comparable to human performance, with application to digital rocks super-resolution and segmentation emerging. Here, we present a framework for using convolutional neural networks (CNNs) to produce super-resolved segmentations of carbonates rock images for the objective of identifying sub-resolution porosity. The volumes used for training and testing are based on two different carbonates rocks imaged in-house at low and high resolutions. We experiment with various implementations of CNNs architectures where super-resolved segmentation is obtained in an end-to-end scheme and using two networks (super-resolution and segmentation) separately. We show the capability of the trained model of producing accurate segmentation by comparing multiple voxel-wise segmentation accuracy metrics, topological features, and measuring effective properties. The results underline the value of integrating deep learning frameworks in digital rock analysis.

Segmentation and Quantitative Analysis of Photoacoustic Imaging: A Review

Photoacoustic imaging is an emerging biomedical imaging technique that combines optical contrast and ultrasound resolution to create unprecedented light absorption contrast in deep tissue. Thanks to its fusional imaging advantages, photoacoustic imaging can provide multiple structural and functional insights into biological tissues such as blood vasculatures and tumors and monitor the kinetic movements of hemoglobin and lipids. To better visualize and analyze the regions of interest, segmentation and quantitative analyses were used to extract several biological factors, such as the intensity level changes, diameter, and tortuosity of the tissues. Over the past 10 years, classical segmentation methods and advances in deep learning approaches have been utilized in research investigations. In this review, we provide a comprehensive review of segmentation and quantitative methods that have been developed to process photoacoustic imaging in preclinical and clinical experiments. We focus on the parametric reliability of quantitative analysis for semantic and instance-level segmentation. We also introduce the similarities and alternatives of deep learning models in qualitative measurements using classical segmentation methods for photoacoustic imaging.

A Modular U-Net for Automated Segmentation of X-Ray Tomography Images in Composite Materials

X-Ray Computed Tomography (XCT) techniques have evolved to a point that high-resolution data can be acquired so fast that classic segmentation methods are prohibitively cumbersome, demanding automated data pipelines capable of dealing with non-trivial 3D images. Meanwhile, deep learning has demonstrated success in many image processing tasks, including materials science applications, showing a promising alternative for a human-free segmentation pipeline. However, the rapidly increasing number of available architectures can be a serious drag to the wide adoption of this type of models by the end user. In this paper a modular interpretation of U-Net (Modular U-Net) is proposed with a parametrized architecture that can be easily tuned to optimize it. As an example, the model is trained to segment 3D tomography images of a three-phased glass fiber-reinforced Polyamide 66. We compare 2D and 3D versions of our model, finding that the former is slightly better than the latter. We observe that human-comparable results can be achievied even with only 13 annotated slices and using a shallow U-Net yields better results than a deeper one. As a consequence, neural networks show indeed a promising venue to automate XCT data processing pipelines needing no human, adhoc intervention.

Breast Cancer Segmentation Methods: Current Status and Future Potentials.

Early breast cancer detection is one of the most important issues that need to be addressed worldwide as it can help increase the survival rate of patients. Mammograms have been used to detect breast cancer in the early stages; if detected in the early stages, it can drastically reduce treatment costs. The detection of tumours in the breast depends on segmentation techniques. Segmentation plays a significant role in image analysis and includes detection, feature extraction, classification, and treatment. Segmentation helps physicians quantify the volume of tissue in the breast for treatment planning. In this work, we have grouped segmentation methods into three groups: classical segmentation that includes region-, threshold-, and edge-based segmentation; machine learning segmentation; and supervised and unsupervised and deep learning segmentation. The findings of our study revealed that region-based segmentation is frequently used for classical methods, and the most frequently used techniques are region growing. Further, a median filter is a robust tool for removing noise. Moreover, the MIAS database is frequently used in classical segmentation methods. Meanwhile, in machine learning segmentation, unsupervised machine learning methods are more frequently used, and U-Net is frequently used for mammogram image segmentation because it does not require many annotated images compared with other deep learning models. Furthermore, reviewed papers revealed that it is possible to train a deep learning model without performing any preprocessing or postprocessing and also showed that the U-Net model is frequently used for mammogram segmentation. The U-Net model is frequently used because it does not require many annotated images and also because of the presence of high-performance GPU computing, which makes it easy to train networks with more layers. Additionally, we identified mammograms and utilised widely used databases, wherein 3 and 28 are public and private databases, respectively.

DEEP LEARNING TECHNOLOGYFOR VIDEOFRAME PROCESSING IN FACE SEGMENTATION ON MOBILE DEVICES

The aim of the research is to reduce the frame processing time for face segmentation on videos on mobile devices using deep learning technologies. The paper analyzes the advantages and disadvantages of existing segmentation methods, as well as their applicability to various tasks. The existing real-time realizations of face segmentation in the most popular mobile applications, which provide the functionality for adding visual effects to videos, were compared. As a result, it was determined that the classical segmentation methods do not have a suitable combination of accuracy and speed, and require manual tuning for a particular task, while the neural network-based segmentation methods determine the deepfeatures automatically and have high accuracy with an acceptablespeed. The method based on convolutional neural networks is chosen for use because, in addition to the advantages of other methods based on neural networks, it does not require such a significant amount of computing resources during its execution. A review of existing convolutional neural networks for segmentation was held, based on which the DeepLabV3+ network was chosen as having sufficiently high accuracy and being optimized for work on mobile devices. Modifications were made to the structure of the selected network to match the task of two classes segmentation and to speed up the work on devices with low performance. 8-bit quantization was appliedto the values processed by the network for further acceleration. The network was adapted to the task of face segmentation by transfer learningperformed on a setof face images from the COCO dataset. Based on the modified and additionally trained segmentation model, a mobile app was created to record video with real-time visual effects, which applies segmentation to separately addeffects on two zones -the face (color filters, brightness adjustment, animated effects) and the background (blurring, hiding, replacement with another image). The time of frames processing in the application was tested on mobile devices with different technical characteristics. We analyzed the differences in testing results for segmentation using the obtained modeland segmentation using the normalized cuts method. The comparison reveals a decrease of frame processing time on the majority of devices with a slight decrease of segmentation accuracy.

Lesion segmentation in breast ultrasound images using the optimized marked watershed method

BackgroundBreast cancer is one of the most serious diseases threatening women’s health. Early screening based on ultrasound can help to detect and treat tumours in the early stage. However, due to the lack of radiologists with professional skills, ultrasound-based breast cancer screening has not been widely used in rural areas. Computer-aided diagnosis (CAD) technology can effectively alleviate this problem. Since breast ultrasound (BUS) images have low resolution and speckle noise, lesion segmentation, which is an important step in CAD systems, is challenging.ResultsTwo datasets were used for evaluation. Dataset A comprises 500 BUS images from local hospitals, while dataset B comprises 205 open-source BUS images. The experimental results show that the proposed method outperformed its related classic segmentation methods and the state-of-the-art deep learning model RDAU-NET. Its accuracy (Acc), Dice similarity coefficient (DSC) and Jaccard index (JI) reached 96.25%, 78.4% and 65.34% on dataset A, and its Acc, DSC and sensitivity reached 97.96%, 86.25% and 88.79% on dataset B, respectively.ConclusionsWe proposed an adaptive morphological snake based on marked watershed (AMSMW) algorithm for BUS image segmentation. It was proven to be robust, efficient and effective. In addition, it was found to be more sensitive to malignant lesions than benign lesions.MethodsThe proposed method consists of two steps. In the first step, contrast limited adaptive histogram equalization (CLAHE) and a side window filter (SWF) are used to preprocess BUS images. Lesion contours can be effectively highlighted, and the influence of noise can be eliminated to a great extent. In the second step, we propose adaptive morphological snake (AMS). It can adjust the working parameters adaptively according to the size of the lesion. Its segmentation results are combined with those of the morphological method. Then, we determine the marked area and obtain candidate contours with a marked watershed (MW). Finally, the best lesion contour is chosen by the maximum average radial derivative (ARD).

A Method of Segmenting Apples Based on Gray-Centered RGB Color Space

In recent years, many agriculture-related problems have been evaluated with the integration of artificial intelligence techniques and remote sensing systems. The rapid and accurate identification of apple targets in an illuminated and unstructured natural orchard is still a key challenge for the picking robot’s vision system. In this paper, by combining local image features and color information, we propose a pixel patch segmentation method based on gray-centered red–green–blue (RGB) color space to address this issue. Different from the existing methods, this method presents a novel color feature selection method that accounts for the influence of illumination and shadow in apple images. By exploring both color features and local variation in apple images, the proposed method could effectively distinguish the apple fruit pixels from other pixels. Compared with the classical segmentation methods and conventional clustering algorithms as well as the popular deep-learning segmentation algorithms, the proposed method can segment apple images more accurately and effectively. The proposed method was tested on 180 apple images. It offered an average accuracy rate of 99.26%, recall rate of 98.69%, false positive rate of 0.06%, and false negative rate of 1.44%. Experimental results demonstrate the outstanding performance of the proposed method.

Multi-Feature Patch-Based Segmentation Technique in the Gray-Centered RGB Color Space for Improved Apple Target Recognition

In the vision system of apple-picking robots, the main challenge is to rapidly and accurately identify the apple targets with varying halation and shadows on their surfaces. To solve this problem, this study proposes a novel, multi-feature, patch-based apple image segmentation technique using the gray-centered red-green-blue (RGB) color space. The developed method presents a multi-feature selection process, which eliminates the effect of halation and shadows in apple images. By exploring all the features of the image, including halation and shadows, in the gray-centered RGB color space, the proposed algorithm, which is a generalization of K-means clustering algorithm, provides an efficient target segmentation result. The proposed method is tested on 240 apple images. It offered an average accuracy rate of 98.79%, a recall rate of 99.91%, an F1 measure of 99.35%, a false positive rate of 0.04%, and a false negative rate of 1.18%. Compared with the classical segmentation methods and conventional clustering algorithms, as well as the popular deep-learning segmentation algorithms, the proposed method can perform with high efficiency and accuracy to guide robotic harvesting.

A New Classification Approach with Deep Mask R-CNN for Synthetic Aperture Radar Image Segmentation

In this paper, a hybrid classification approach which is combined with a more deep mask region-convolutional neural network and sparsity driven despeckling algorithm is proposed for synthetic aperture radar (SAR) image segmentation instead of the classical segmentation methods. In satellite technology, synthetic aperture radar images are strongly used for a lot of areas, such as evaluating air conditions, determining agricultural fields, climatic changes, and as a target in the military. Synthetic aperture radar images must be segmented to each meaningful point in the image for a quality segmentation process. In contrast, synthetic aperture radar images have a lot of noisy speckles and these speckles should be also reduced for a quality segmentation. Current studies show that deep learning techniques are widely used for segmentation methods. High accuracy and fast results can be obtained with deep learning techniques for image segmentation. Mask region-convolutional neural network can not only separate each meaningful field in the image, but it can also generate a high accuracy prediction for each meaningful field of synthetic aperture radar images. The study shows that smoothed SAR images can be classified as multiple regions with deep neural networks.

Brain tumor segmentation in MR images using a sparse constrained level set algorithm

Brain tumor segmentation using Magnetic Resonance (MR) Imaging technology plays a significant role in computer-aided brain tumor diagnosis. However, when applying classic segmentation methods, limitations such as inhomogeneous intensity, complex physiological structure and blurred tissues boundaries in brain MR images usually lead to unsatisfactory results. To address these issues, this paper proposes an automatic sparse constrained level set method to realize the brain tumor segmentation in MR images. By studying brain tumor images, this method finds out common characteristics of brain tumors’ shape and constructs a sparse representation model. By considering this model as a prior constraint, an energy function based on level set method is constructed. In experiments, the proposed method can achieve an average accuracy of 96.20% for the MR images from the dataset Brats2017 and performs better than the others. With lower false positive rate and stronger robustness, the experimental results show that the proposed method can segment brain tumor from MR image accurately and stably.

Nuclear Magnetic Image Segmentation of Rectal Cancer Based on Improved Genetic Neural Network

Image segmentation is the basis of image analysis and understanding, and has an unshakable position in the field of computer vision. In order to improve the accuracy of nuclear magnetic image segmentation of rectal cancer, this paper proposes an improved genetic neural network algorithm for the problems of traditional BP neural network algorithm. In order to enhance the network performance, this paper improves the genetic neural network from the two aspects of fitness function and genetic operator, which makes the training speed and convergence precision greatly improved. Target samples were analyzed by image histogram analysis, and the improved genetic neural network was used to learn the samples to obtain the training network. Taking the pixel matrix of the image as the input vector, it is put into the trained network for classification, and finally the segmentation is realized. The simulation experiment proves that compared with the classical image segmentation method, the improved genetic neural network image segmentation method has a good segmentation effect and is a feasible image segmentation method.

Cardiac MR segmentation based on sequence propagation by deep learning.

Accurate segmentation of myocardial in cardiac MRI (magnetic resonance image) is key to effective rapid diagnosis and quantitative pathology analysis. However, a low-quality CMR (cardiac magnetic resonance) image with a large amount of noise makes it extremely difficult to accurately and quickly manually segment the myocardial. In this paper, we propose a method for CMR segmentation based on U-Net and combined with image sequence information. The method can effectively segment from the top slice to the bottom slice of the CMR. During training, each input slice depends on the slice below it. In other words, the predicted segmentation result depends on the existing segmentation label of the previous slice. 3D sequence information is fully utilized. Our method was validated on the ACDC dataset, which included CMR images of 100 patients (1700 2D MRI). Experimental results show that our method can segment the myocardial quickly and efficiently and is better than the current state-of-the-art methods. When evaluating 340 CMR image, our model yielded an average dice score of 85.02 ± 0.15, which is much higher than the existing classical segmentation method(Unet, Dice score = 0.78 ± 0.3).