Image Segmentation Problem Research Articles

Aerial Images Segmentation with Graph Neural Network

Abstract. The development of remote sensing platforms and sensors, as well as the improvement of remote data processing tools and methods, create new opportunities for automatic updating of maps. Currently, aerial photographs serve as the main source for automatic map updates due to their accessibility and significant informational value. One of the core elements for image to maps transition is accurate image segmentation. Nowadays, machine learning methods demonstrate the best results in task of image segmentation. At its core, maps represent information about a certain area in a vector form, that not only contains visual information about area, but also reflects some relations between objects in the map. This quality makes a map more convenient for human perception than an aerial photograph (raster image). This study addresses the problem of accurate aerial image segmentation with taking the advantages of using graph neural network as the more adequate model of map structure. We use graph neural network for retrieving semantic and vector information about a captured area from its aerial image. The developed framework at first phase utilizes visual transformer for retrieving deep features from the input aerial image. The graph neural network then performs clustering of the extracted deep features to obtain semantic segmentation of the image. To train and evaluate the developed framework, a special dataset is collected and annotated. It contains more than 10k aerial photographs representing various types of objects taken in different years and seasons. The evaluation results on the created dataset proved the state-of-the-art performance of the developed framework.

CIS-UNet: Multi-class segmentation of the aorta in computed tomography angiography via context-aware shifted window self-attention

Advancements in medical imaging and endovascular grafting have facilitated minimally invasive treatments for aortic diseases. Accurate 3D segmentation of the aorta and its branches is crucial for interventions, as inaccurate segmentation can lead to erroneous surgical planning and endograft construction. Previous methods simplified aortic segmentation as a binary image segmentation problem, overlooking the necessity of distinguishing between individual aortic branches. In this paper, we introduce Context-Infused Swin-UNet (CIS-UNet), a deep learning model designed for multi-class segmentation of the aorta and thirteen aortic branches. Combining the strengths of Convolutional Neural Networks (CNNs) and Swin transformers, CIS-UNet adopts a hierarchical encoder–decoder structure comprising a CNN encoder, a symmetric decoder, skip connections, and a novel Context-aware Shifted Window Self-Attention (CSW-SA) module as the bottleneck block. Notably, CSW-SA introduces a unique adaptation of the patch merging layer, distinct from its traditional use in the Swin transformers. CSW-SA efficiently condenses the feature map, providing a global spatial context, and enhances performance when applied at the bottleneck layer, offering superior computational efficiency and segmentation accuracy compared to the Swin transformers. We evaluated our model on computed tomography (CT) scans from 59 patients through a 4-fold cross-validation. CIS-UNet outperformed the state-of-the-art Swin UNetR segmentation model by achieving a superior mean Dice coefficient of 0.732 compared to 0.717 and a mean surface distance of 2.40 mm compared to 2.75 mm. CIS-UNet’s superior 3D aortic segmentation offers improved accuracy and optimization for planning endovascular treatments. Our dataset and code will be made publicly available at https://github.com/mirthAI/CIS-UNet.

An enhanced seagull algorithm for multi-threshold image segmentation based on Kapur entropy

Abstract To address the shortcomings of the Seagull Optimization Algorithm (SOA), such as poor distribution of individuals’ positions and low population diversity leading to susceptibility to local optima after the collision avoidance phase, an Enhanced Seagull Algorithm (ESOA) is proposed. Firstly, a chaos-guided mechanism is utilized to replace the original individual positions after collision avoidance with the mean of individual optima and current positions, effectively improving the distribution of the population. Secondly, a circular-topology neighborhood structure with random weights is introduced, allowing interaction between the current individual’s position and the information from its two adjacent optimal positions, thereby improving population clustering and enhancing diversity. Thirdly, a dynamic lens mapping strategy is employed to direct the optimal seagull individuals toward their newly generated mapped points, enhancing their ability to avoid local optima. ESOA is applied to the problem of multi-threshold image segmentation based on Kapur entropy, and experimental results demonstrate its effectiveness in improving segmentation performance.

Edge morphology attention mechanism and optimal geometric matching connection model for vascular segmentation

Over the past decades, medical image segmentation methods have been intensively developed, but there are still a number of unsolved challenging problems in vascular image segmentation, including broken vessels, insufficient vessel branches, and missing small vessels. In order to optimize the topology and accuracy of segmented vessels, we propose a novel Edge Morphology Attention Network (EMA-Net) for the segmentation of vessel-like structures, and an Optimal Geometric Matching Connection (OGMC) model to connect the broken vessel fragments. The EMA-Net has an edge attention module that is able to improve segmented performances of edges and small branches by morphology operation extracting boundary voxels on multi-scales. The OGMC model uses the geometry concept of curve touching to filter out fragmented vessel endpoints, and then employs minimal surfaces to determine the optimal connection order between blood vessels. Finally, we adopt geodesics to repair missing vessels under a Riemannian metric. Our method achieves superior or competitive results compared to state-of-the-art methods on four datasets of 3D vascular segmentation tasks, both effectively reducing vessel broken and increasing vessel branch richness, yielding blood vessels with more precise topological structures.

An improved nutcracker optimization algorithm for discrete and continuous optimization problems: Design, comprehensive analysis, and engineering applications

This study is presented to examine the performance of a newly proposed metaheuristic algorithm within discrete and continuous search spaces. Therefore, the multithresholding image segmentation problem and parameter estimation problem of both the proton exchange membrane fuel cell (PEMFC) and photovoltaic (PV) models, which have different search spaces, are used to test and verify this algorithm. The traditional techniques could not find approximate solutions for those problems in a reasonable amount of time, so researchers have used metaheuristic algorithms to overcome those shortcomings. However, the majority of metaheuristic algorithms still suffer from slow convergence speed and stagnation into local minima problems, which makes them unsuitable for tackling these optimization problems. Therefore, this study proposes an improved nutcracker optimization algorithm (INOA) for better solving those problems in an acceptable amount of time. INOA is based on improving the performance of the standard algorithm using a newly proposed convergence improvement strategy that aims to improve the convergence speed and prevent stagnation in local minima. This algorithm is first applied to estimating the unknown parameters of the single-diode, double-diode, and triple-diode models for a PV module and a solar cell. Second, four PEMFC modules are used to further observe INOA's performance for the continuous optimization challenge. Finally, the performance of INOA is investigated for solving the multi-thresholding image segmentation problem to test its effectiveness in a discrete search space. Several test images with different threshold levels were used to validate its effectiveness, stability, and scalability. Comparison to several rival optimizers using various performance indicators, such as convergence curve, standard deviation, average fitness value, and Wilcoxon rank-sum test, demonstrates that INOA is an effective alternative for solving both discrete and continuous optimization problems. Quantitively, INOA could solve those problems better than the other rival optimizers, with improvement rates for final results ranging between 0.8355 % and 3.34 % for discrete problems and 4.97 % and 99.9 % for continuous problems.

Medical image segmentation approach based on hybrid adaptive differential evolution and crayfish optimizer

Image segmentation plays a pivotal role in medical image analysis, particularly for accurately isolating tumors and lesions. Effective segmentation improves diagnostic precision and facilitates quantitative analysis, which is vital for medical professionals. However, traditional segmentation methods often struggle with multilevel thresholding due to the associated computational complexity. Therefore, determining the optimal threshold set is an NP-hard problem, highlighting the pressing need for efficient optimization strategies to overcome these challenges. This paper introduces a multi-threshold image segmentation (MTIS) method that integrates a hybrid approach combining Differential Evolution (DE) and the Crayfish Optimization Algorithm (COA), known as HADECO. Utilizing two-dimensional (2D) Kapur’s entropy and a 2D histogram, this method aims to enhance the efficiency and accuracy of subsequent image analysis and diagnosis. HADECO is a hybrid algorithm that combines DE and COA by exchanging information based on predefined rules, leveraging the strengths of both for superior optimization results. It employs Latin Hypercube Sampling (LHS) to generate a high-quality initial population. HADECO introduces an improved DE algorithm (IDE) with adaptive and dynamic adjustments to key DE parameters and new mutation strategies to enhance its search capability. In addition, it incorporates an adaptive COA (ACOA) with dynamic adjustments to the switching probability parameter, effectively balancing exploration and exploitation. To evaluate the effectiveness of HADECO, its performance is initially assessed using CEC’22 benchmark functions. HADECO is evaluated against several contemporary algorithms using the Wilcoxon signed rank test (WSRT) and the Friedman test (FT) to integrate the results. The findings highlight HADECO’s superior optimization abilities, demonstrated by its lowest average Friedman ranking of 1.08. Furthermore, the HADECO-based MTIS method is evaluated using MRI images for knee and CT scans for brain intracranial hemorrhage (ICH). Quantitative results in brain hemorrhage image segmentation show that the proposed method achieves a superior average peak signal-to-noise ratio (PSNR) and feature similarity index (FSIM) of 1.5 and 1.7 at the 6-level threshold. In knee image segmentation, it attains an average PSNR and FSIM of 1.3 and 1.2 at the 5-level threshold, demonstrating the method’s effectiveness in solving image segmentation problems.

A Graph Multi-separator Problem for Image Segmentation

We propose a novel abstraction of the image segmentation task in the form of a combinatorial optimization problem that we call the multi-separator problem. Feasible solutions indicate for every pixel whether it belongs to a segment or a segment separator, and indicate for pairs of pixels whether or not the pixels belong to the same segment. This is in contrast to the closely related lifted multicut problem, where every pixel is associated with a segment and no pixel explicitly represents a separating structure. While the multi-separator problem is np-hard, we identify two special cases for which it can be solved efficiently. Moreover, we define two local search algorithms for the general case and demonstrate their effectiveness in segmenting simulated volume images of foam cells and filaments.

Heterogeneous pbest-guided comprehensive learning particle swarm optimization

The balance between exploration and exploitation in the particle swarm optimization (PSO) algorithm is often discussed but has not been well solved. To further improve the capabilities of exploration and exploitation in the search space, the aim of this study is to focus on a novel comprehensive learning strategy for the PSO algorithm, which is named the heterogeneous pbest-guided comprehensive learning particle swarm optimizer (HPBPSO). In this algorithm, the population is divided into two subpopulations. In the exploration subpopulation, a pbest-guided mechanism is designed for obtaining better particles’ personal best information. Meanwhile, the non-elite personal best particles learn from elite personal best particles by using dynamic crossover strategy. In the exploitation subpopulation, in order to fully leverage the benefits of learning from high-quality individuals, the learning process is divided into two distinct categories: elite individual learning and intragroup learning. Furthermore, intergroup perturbation is designed to accelerate convergence performance. The experiments on classical functions are firstly used to verify the effectiveness and mutuality of the proposed strategies. Moreover, the performance of the proposed algorithm is evaluated on two well-known benchmarks and medical image segmentation problem. According to the statistical results, the HPBPSO algorithm is not only better than the other existing state-of-the-art PSO variants and metaheuristic evolutionary algorithms, but also applicable to real-world optimization problems.

Few-Shot Learning for Medical Image Segmentation Using 3D U-Net and Model-Agnostic Meta-Learning (MAML).

Deep learning has attained state-of-the-art results in general image segmentation problems; however, it requires a substantial number of annotated images to achieve the desired outcomes. In the medical field, the availability of annotated images is often limited. To address this challenge, few-shot learning techniques have been successfully adapted to rapidly generalize to new tasks with only a few samples, leveraging prior knowledge. In this paper, we employ a gradient-based method known as Model-Agnostic Meta-Learning (MAML) for medical image segmentation. MAML is a meta-learning algorithm that quickly adapts to new tasks by updating a model's parameters based on a limited set of training samples. Additionally, we use an enhanced 3D U-Net as the foundational network for our models. The enhanced 3D U-Net is a convolutional neural network specifically designed for medical image segmentation. We evaluate our approach on the TotalSegmentator dataset, considering a few annotated images for four tasks: liver, spleen, right kidney, and left kidney. The results demonstrate that our approach facilitates rapid adaptation to new tasks using only a few annotated images. In 10-shot settings, our approach achieved mean dice coefficients of 93.70%, 85.98%, 81.20%, and 89.58% for liver, spleen, right kidney, and left kidney segmentation, respectively. In five-shot sittings, the approach attained mean Dice coefficients of 90.27%, 83.89%, 77.53%, and 87.01% for liver, spleen, right kidney, and left kidney segmentation, respectively. Finally, we assess the effectiveness of our proposed approach on a dataset collected from a local hospital. Employing five-shot sittings, we achieve mean Dice coefficients of 90.62%, 79.86%, 79.87%, and 78.21% for liver, spleen, right kidney, and left kidney segmentation, respectively.

Study of the effectiveness of the search algorithm in image segmentation problems

Study of the effectiveness of the search algorithm in image segmentation problems

WSM-MIL: a weakly supervised segmentation method with multiple instance learning for C elegans image

Recently, image analysis techniques have been introduced to automate nematode information assessment. In image analysis-based nematode information assessment, the initial step involves detecting and segmenting C. elegans from microscopic images and network-based methods have been investigated. However, training a network for C. elegans image segmentation is typically associated with the labor-intensive process of pixel-level mask labeling. To address this challenge, we introduced a weakly supervised segmentation method using multiple instance learning (WSM-MIL). The proposed multi-instance weakly supervised segmentation method comprises three key components: a backbone network, a detection branch, and a segmentation branch. In contrast to fully supervised pixel-level annotation, we opted for weakly supervised bounding box-level annotation. This approach reduces the labour cost of annotation to some extent. The approach offers several advantages, such as simplicity, an end-to-end architecture, and good scalability. We conducted experiments comparing the proposed network with benchmark methods, and the results showed that the network exhibits competitive performance in the image segmentation task of C. elegans. The results of this study provide an effective method in the field of biological image analysis, as well as new ideas for solving complex segmentation tasks. The method is not only applicable to the study of C. elegans but also has wide applicability in biological image segmentation problems in other fields.

An alternate representation of the geomagnetic core field obtained using machine learning

Machine learning (ML) as a tool is rapidly emerging in various branches of contemporary geophysical research. To date, however, rarely has it been applied specifically for the study of Earth’s internal magnetic field and the geodynamo. Prevailing methods currently used in inferring the characteristic properties and the probable time evolution of the geodynamo are mostly based on reduced representations of magnetohydrodynamics (MHD). This study introduces a new inference method, referred to as Current Loop-based UNet Model Segmentation Inference (CLUMSI). Its long-term goal focuses on uncovering concentrations of electric current densities inside the core as the direct sources of the magnetic field itself, rather than computing the fluid motion using MHD. CLUMSI relies on simplified models in which equivalent current loops represent electric current systems emerging in turbulent geodynamo simulations. Various configurations of such loop models are utilized to produce synthetic magnetic field and secular variation (SV) maps computed at the core–mantle boundary (CMB). The resulting maps are then presented as training samples to an image-processing neural network designed specifically for solving image segmentation problems. This network essentially learns to infer the parameters and configuration of the loops in each model based on the corresponding CMB maps. In addition, with the help of the Domain Adversarial Training of Neural Networks (DANN) method during training, historical geomagnetic field data could also be considered alongside the synthetic samples. This implementation can increase the likelihood that a network trained primarily on synthetic data will appropriately handle real inputs. Our results focus mainly on the method's feasibility when applied to synthetic data and the quality of these inferences. A single evaluation of the trained network can recover the overall distribution of loop parameters with reasonable accuracy. To better represent conditions in the outer core, the study also proposes a computationally feasible process to account for magnetic diffusion and the corresponding induced currents in the loop models. However, the quality of the reconstruction of magnetic field properties is compromised by occasional poor inferences, and an inability to recover realistic SV.Graphical

New Tiger Beetle Algorithm for Cybersecurity, Medical Image Segmentation and Other Global Problems Optimization

The tiger beetle is a fierce and cunning predator insect that uses deception to hunt its prey. The tiger beetle traps and hunts them by digging holes along the path of other insects. This study has used the tiger beetle's hunting strategy to create the tiger beetle optimization (TBO) algorithm. In this algorithm, each solution represents the position of a tiger beetle, with the optimal position being the prey's location. Using this method, the tiger beetles gradually converge to the optimal solution, creating holes around them and searching for them. We evaluate the TBO algorithm's search capability using a series of well-known mathematical test functions. Moreover, Among the sophisticated forms of malware are polymorphic viruses, which are adept at changing their behaviour while maintaining the same essential functions. Thus, a machine learning-based malware analysis system utilizing the power of the proposed TBO is introduced in this article. Compared to other optimization methods, the proposed algorithm has shown less error in finding the optimal solution when implemented and evaluated on different functions. The tiger beetle optimization algorithm has proven helpful in various applications, including image clustering and reservoir well placement, where it can identify damaged areas or tissues with greater accuracy. When diagnosing lung cancer, the proposed method has shown a sensitivity, validity, and accuracy of 88.63\%, 87.58\%, and 89.86\%, respectively, using EBT, WKNN, ESKNM, and QSVM methods.

Chaotic opposition Golden Sinus Algorithm for global optimization problems

Optimization techniques are required to find the best solutions to challenging problems in many engineering disciplines. Metaheuristic algorithms that effectively increase search performance using various evolutionary strategies have become increasingly popular in recent years. The Golden Sinus Algorithm (GoldSa) is a population-based optimization algorithm that uses the sine function and the golden ratio. In this study, a chaotically enhanced opposition-based Golden Sinus Algorithm (Co-GoldSa) has been proposed to improve the efficiency of the exploitation and exploration ability of the GoldSa method. While designing this approach, it is first necessary to analyze the chaotic behavior of the GoldSa parameters. For this purpose, three chaotic GoldSa methods have been developed using eight chaotic maps to determine the effect of the chaotic maps on the parameters with different behaviors. Secondly, the opposition-based learning strategy is adjusted to the cGoldSa to enhance the searching ability. To investigate the proficiency of the proposed Co-GoldSa method, it has been examined with well-known and newly introduced metaheuristic approaches on benchmark functions and classical engineering design problems. Besides, an efficient framework of the multilevel thresholding image segmentation has been presented based on the Co-GoldSa method since the efficient processing of pathological images is quite important in medical diagnostics. The experimental outcomes reveal the superiority of the proposed method in solving global optimization problems, image segmentation, and engineering problems. Thus, the outcomes of the benchmark functions, image segmentation, and classical engineering problems support that the proposed Co-GoldSa approach can be considered a promising method for resolving challenging optimization problems.

RETRACTED: Multiview Type-2 kernelized fuzzy C-means clustering with local information for noisy color image segmentation

This article has been retracted. A retraction notice can be found at https://doi.org/10.3233/JIFS-219433.

Image Segmentation Based On U-Net and Adjusted U-Nets

Image segmentation, the process of dividing an image into various areas is quite crucial recently. Numerous industries, including robotics, remote sensing, and medical imaging, have applications for this task. In recent years, deep learning techniques, especially U-shaped networks (U-nets), have shown remarkable success in solving image segmentation problems. This paper provides an overview of image segmentation using neural networks, introduces different types of adjusted U-nets used for this task, including the implementation of attention gates and the use of residual neural network as the encoder path based on the original encoder-decoder structure, and then uses U-net and adjusted U-nets to conduct image segmentation on the black sea sprat. The study uses dice similarity coefficient and binary cross entropy loss function to compare the model training results and further judges the functionality of the models by the predicted segmented images. According to the test results, the Res34-UNet with attention gates performs most efficiently in segmenting this image dataset, although it's more unstable compared to the basic U-Net.

Model-based hybrid variational level set method applied to lung cancer detection

<p>The precise segmentation of lung lesions in computed tomography (CT) scans holds paramount importance for lung cancer research, offering invaluable information for clinical diagnosis and treatment. Nevertheless, achieving efficient detection and segmentation with acceptable accuracy proves to be challenging due to the heterogeneity of lung nodules. This paper presents a novel model-based hybrid variational level set method (VLSM) tailored for lung cancer detection. Initially, the VLSM introduces a scale-adaptive fast level-set image segmentation algorithm to address the inefficiency of low gray scale image segmentation. This algorithm simplifies the (Local Intensity Clustering) LIC model and devises a new energy functional based on the region-based pressure function. The improved multi-scale mean filter approximates the image’s offset field, effectively reducing gray-scale inhomogeneity and eliminating the influence of scale parameter selection on segmentation. Experimental results demonstrate that the proposed VLSM algorithm accurately segments images with both gray-scale inhomogeneity and noise, showcasing robustness against various noise types. This enhanced algorithm proves advantageous for addressing real-world image segmentation problems and nodules detection challenges.</p>

Few-shot segmentation based on multi-level and cross-scale clustering

The problem of image segmentation with few-shot learning is addressed in this paper, which is a challenging task due to the lack of sufficient high-precision annotated data. A novel method that consists of two modules is proposed: a multi-level fuzzy clustering guidance module and a cross-scale feature fusion module. The former module can extract image features in a class-independent feature space and fuse them with different scale information, while the latter module can reduce the information loss caused by cross-scale transmission. The feature association map between the support image and the query image can be learned by the proposed method, and the inconsistency of target object categories can be overcome. The proposed method is evaluated on Pascal and COCO datasets, and it is shown that it outperforms the state-of-the-art algorithms in both one-shot and k-shot segmentation scenarios.

Improving small sample medical image segmentation using CBAM: Insights from two datasets

Medical image segmentation is one of the key tasks in the medical field and is crucial for accurate lesion detection and treatment planning. However, the small sample problem has been one of the challenges in medical image segmentation. In this study, the small sample medical image segmentation problem is evaluated experimentally based on two different datasets, the VOC dataset and the esophageal medical images. In the VOC dataset experiments, we used the code from the BAM project on GitHub as a benchmark and compared it. Additionally, we improved the benchmark code by adding the Attention Mechanism (CBAM), placing its position before the support set undergoes global average pooling. By comparing the experimental results, we found that the model with the added CBAM achieved better segmentation results under small sample conditions. In the esophageal medical image experiments, the code from the BAM project was also used as a benchmark and a similar experimental design was performed. We further added CBAM by placing it before the global average pooling of the support set. The experimental results show that the model with the addition of CBAM achieves a significant improvement in the small sample segmentation task of esophageal medical images. To summarize, this study validates the effectiveness of using the BAM model in the small-sample medical image segmentation task on two different datasets. By adding CBAM, our model exhibits better segmentation performance under small sample conditions. These findings provide useful insights for small-sample medical image segmentation and are important for future research and practice.

Riemannian Manifold-Based Feature Space and Corresponding Image Clustering Algorithms.

Image feature representation is a key factor influencing the accuracy of clustering. Traditional point-based feature spaces represent spectral features of an image independently and introduce spatial relationships of pixels in the image domain to enhance the contextual information expression ability. Mapping-based feature spaces aim to preserve the structure information, but the complex computation and the unexplainability of image features have a great impact on their applications. To this end, we propose an explicit feature space called Riemannian manifold feature space (RMFS) to present the contextual information in a unified way. First, the Gaussian probability distribution function (pdf) is introduced to characterize the features of a pixel in its neighborhood system in the image domain. Then, the feature-related pdfs are mapped to a Riemannian manifold, which constructs the proposed RMFS. In RMFS, a point can express the complex contextual information of corresponding pixel in the image domain, and pixels representing the same object are linearly distributed. This gives us a chance to convert nonlinear image segmentation problems to linear computation. To verify the superiority of the expression ability of the proposed RMFS, a linear clustering algorithm and a fuzzy linear clustering algorithm are proposed. Experimental results show that the proposed RMFS-based algorithms outperform their counterparts in the spectral feature space and the RMFS-based ones without the linear distribution characteristics. This indicates that the RMFS can better express features of an image than spectral feature space, and the expressed features can be easily used to construct linear segmentation models.