Image Segmentation Model Research Articles

Interactive image segmentation based field boundary perception method and software for autonomous agricultural machinery path planning

Autonomous agricultural machinery path planning requires high-precision field boundary information. To address the challenge of rapidly acquiring accurate information about different types of land objects in complex field scenarios, this study introduces an interactive segmentation-based method and software for agricultural field boundary perception, specifically designed for high-resolution orthophotos. The method aims to accurately delineate various agricultural objects in the image, such as crops, soil, roads, edges, exits to fields, and obstacles. Compared to previous research on agricultural object detection and segmentation, this study proposes an interactive deep image segmentation model for perceiving multiple types of agricultural features. During the image segmentation process, manually adding positive and negative points provides supervised information for the segmentation of agricultural images. In addition, this research uses the PaddlePaddle deep learning framework to implement the proposed method and extends the open-source software EISeg to develop a dedicated tool for agricultural image segmentation. Through 3 to 4 interactive iterations, the method achieves an impressive mean Intersection over Union (mIoU) segmentation accuracy of about 90%. The model's average inference time on the training server was 0.197 s, meeting the real-time requirements of the interactive segmentation method. By accurately segmenting agricultural land features from high-resolution orthoimagery, the proposed method can provide valuable support for the construction of high-precision navigation maps for autonomous agricultural machinery.

A Level Set-Based Model for Image Segmentation under Geometric Constraints and Data Approximation.

In this paper, we propose a new model for image segmentation under geometric constraints. We define the geometric constraints and we give a minimization problem leading to a variational equation. This new model based on a minimal surface makes it possible to consider many different applications from image segmentation to data approximation.

Deep Learning-Based Hip X-ray Image Analysis for Predicting Osteoporosis

Osteoporosis is a common problem in orthopedic medicine, and it has become an important medical issue in orthopedics as Taiwan is gradually becoming an aging society. In the diagnosis of osteoporosis, the bone mineral density (BMD) derived from dual-energy X-ray absorptiometry (DXA) is the main criterion for orthopedic diagnosis of osteoporosis, but due to the high cost of this equipment and the lower penetration rate of the equipment compared to the X-ray images, the problem of osteoporosis has not been effectively solved for many people who suffer from osteoporosis. At present, in clinical diagnosis, doctors are not yet able to accurately interpret X-ray images for osteoporosis manually and must rely on the data obtained from DXA. In recent years, with the continuous development of artificial intelligence, especially in the fields of machine learning and deep learning, significant progress has been made in image recognition. Therefore, it is worthwhile to revisit the question of whether it is possible to use a convolutional neural network model to read a hip X-ray image and then predict the patient’s BMD. In this study, we proposed a hip X-ray image segmentation model and a hip X-ray image recognition classification model. First, we used the U-Net model as a framework to segment the femoral neck, greater trochanter, Ward’s triangle, and the total hip in the hip X-ray images. We then performed image matting and data augmentation. Finally, we constructed a predictive model for osteoporosis using deep learning algorithms. In the segmentation experiments, we used intersection over union (IoU) as the evaluation metric for image segmentation, and both the U-Net model and the U-Net++ model achieved segmentation results greater than or equal to 0.5. In the classification experiments, using the T-score as the classification basis, the total hip using the DenseNet121 model has the highest accuracy of 74%.

An attention-based dense network model for cardiac image segmentation using learning approaches

An attention-based dense network model for cardiac image segmentation using learning approaches

Image Segmentation of Triple-Negative Breast Cancer by Incorporating Multiscale and Parallel Attention Mechanisms

Breast cancer is a highly prevalent cancer. Triple-negative breast cancer (TNBC) is more likely to recur and metastasize than other subtypes of breast cancer. Research on the treatment of TNBC is of great importance, and accurate segmentation of the breast lesion area is an important step in the treatment of TNBC. Currently, the gold standard for tumor segmentation is still sketched manually by doctors, which requires expertise in the field of medical imaging and consumes a great deal of doctors’ time and energy. Automatic segmentation of breast cancer not only reduces the burden of doctors but also improves work efficiency. Therefore, it is of great significance to study the automatic segmentation technique for breast cancer lesion regions. In this paper, a deep-learning-based automatic segmentation algorithm for TNBC images is proposed. The experimental data were dynamic contrast-enhanced magnetic resonance imaging TNBC dataset provided by the Cancer Hospital of Zhengzhou University. The experiments were analyzed by comparing several models with UNet, Attention-UNet, ResUNet, and SegNet and using evaluation indexes such as Dice score and Iou. Compared to UNet, Attention-UNet, ResUNet, and SegNet, the proposed method improved the Dice score by 2.1%, 1.54%, 0.88%, and 9.65%, respectively. The experimental results show that the proposed deep-learning-based TNBC image segmentation model can effectively improve the segmentation performance of TNBC tumors.

Selective image segmentation driven by region, edge and saliency functions

Present active contour methods often struggle with the segmentation of regions displaying variations in texture, color, or intensity a phenomenon referred to as inhomogeneities. These limitation impairs their ability to precisely distinguish and outline diverse components within an image. Further some of these methods employ intricate mathematical formulations for energy minimization. Such complexity introduces computational sluggishness, making these methods unsuitable for tasks requiring real-time processing or rapid segmentation. Moreover, these methods are susceptible to being trapped in energy configurations corresponding to local minimum points. Consequently, the segmentation process fails to converge to the desired outcome. Additionally, the efficacy of these methods diminishes when confronted with regions exhibiting weak or subtle boundaries. To address these limitations comprehensively, our proposed approach introduces a fresh paradigm for image segmentation through the synchronization of region-based, edge-based, and saliency-based segmentation techniques. Initially, we adapt an intensity edge term based on the zero crossing feature detector (ZCD), which is used to highlight significant edges of an image. Secondly, a saliency function is formulated to detect salient regions from an image. We have also included a globally tuned region based SPF (signed pressure force) term to move contour away and capture homogeneous regions. ZCD, saliency and global SPF are jointly incorporated with some scaled value for the level set evolution to develop an effective image segmentation model. In addition, proposed method is capable to perform selective object segmentation, which enables us to choose any single or multiple objects inside an image. Saliency function and ZCD detector are considered feature enhancement tools, which are used to get important features of an image, so this method has a solid capacity to segment nature images (homogeneous or inhomogeneous) precisely. Finally, the adaption of the Gaussian kernel removes the need of any penalization term for level set reinitialization. Experimental results will exhibit the efficiency of the proposed method.

Evaluating synthetic neuroimaging data augmentation for automatic brain tumour segmentation with a deep fully-convolutional network

Gliomas observed in medical images require expert neuro-radiologist evaluation for treatment planning and monitoring, motivating development of intelligent systems capable of automating aspects of tumour evaluation. Deep learning models for automatic image segmentation rely on the amount and quality of training data. In this study we developed a neuroimaging synthesis technique to augment data for training fully-convolutional networks (U-nets) to perform automatic glioma segmentation. We used StyleGAN2-ada to simultaneously generate fluid-attenuated inversion recovery (FLAIR) magnetic resonance images and corresponding glioma segmentation masks. Synthetic data were successively added to real training data (n = 2751) in fourteen rounds of 1000 and used to train U-nets that were evaluated on held-out validation (n = 590) and test sets (n = 588). U-nets were trained with and without geometric augmentation (translation, zoom and shear), and Dice coefficients were computed to evaluate segmentation performance. We also monitored the number of training iterations before stopping, total training time, and time per iteration to evaluate computational costs associated with training each U-net. Synthetic data augmentation yielded marginal improvements in Dice coefficients (validation set +0.0409, test set +0.0355), whereas geometric augmentation improved generalization (standard deviation between training, validation and test set performances of 0.01 with, and 0.04 without geometric augmentation). Based on the modest performance gains for automatic glioma segmentation we find it hard to justify the computational expense of developing a synthetic image generation pipeline. Future work may seek to optimize the efficiency of synthetic data generation for augmentation of neuroimaging data.

Medical image segmentation of gastric adenocarcinoma based on dense connection of residuals.

Accurate segmentation of gastric cancer based on CT images of gastric adenocarcinoma is crucial for physicians to screen gastric diseases, clinical diagnosis, preoperative prediction, and postoperative evaluation plans. To address the issue of the inability of the segmentation algorithm to depict the correct boundaries due to unclear gastric contours in the lesion area and the visible irregular band-like dense shadow extending to the perigastric region, a 3D medical image segmentation model 3D UNet based on residual dense jumping method is proposed. In the method we proposed, Residual Dense Block, which is applied to the image super-resolution module to remove CT artifacts, and Residual Block in ResNet are further fused. The quality of CT images is improved by Residual Dense Skip Block, which removes banded dense shadows, preserves image details and edge information, captures features, and improves the segmentation performance of gastric adenocarcinoma. The Instance Normalization layer position is modified to select the best result. Different loss functions are also combined in order to obtain the best gastric adenocarcinoma segmentation performance. We tested the model on a hospital-provided gastric adenocarcinoma dataset. The experimental results show that our model outperforms the existing methods in CT gastric adenocarcinoma segmentation, in which the method combining the hybrid loss function of Dice and CE obtains an average dice score of 82.3%, which is improved by 5.3% and 3.8% compared to TransUNet and Hiformer, respectively, and improves the cross-merge rate to 70.8%, compared to nnFormer, nnUNet by 1% and 0.9%, respectively. The residual jump connection structure indeed improves segmentation performance. The proposed method has the potential to be used as a screen for gastric diseases and to assist physicians in diagnosis.

Enhancing breast ultrasound segmentation through fine-tuning and optimization techniques: Sharp attention UNet.

Segmentation of breast ultrasound images is a crucial and challenging task in computer-aided diagnosis systems. Accurately segmenting masses in benign and malignant cases and identifying regions with no mass is a primary objective in breast ultrasound image segmentation. Deep learning (DL) has emerged as a powerful tool in medical image segmentation, revolutionizing how medical professionals analyze and interpret complex imaging data. The UNet architecture is a highly regarded and widely used DL model in medical image segmentation. Its distinctive architectural design and exceptional performance have made it popular among researchers. With the increase in data and model complexity, optimization and fine-tuning models play a vital and more challenging role than before. This paper presents a comparative study evaluating the effect of image preprocessing and different optimization techniques and the importance of fine-tuning different UNet segmentation models for breast ultrasound images. Optimization and fine-tuning techniques have been applied to enhance the performance of UNet, Sharp UNet, and Attention UNet. Building upon this progress, we designed a novel approach by combining Sharp UNet and Attention UNet, known as Sharp Attention UNet. Our analysis yielded the following quantitative evaluation metrics for the Sharp Attention UNet: the Dice coefficient, specificity, sensitivity, and F1 score values obtained were 0.93, 0.99, 0.94, and 0.94, respectively. In addition, McNemar's statistical test was applied to assess significant differences between the approaches. Across a number of measures, our proposed model outperformed all other models, resulting in improved breast lesion segmentation.

Expert teacher based on foundation image segmentation model for object detection in aerial images

Despite the remarkable progress of general object detection, the lack of labeled aerial images limits the robustness and generalization of the detector. Teacher–student learning is a feasible solution on natural image domain, but few works focus on unlabeled aerial images. Inspired by foundation models with the powerful generalization in computer vision field, we propose an expert teacher framework based on foundation image segmentation model called ET-FSM. Our approach provides the performance gains for the student detector by generating high-quality pseudo-labels for unlabeled aerial images. In the ET-FSM, we design the binary detector with expert guidance mechanism to sufficiently leverage the extra knowledge obtained from the foundation image segmentation model, which accurately detects object positions in the complex backgrounds. Also, we present the momentum contrast classification module to distinguish confused object categories in aerial images. To demonstrate the effectiveness of the proposed method, we construct an unlabeled aerial image dataset covering various scenes. The experiments are conducted on diverse types of student detectors. The results show that the proposed approach achieves superior performance compared to existing methods, and allows the student detector to achieve fully supervised performance with much less labeled aerial images. Our dataset and code are available at https://github.com/cq100/ET-FSM.

Segment anything model for medical images?

The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging because of the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. To fully validate SAM’s performance on medical data, we collected and sorted 53 open-source datasets and built a large medical segmentation dataset with 18 modalities, 84 objects, 125 object-modality paired targets, 1050K 2D images, and 6033K masks. We comprehensively analyzed different models and strategies on the so-called COSMOS 1050K dataset. Our findings mainly include the following: (1) SAM showed remarkable performance in some specific objects but was unstable, imperfect, or even totally failed in other situations. (2) SAM with the large ViT-H showed better overall performance than that with the small ViT-B. (3) SAM performed better with manual hints, especially box, than the Everything mode. (4) SAM could help human annotation with high labeling quality and less time. (5) SAM was sensitive to the randomness in the center point and tight box prompts, and may suffer from a serious performance drop. (6) SAM performed better than interactive methods with one or a few points, but will be outpaced as the number of points increases. (7) SAM’s performance correlated to different factors, including boundary complexity, intensity differences, etc. (8) Finetuning the SAM on specific medical tasks could improve its average DICE performance by 4.39% and 6.68% for ViT-B and ViT-H, respectively. Codes and models are available at: https://github.com/yuhoo0302/Segment-Anything-Model-for-Medical-Images. We hope that this comprehensive report can help researchers explore the potential of SAM applications in MIS, and guide how to appropriately use and develop SAM.

A novel full-convolution UNet-transformer for medical image segmentation

The Transformer-based methods are still unable to effectively model local contexts although they make up for the deficiency of remote information dependencies for approaches based on small kernel CNNs. To overcome such a shortage, this paper proposes a novel full-convolution UNet Transformer model, FC-UNETTR, for medical image segmentation. First, a novel global–local attention module is proposed by utilizing multiple small kernels of different sizes for depth-wise convolutions to expand the receptive field of the network model, increase the remote dependence of semantic information in the encoder stage, and also improve the feature extraction capability of the network for fuzzy edges. Then, a reparametrized feedforward network is developed to further improve the local information extraction and mitigate the coupling between feature maps such that the relationship between feature map channels can be better revealed. Furthermore, the skip connection and decoder are redesigned by constructing a dense multiscale module instead of traditional ResNet modules to mitigate semantic bias. Benefiting from the above improvements, the constructed FC-UNETTR without pre-training demonstrates a strong capability to extract local features and capture long-range dependencies of images in medical image segmentation. Experiments show that FC-UNETTR achieves an excellent performance of 85.67% for DSC and 7.82 for HD metrics on the Synapse dataset with fewer model parameters compared with state-of-the-art networks. Furthermore, DSC reaches 92.46% and 94.76% on the ACDC dataset and the private dataset of oral graft bone, respectively, outperforming some of the latest medical image segmentation models..

Retracted: Construction of Computer Microscope Image Segmentation Model Based on Fourth-Order Partial Differential Equation Smoothing

Retracted: Construction of Computer Microscope Image Segmentation Model Based on Fourth-Order Partial Differential Equation Smoothing

DSLSM: Dual-kernel-induced statistic level set model for image segmentation

Even though the level set method is driving progress in image segmentation, its performance is still affected adversely in the presence of severe intensity inhomogeneity and multi-noise. Therefore, developing an accurate, robust, and fast technique to solve this becomes a pressing need. In this paper, we propose DSLSM, a novel unsupervised statistic level set model based on dual-kernel-induced local similarity measures, which not only can effectively handle severe intensity inhomogeneity and multi-noise, but also stabilizes and accelerates the evolution process. Specifically, we first define a novel dual-kernel-induced data term, which stacks multiple kernel functions and manipulates local similarity measurement via the Jensen–Shannon divergence, aiming at extracting more discriminative higher-dimension features and correlations between adjacent pixels. Then, a novel rectified morphological scheme is proposed to numerically implement our method, which not only precludes the issues of numerical instability and level set function degradation but significantly accelerates the contour evolution. In addition, an inverted pyramid-style scheduling strategy is proposed to explore the complementary utilization of local and non-local contextual features at different evolution phases to avoid tedious heuristics parameter tuning for the size of local regions. Extensive experiments on multiple challenging segmentation benchmarks demonstrate the effectiveness and superiority of our DSLSM over recent state-of-the-art methods.

UPL-SFDA: Uncertainty-Aware Pseudo Label Guided Source-Free Domain Adaptation for Medical Image Segmentation.

Domain Adaptation (DA) is important for deep learning-based medical image segmentation models to deal with testing images from a new target domain. As the source-domain data are usually unavailable when a trained model is deployed at a new center, Source-Free Domain Adaptation (SFDA) is appealing for data and annotation-efficient adaptation to the target domain. However, existing SFDA methods have a limited performance due to lack of sufficient supervision with source-domain images unavailable and target-domain images unlabeled. We propose a novel Uncertainty-aware Pseudo Label guided (UPL) SFDA method for medical image segmentation. Specifically, we propose Target Domain Growing (TDG) to enhance the diversity of predictions in the target domain by duplicating the pre-trained model's prediction head multiple times with perturbations. The different predictions in these duplicated heads are used to obtain pseudo labels for unlabeled target-domain images and their uncertainty to identify reliable pseudo labels. We also propose a Twice Forward pass Supervision (TFS) strategy that uses reliable pseudo labels obtained in one forward pass to supervise predictions in the next forward pass. The adaptation is further regularized by a mean prediction-based entropy minimization term that encourages confident and consistent results in different prediction heads. UPL-SFDA was validated with a multi-site heart MRI segmentation dataset, a cross-modality fetal brain segmentation dataset, and a 3D fetal tissue segmentation dataset. It improved the average Dice by 5.54, 5.01 and 6.89 percentage points for the three tasks compared with the baseline, respectively, and outperformed several state-of-the-art SFDA methods.

USE-Evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging.

Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and the difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics used to assess performance fail to capture the impact of this mismatch, particularly when dealing with datasets in clinical settings that involve challenging segmentation tasks, pathologies with low signal, and reference annotations that are uncertain, small, or empty. Limitations of common metrics may result in ineffective machine learning research in designing and optimizing models. To effectively evaluate the clinical value of such models, it is essential to consider factors such as the uncertainty associated with reference annotations, the ability to accurately measure performance regardless of the size of the reference annotation volume, and the classification of cases where reference annotations are empty. We study how uncertain, small, and empty reference annotations influence the value of metrics on a stroke in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify suitable metrics in such a setting. We compare our results to the BRATS 2019 and Spinal Cord public data sets. We show how uncertain, small, or empty reference annotations require a rethinking of the evaluation. The evaluation code was released to encourage further analysis of this topic https://github.com/SophieOstmeier/UncertainSmallEmpty.git.

Learning from partially labeled data for multi-organ and tumor segmentation.

Medical image benchmarks for the segmentation of organs and tumors suffer from the partially labeling issue due to its intensive cost of labor and expertise. Current mainstream approaches follow the practice of one network solving one task. With this pipeline, not only the performance is limited by the typically small dataset of a single task, but also the computation cost linearly increases with the number of tasks. To address this, we propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple partially labeled datasets. Specifically, TransDoDNet has a hybrid backbone that is composed of the convolutional neural network and Transformer. A dynamic head enables the network to accomplish multiple segmentation tasks flexibly. Unlike existing approaches that fix kernels after training, the kernels in the dynamic head are generated adaptively by the Transformer, which employs the self-attention mechanism to model long-range organ-wise dependencies and decodes the organ embedding that can represent each organ. We create a large-scale partially labeled Multi-Organ and Tumor Segmentation benchmark, termed MOTS, and demonstrate the superior performance of our TransDoDNet over other competitors on seven organ and tumor segmentation tasks. This study also provides a general 3D medical image segmentation model, which has been pre-trained on the large-scale MOTS benchmark and has demonstrated advanced performance over current predominant self-supervised learning methods. Code and data are available at https://github.com/jianpengz/DoDNet.

Deep learning muscle segmentation model for CT images in DICOM format

This work solves the problem of automatic segmentation of medical images in DICOM format using machine learning methods. A new developed tool is used in the form of a separate module for labeling medical data in the DICOM format. The trained model, proposed in the paper, can be useful in the tasks of muscle segmentation. One can apply it in different ways, but some of the most common include assessment of diseases related to muscles, and sarcopenia is one of them. The further applications of the muscle segmentation model may include examining various medical cases with patients, that tend to have muscle-related diseases. For instance, detecting cachexia may be one of the extensions of the model’s application field.

Multilevel thresholding image segmentation algorithm based on Mumford–Shah model

Abstract Image segmentation is one of the important tasks of computer vision and computer image processing, and the purpose of image segmentation is to achieve the extraction and recognition of the target image region. The classical Mumford–Shah (MSh) image segmentation model is used to achieve the segmentation of images. With the goal to get the best segmentation effect on images by minimizing the MSh energy generalization function, a level set strategy is developed, and a model with global information infinite curve evolution is utilized. However, considering the low efficiency of this model for processing level set curves and the general quality of image segmentation. A multi-layer threshold search scheme is proposed to achieve rapid convergence of the target image level set curve. The experimental results showed that the multi-level thresholding image segmentation algorithm based on the MSh model can significantly improve the segmentation effect of images and reduce the segmentation time. The suggested MSK method outperforms the MPO algorithm, SSA algorithm, and EMO algorithm in the picture segmentation convergence time test, respectively, in terms of runtime efficiency by 356, 289, and 71%. Additionally, it performs superbly in both threshold searches and picture quality tests. The research topic has significant reference value for the study of contemporary computer vision imaging technologies.

Segmentation of lung lobes and lesions in chest CT for the classification of COVID-19 severity

To precisely determine the severity of COVID-19-related pneumonia, computed tomography (CT) is an imaging modality beneficial for patient monitoring and therapy planning. Thus, we aimed to develop a deep learning-based image segmentation model to automatically assess lung lesions related to COVID-19 infection and calculate the total severity score (TSS). The entire dataset consisted of 124 COVID-19 patients acquired from Chulabhorn Hospital, divided into 28 cases without lung lesions and 96 cases with lung lesions categorized severity by radiologists regarding TSS. The model used a 3D-UNet along with DenseNet and ResNet models that had already been trained to separate the lobes of the lungs and figure out the percentage of lung involvement due to COVID-19 infection. It also used the Dice similarity coefficient (DSC) to measure TSS. Our final model, consisting of 3D-UNet integrated with DenseNet169, achieved segmentation of lung lobes and lesions with the Dice similarity coefficients of 91.52% and 76.89%, respectively. The calculated TSS values were similar to those evaluated by radiologists, with an R2 of 0.842. The correlation between the ground-truth TSS and model prediction was greater than that of the radiologist, which was 0.890 and 0.709, respectively.