Image Segmentation Model Research Articles

Using diffusion models to generate synthetic labeled data for medical image segmentation.

Medical image analysis has become a prominent area where machine learning has been applied. However, high-quality, publicly available data are limited either due to patient privacy laws or the time and cost required for experts to annotate images. In this retrospective study, we designed and evaluated a pipeline to generate synthetic labeled polyp images for augmenting medical image segmentation models with the aim of reducing this data scarcity. We trained diffusion models on the HyperKvasir dataset, comprising 1000 images of polyps in the human GI tract from 2008 to 2016. Qualitative expert review, Fréchet Inception Distance (FID), and Multi-Scale Structural Similarity (MS-SSIM) were tested for evaluation. Additionally, various segmentation models were trained with the generated data and evaluated using Dice score (DS) and Intersection over Union (IoU). Our pipeline produced images more akin to real polyp images based on FID scores. Segmentation model performance also showed improvements over GAN methods when trained entirely, or partially, with synthetic data, despite requiring less compute for training. Moreover, the improvement persists when tested on different datasets, showcasing the transferability of the generated images. The proposed pipeline produced realistic image and mask pairs which could reduce the need for manual data annotation when performing a machine learning task. We support this use case by showing that the methods proposed in this study enhanced segmentation model performance, as measured by Dice and IoU scores, when trained fully or partially on synthetic data.

Source domain prior-assisted segment anything model for single domain generalization in medical image segmentation

Deep learning-based medical image segmentation models often suffer from performance degradation across domains due to domain discrepancies arising from data collected by various healthcare centers. Recent advancements, particularly the Segment Anything Model (SAM), have shown promising generalization abilities in this field, inspiring the development of several SAM-based approaches to address domain discrepancy issue. Nevertheless, these methods seldom exploit the full potential of the rich source domain knowledge to improve segmentation accuracy in unseen target domains. In this paper, we propose a source domain prior-assisted module for generalizable medical image segmentation based on SAM. Specifically, we store diverse features of the source domain data in a memory bank. When applying the model to the target domain, the features of the target domain first match with the features in the memory bank to obtain invaluable prior information. This strategy enables the model to utilize the prior information from the source domain to adapt to the target domain. We validate the proposed method on two widely used medical image segmentation tasks across multiple domains and achieve state-of-the-art performance.

Semantic segmentation and thermal imaging for forest fires detection and monitoring by drones

Forest ecosystems play a crucial role in providing a wide range of ecological, social, and economic benefits. However, the increasing frequency and severity of forest fires pose a significant threat to the sustainability of forests and their functions, highlighting the need for early detection and swift action to mitigate damage. The combination of drones and artificial intelligence, particularly deep learning, proves to be a cost-effective solution for accurately and efficiently detecting forest fires in real-time. Deep learning-based image segmentation models can not only be employed for forest fire detection but also play a vital role in damage assessment and support reforestation efforts. Furthermore, the integration of thermal cameras on drones can significantly enhance the sensitivity in forest fire detection. This study undertakes an in-depth analysis of recent advancements in deep learning-based semantic segmentation, with a particular focus on model’s mask region convolutional neural network (Mask R-CNN) and you only look once (YOLO) v5, v7, and v8 variants. Emphasis is placed on their suitability for forest fire monitoring using drones equipped with RGB and/or thermal cameras. The conducted experiments have yielded encouraging outcomes across various metrics, underscoring its significance as an invaluable asset for both fire detection and continuous monitoring endeavors.

Digital Medical Images Segmentation by Active Contour Model based on the Signed Pressure Force Function

The signed pressure force (SPF) function has recently become a popular function for guiding the curve evolution of the active contour model (ACM) for image segmentation. The aim is to extract the boundaries of digital medical images for shape and image analysis. The recent SPF-based ACM demonstrates effectiveness in image segmentation. However, it may fail if the targeted object is close to a neighbouring object. Additionally, the presence of intensity inhomogeneity and noise in medical images degrades segmentation accuracy and local target areas. Thus, we proposed a new SPF-based ACM, namely the Selective Segmentation with Signed Pressure Force 1 (SSPF1) model, by incorporating the ideas of the SPF function and the distance fitting term based on geometrical constraints. Then, the new SSPF1 model was extended by incorporating an image enhancement technique to develop our second new model, termed the Selective Segmentation with Signed Pressure Force 2 (SSPF2). Numerical results indicated that the SSPF2 model was more recommended than SSPF1 as the SSPF2 model was approximately 4.7% more accurate, as indicated by the Jaccard value and was about 112 times faster in segmenting noisy images compared to the existing selective segmentation model.

Boundary guidance network for medical image segmentation

Accurate segmentation of the tumor area is crucial for the treatment and prognosis of patients with bladder cancer. Cystoscopy is the gold standard for diagnosing bladder tumors. However, The vast majority of current work uses deep learning to identify and segment tumors from CT and MRI findings, and rarely involves cystoscopy findings. Accurately segmenting bladder tumors remains a great challenge due to their diverse morphology and fuzzy boundaries. In order to solve the above problems, this paper proposes a medical image segmentation network with boundary guidance called boundary guidance network. This network combines local features extracted by CNNs and long-range dependencies between different levels inscribed by Parallel ViT, which can capture tumor features more effectively. The Boundary extracted module is designed to extract boundary features and utilize the boundary features to guide the decoding process. Foreground-background dual-channel decoding is performed by boundary integrated module. Experimental results on our proposed new cystoscopic bladder tumor dataset (BTD) show that our method can efficiently perform accurate segmentation of tumors and retain more boundary information, achieving an IoU score of 91.3%, a Hausdorff Distance of 10.43, an mAP score of 85.3%, and a F1 score of 94.8%. On BTD and three other public datasets, our model achieves the best scores compared to state-of-the-art methods, which proves the effectiveness of our model for common medical image segmentation.

Intelligent assessment system of material deterioration in masonry tower based on improved image segmentation model

Accurate and timely data collection of material deterioration on the surfaces of architectural heritage is crucial for effective conservation and restoration. Traditional methods rely heavily on extensive field surveys and manual feature identification, which are significantly affected by objective conditions and subjective factors. While machine vision-based methods can help address these issues, the accuracy, intelligence, and systematic nature of material deterioration assessment for large-scale masonry towers with complex geometries still require significant improvement. This research focuses on the architectural heritage of masonry towers and proposes an intelligent assessment system that integrates an improved YOLOv8-seg machine vision image segmentation model with refined 3D reconstruction technology. By optimizing the YOLOv8-seg model, the system enhances the extraction capabilities of both detailed and global features of material deterioration in masonry towers. Furthermore, by complementing it with image processing methods for the global visualization of large-scale objects, this research constructs a comprehensive intelligent assessment process that includes "deterioration feature extraction—global visualization—quantitative and qualitative comprehensive assessment." Experimental results demonstrate that the intelligent assessment system significantly improves the performance of target feature extraction for material deterioration in masonry towers compared to existing methods. The improved model shows improvements of 3.39% and 4.55% in the key performance metrics of mAP50 and mAP50-95, respectively, over the baseline model. Additionally, the efficiency of global feature extraction and visualization of material deterioration increased by 66.36%, with an average recognition accuracy of 95.78%. Consequently, this system effectively overcomes the limitations and subjective influences of field surveys, enhancing the objectivity and efficiency of identifying and analyzing material deterioration in masonry towers, and providing invaluable data support for the subsequent preservation and restoration efforts.

A Method for Extracting Joints on Mountain Tunnel Faces Based on Mask R-CNN Image Segmentation Algorithm

The accurate distribution of joints on the tunnel face is crucial for assessing the stability and safety of surrounding rock during tunnel construction. This paper introduces the Mask R-CNN image segmentation algorithm, a state-of-the-art deep learning model, to achieve efficient and accurate identification and extraction of joints on tunnel face images. First, digital images of tunnel faces were captured and stitched, resulting in 286 complete images suitable for analysis. Then, the joints on the tunnel face were extracted using traditional image processing algorithms, the commonly used U-net image segmentation model, and the Mask R-CNN image segmentation model introduced in this paper to address the lack of recognition accuracy. Finally, the extraction results obtained by the three methods were compared. The comparison results show that the joint extraction method based on the Mask R-CNN image segmentation deep learning model introduced in this paper achieved the best joint extraction effect with a Dice similarity coefficient of 87.48%, outperforming traditional methods and the U-net model, which scored 60.59% and 75.36%, respectively, realizing accurate and efficient acquisition of tunnel face rock joints. These findings suggest that the Mask R-CNN model can be effectively implemented in real-time monitoring systems for tunnel construction projects.

KACM: A KIS-awared active contour model for low-contrast image segmentation

Low-contrast image segmentation is a challenging task, as it requires distinguishing target objects from the background with minimal intensity differences, while also dealing with factors such as noise and blur. In recent years, the active contour model (ACM) has gained popularity for its high accuracy and efficiency in image segmentation. However, it often struggles with low-contrast images segmentation. To address this issue, we propose a KIS-awared ACM (KACM) in this paper. Firstly, based on the Koschmieder imaging system (KIS), the observed low-contrast image is modeled as a combination of the true image and two imaging factors: the scene transmission function and the global intensity of light in the imaging environment. Next, we pursue the true image in the logarithmic domain using the maximum a posteriori (MAP) criterion, and establish the active contour model. In this model, the true image follows a piecewise lognormal distribution, while the scene transmission function is described as a Markov random field (MRF), with its prior probability being defined as a Gibbs energy function. Lastly, an alternating iterative algorithm that combines variation calculus and gradient descent of the three-step time-splitting method is introduced to solve the proposed model. We validate the proposed model through qualitative and quantitative experiments, demonstrating its effectiveness in low-contrast image segmentation. Compared to several state-of-the-art models, the proposed KACM exhibits competitive performance in terms of both accuracy and efficiency.

A Deep Learning Approach to Segment Coastal Marsh Tidal Creek Networks from High-Resolution Aerial Imagery

Tidal creeks play a vital role in influencing geospatial evolution and marsh ecological communities in coastal landscapes. However, evaluating the geospatial characteristics of numerous creeks across a site and understanding their ecological relationships pose significant challenges due to the labor-intensive nature of manual delineation from imagery. Traditional methods rely on manual annotation in GIS interfaces, which is slow and tedious. This study explores the application of Attention-based Dense U-Net (ADU-Net), a deep learning image segmentation model, for automatically classifying creek pixels in high-resolution (0.5 m) orthorectified aerial imagery in coastal Georgia, USA. We observed that ADU-Net achieved an outstanding F1 score of 0.98 in identifying creek pixels, demonstrating its ability in tidal creek mapping. The study highlights the potential of deep learning models for automated tidal creek mapping, opening avenues for future investigations into the role of creeks in marshes’ response to environmental changes.

Narrowing the semantic gaps in U-Net with learnable skip connections: The case of medical image segmentation

Current state-of-the-art medical image segmentation techniques predominantly employ the encoder–decoder architecture. Despite its widespread use, this U-shaped framework exhibits limitations in effectively capturing multi-scale features through simple skip connections. In this study, we made a thorough analysis to investigate the potential weaknesses of connections across various segmentation tasks, and suggest two key aspects of potential semantic gaps crucial to be considered: the semantic gap among multi-scale features in different encoding stages and the semantic gap between the encoder and the decoder. To bridge these semantic gaps, we introduce a novel segmentation framework, which incorporates a Dual Attention Transformer module for capturing channel-wise and spatial-wise relationships, and a Decoder-guided Recalibration Attention module for fusing DAT tokens and decoder features. These modules establish a principle of learnable connection that resolves the semantic gaps, leading to a high-performance segmentation model for medical images. Furthermore, it provides a new paradigm for effectively incorporating the attention mechanism into the traditional convolution-based architecture. Comprehensive experimental results demonstrate that our model achieves consistent, significant gains and outperforms state-of-the-art methods with relatively fewer parameters. This study contributes to the advancement of medical image segmentation by offering a more effective and efficient framework for addressing the limitations of current encoder–decoder architectures. Code: https://github.com/McGregorWwww/UDTransNet.

G-SAM: GMM-based segment anything model for medical image classification and segmentation

G-SAM: GMM-based segment anything model for medical image classification and segmentation

Image Segmentation Model for Vicinagearth Security Technology of Unmanned Aerial Vehicle Using Improved Pigeon-Inspired Optimization

The vicinagearth security technology system covers a wide range of fields such as low-altitude security, underwater security, and cross-domain security. Among them, unmanned aerial vehicle (UAV) security will become one of the evolving forms of its security technology, and how to improve the segmentation and recognition ability of UAV visual reconnaissance system for maritime targets through improvement will become the key to low-altitude security. Due to the fact that maritime target images are characterized by complex weather, strong interference, high speed requirement and large data volume, the traditional segmentation methods are not suitable for maritime small-target (MST) segmentation and recognition. Therefore, this paper proposes a threshold image segmentation (TIS) method based on an improved pigeon-inspired optimization (PIO) algorithm to provide a better method for segmentation and recognition of MST. First, this study proposes CCPIO based on the horizontal crossover search (HCS) and vertical crossover search (VCS) strategy, which effectively improves the search efficiency of PIO and the ability to jump out of local optimum. And the optimization performance of CCPIO is effectively verified by comparing it with 10 peer algorithms through benchmark function experiments. Further, in this paper, the proposed CCPIO-TIS segmentation model is proposed by combining CCPIO with non-local means, 2D histogram, and Kapur’s entropy. The proposed CCPIO-TIS model is also used for the segmentation and recognition of real MST images, and the results of the experimental comparison and evaluation analysis show that the proposed model has higher quality segmentation results than 12 models of the same type. In summary, this study can provide an efficient and accurate artificial intelligence model for segmentation and recognition of maritime small-target.

DFA-UNet: dual-stream feature-fusion attention U-Net for lymph node segmentation in lung cancer diagnosis.

In bronchial ultrasound elastography, accurately segmenting mediastinal lymph nodes is of great significance for diagnosing whether lung cancer has metastasized. However, due to the ill-defined margin of ultrasound images and the complexity of lymph node structure, accurate segmentation of fine contours is still challenging. Therefore, we propose a dual-stream feature-fusion attention U-Net (DFA-UNet). Firstly, a dual-stream encoder (DSE) is designed by combining ConvNext with a lightweight vision transformer (ViT) to extract the local information and global information of images; Secondly, we propose a hybrid attention module (HAM) at the bottleneck, which incorporates spatial and channel attention to optimize the features transmission process by optimizing high-dimensional features at the bottom of the network. Finally, the feature-enhanced residual decoder (FRD) is developed to improve the fusion of features obtained from the encoder and decoder, ensuring a more comprehensive integration. Extensive experiments on the ultrasound elasticity image dataset show the superiority of our DFA-UNet over 9 state-of-the-art image segmentation models. Additionally, visual analysis, ablation studies, and generalization assessments highlight the significant enhancement effects of DFA-UNet. Comprehensive experiments confirm the excellent segmentation effectiveness of the DFA-UNet combined attention mechanism for ultrasound images, underscoring its important significance for future research on medical images.

UCSwin‐UNet model for medical image segmentation based on cardiac haemangioma

AbstractCardiac hemangioma is a rare benign tumour that presents diagnostic challenges due to its variable clinical symptoms, imaging features, and locations. This study proposes a novel segmentation method based on a Convolutional Neural Network (CNN) and Transformer integration, with Swin‐UNet as the core model. We incorporated a U‐shaped convolutional neural network block into the original jump connection of Swin‐UNet. The Binary Cross Entropy Loss (BCE Loss) algorithm was added, and the learning rate decay algorithm was modified to select the appropriate one by comparing loss values. This paper utilizes the publicly available cardiac angioma dataset in AI Studio, consisting of 215 images for training and testing. To evaluate the effectiveness of the proposed model, this paper demonstrates its optimality through ablation experiments and comparisons with other mainstream models. The comparison experiments show that this model improves Dice by approximately 12%, HD95 by approximately 4.7 mm, Accuracy by approximately 6.1%, and F1 score by 0.11 compared to models such as UNet, UNet++, and Deeplabv3+, etc. For the recently proposed SOTO models, such as TransUNet, Swin‐UNet, and MultiResUnet, the Dice score improved by about 1.2%, HD95 reduced by about 1mm, Accuracy improved by about 0.3%, and F1 score improved by 0.015.

CapNet: An Automatic Attention-Based with Mixer Model for Cardiovascular Magnetic Resonance Image Segmentation.

Deep neural networks have shown excellent performance in medical image segmentation, especially for cardiac images. Transformer-based models, though having advantages over convolutional neural networks due to the ability of long-range dependence learning, still have shortcomings such as having a large number of parameters and and high computational cost. Additionally, for better results, they are often pretrained on a larger data, thus requiring large memory size and increasing resource expenses. In this study, we propose a new lightweight but efficient model, namely CapNet, based on convolutions and mixing modules for cardiac segmentation from magnetic resonance images (MRI) that can be trained from scratch with a small amount of parameters. To handle varying sizes and shapes which often occur in cardiac systolic and diastolic phases, we propose attention modules for pooling, spatial, and channel information. We also propose a novel loss called the Tversky Shape Power Distance function based on the shape dissimilarity between labels and predictions that shows promising performances compared to other losses. Experiments on three public datasets including ACDC benchmark, Sunnybrook data, and MS-CMR challenge are conducted and compared with other state of the arts (SOTA). For binary segmentation, the proposed CapNet obtained the Dice similarity coefficient (DSC) of 94% and 95.93% for respectively the Endocardium and Epicardium regions with Sunnybrook dataset, 94.49% for Endocardium, and 96.82% for Epicardium with the ACDC data. Regarding the multiclass case, the average DSC by CapNet is 93.05% for the ACDC data; and the DSC scores for the MS-CMR are 94.59%, 92.22%, and 93.99% for respectively the bSSFP, T2-SPAIR, and LGE sequences of the MS-CMR. Moreover, the statistical significance analysis tests with p-value compared with transformer-based methods and some CNN-based approaches demonstrated that the CapNet, though having fewer training parameters, is statistically significant. The promising evaluation metrics show comparative results in both Dice and IoU indices compared to SOTA CNN-based and Transformer-based architectures.

Orthopedic Image Segmentation and Recognition Based on Deep Neural Networks

In the field of medicine, especially orthopedics, accurate diagnosis is crucial. However, due to the uneven distribution of medical resources and the continuous growth of patient numbers, the diagnostic efficiency of hospitals has been significantly affected, and the problems of misdiagnosis and missed diagnosis are becoming increasingly prominent. To address these issues, this study explores a deep neural network-based orthopedic image segmentation and recognition model, aiming to improve the accuracy and efficiency of diagnosis. Firstly, we use U-Net neural network for segmentation training of orthopedic images in order to extract detailed features from the images.

Optimising robotic operation speed with edge computing via 5G network: Insights from selective harvesting robots

AbstractSelective harvesting by autonomous robots will be a critical enabling technology for future farming. Increases in inflation and shortages of skilled labor are driving factors that can help encourage user acceptability of robotic harvesting. For example, robotic strawberry harvesting requires real‐time high‐precision fruit localization, three‐dimensional (3D) mapping, and path planning for 3D cluster manipulation. Whilst industry and academia have developed multiple strawberry harvesting robots, none have yet achieved human–cost parity. Achieving this goal requires increased picking speed (perception, control, and movement), accuracy, and the development of low‐cost robotic system designs. We propose the edge‐server over 5G for Selective Harvesting (E5SH) system, which is an integration of high bandwidth and low latency Fifth‐Generation (5G) mobile network into a crop harvesting robotic platform, which we view as an enabler for future robotic harvesting systems. We also consider processing scale and speed in conjunction with system environmental and energy costs. A system architecture is presented and evaluated with support from quantitative results from a series of experiments that compare the performance of the system in response to different architecture choices, including image segmentation models, network infrastructure (5G vs. Wireless Fidelity), and messaging protocols, such as Message Queuing Telemetry Transport and Transport Control Protocol Robot Operating System. Our results demonstrate that the E5SH system delivers step‐change peak processing performance speedup of above 18‐fold than a standalone embedded computing Nvidia Jetson Xavier NX system.

Research on Segmentation Method of Maize Seedling Plant Instances Based on UAV Multispectral Remote Sensing Images.

The accurate instance segmentation of individual crop plants is crucial for achieving a high-throughput phenotypic analysis of seedlings and smart field management in agriculture. Current crop monitoring techniques employing remote sensing predominantly focus on population analysis, thereby lacking precise estimations for individual plants. This study concentrates on maize, a critical staple crop, and leverages multispectral remote sensing data sourced from unmanned aerial vehicles (UAVs). A large-scale SAM image segmentation model is employed to efficiently annotate maize plant instances, thereby constructing a dataset for maize seedling instance segmentation. The study evaluates the experimental accuracy of six instance segmentation algorithms: Mask R-CNN, Cascade Mask R-CNN, PointRend, YOLOv5, Mask Scoring R-CNN, and YOLOv8, employing various combinations of multispectral bands for a comparative analysis. The experimental findings indicate that the YOLOv8 model exhibits exceptional segmentation accuracy, notably in the NRG band, with bbox_mAP50 and segm_mAP50 accuracies reaching 95.2% and 94%, respectively, surpassing other models. Furthermore, YOLOv8 demonstrates robust performance in generalization experiments, indicating its adaptability across diverse environments and conditions. Additionally, this study simulates and analyzes the impact of different resolutions on the model's segmentation accuracy. The findings reveal that the YOLOv8 model sustains high segmentation accuracy even at reduced resolutions (1.333 cm/px), meeting the phenotypic analysis and field management criteria.

SymTC: A symbiotic Transformer-CNN net for instance segmentation of lumbar spine MRI

Intervertebral disc disease, a prevalent ailment, frequently leads to intermittent or persistent low back pain, and diagnosing and assessing of this disease rely on accurate measurement of vertebral bone and intervertebral disc geometries from lumbar MR images. Deep neural network (DNN) models may assist clinicians with more efficient image segmentation of individual instances (discs and vertebrae) of the lumbar spine in an automated way, which is termed as instance image segmentation. In this work, we proposed SymTC, an innovative lumbar spine MR image segmentation model that combines the strengths of Transformer and Convolutional Neural Network (CNN). Specifically, we designed a parallel dual-path architecture to merge CNN layers and Transformer layers, and we integrated a novel position embedding into the self-attention module of Transformer, enhancing the utilization of positional information for more accurate segmentation. To further improve model performance, we introduced a new data synthesis technique to create synthetic yet realistic MR image dataset, named SSMSpine, which is made publicly available. We evaluated our SymTC and the other 16 representative image segmentation models on our private in-house dataset and public SSMSpine dataset, using two metrics, Dice Similarity Coefficient and the 95th percentile Hausdorff Distance. The results indicate that SymTC surpasses the other 16 methods, achieving the highest dice score of 96.169 % for segmenting vertebral bones and intervertebral discs on the SSMSpine dataset. The SymTC code and SSMSpine dataset are publicly available at https://github.com/jiasongchen/SymTC.

An advanced heat source localization technology for intelligent warehousing: A multi-source fusion image segmentation approach leveraging infrared and visible light data

Infrared thermography technology, leveraging its unique ability to capture temperature features, has significantly improved the precision of high-temperature target localization. However, infrared imaging technology is limited by issues such as low image contrast, difficulty in distinguishing object categories, and limited image clarity. To enable intelligent detection of high-temperature objects that may cause fires in warehouses, this paper proposes an innovative method that integrates deep learning image segmentation with infrared and visible light image technology. We developed a new image segmentation model based on improved Fully Convolutional Networks and Deconvolutional Networks, introducing a batch normalization layer to accelerate convergence and employing the PReLU activation function to prevent neuron death, thereby enhancing convergence speed and accuracy. Through a feature dynamic image registration method combining a joint model and a cross-modulation strategy, we achieved efficient image fusion. In addition, a game theory-based strategy was adopted to correct localization results, ensuring accuracy. Experimental results demonstrate that the improved model achieves localization accuracy and precision rates of up to 89.30% and 88.00%, respectively, in real-world warehouse heat source scenarios, representing a significant improvement of 9.90% and 2.85% compared to the pre-improvement model, fully validating its advancement and effectiveness.