Segmentation Model Research Articles

Automatic geometric digital twin of box girder bridge using a laser-scanned point cloud

Geometric modeling is a pivotal step in creating a digital twin for existing bridge structures. Its deficiency of automation makes geometric modeling step time-consuming and laborious. This paper presents a solution for automatically modeling box girder bridges, including external and internal structures, based on laser-scanned point cloud. The solution includes three vital methods: component segmentation, key points extraction of cross-section, and internal structure reconstruction. The results indicate that the established segmentation model, BCR-Net, exhibited better performance than PointNet++ in component segmentation, as demonstrated by mIoU of 0.9751 on the test set. The mean absolute error on the dimension of the pier, bent cap and external and internal structure of the box girder is 0.27 %, 0.38 %, 0.47 %, and 0.48 %, respectively. It means the proposed methods possessed excellent modeling accuracy while ensuring high efficiency than manual modeling, providing a promising solution for digital twin modeling of bridge structures.

Children Are Not Small Adults: Addressing Limited Generalizability of an Adult Deep Learning CT Organ Segmentation Model to the Pediatric Population

Children Are Not Small Adults: Addressing Limited Generalizability of an Adult Deep Learning CT Organ Segmentation Model to the Pediatric Population

Artificial intelligence-based analysis of behavior and brain images in cocaine-self-administered marmosets

BackgroundThe sophisticated behavioral and cognitive repertoires of non-human primates (NHPs) make them suitable subjects for studies involving cocaine self-administration (SA) schedules. However, ethical considerations, adherence to the 3Rs principle (replacement, reduction and refinement), and other factors make it challenging to obtain NHPs individuals for research. Consequently, there is a need for methods that can comprehensively analyze small datasets using artificial intelligence (AI). New methodsWe employed AI to identify cocaine dependence patterns from collected data. First, we collected behavioral data from cocaine SA marmosets (Callithrix jacchus) to develop a dependence prediction model. SHapley Additive exPlanations (SHAP) values were used to demonstrate the importance of various variables. Additionally, we collected positron emission tomographic (PET) images showing dopamine transporter (DAT) binding potential and developed an algorithm for PET image segmentation. ResultsThe prediction model indicated that the Random Forest (RF) algorithm performed best, with an area under the curve (AUC) of 0.92. The top five variables influencing the model were identified using SHAP values. The PET image segmentation model achieved an accuracy of 0.97, a mean squared error of 0.02, an intersection over union (IoU) of 0.845, and a Dice coefficient of 0.913. Comparison with existing methods and conclusionUtilizing data from the marmoset SA experiment, we developed an ML-based dependence prediction model and analyzed variable importance rankings using SHAP. AI-based imaging segmentation methods offer a valuable tool for evaluating DAT availability in NHPs following chronic cocaine administration.

A Deep Learning-Based Approach for Identifying Defects in Solar Panels

In the solar energy sector, the task of monitoring and maintaining large photovoltaic (PV) system portfolios is essential for ensuring optimal performance and reliability. Prominent solar energy companies face challenges with their current fault detection methods, which are inefficient and resource- intensive. This paper addresses the critical need for improved fault detection in solar PV systems to maximize uptime and minimize maintenance costs. We employed advanced data preprocessing and augmentation techniques using Roboflow and developed a YOLOv8 segmentation model in Google Colab with GPU. This model was then deployed using Streamlit, providing a robust solution for identifying faulty solar modules. The proposed approach significantly enhances fault detection accuracy, achieving a minimum accuracy rate of 85%, thus ensuring reliable operation of the PV systems. Additionally, the implementation of this model contributes to a 15% reduction in system downtime and a 10% reduction in maintenance costs. By leveraging advanced machine learning techniques, our solution transforms the maintenance process, making it more efficient and cost-effective. Consequently, this work not only improves the reliability and performance of solar PV systems but also supports the broader goal of sustainable energy through more efficient resource usage.

Multimodal Data-Driven Segmentation of Bone Metastasis Lesions in SPECT Bone Scans Using Deep Learning

Background: Patients with malignant tumors often develop bone metastases. SPECT bone scintigraphy is an effective tool for surveying bone metastases due to its high sensitivity, low-cost equipment, and radiopharmaceutical. However, the low spatial resolution of SPECT scans significantly hinders manual analysis by nuclear medicine physicians. Deep learning, a promising technique for automated image analysis, can extract hierarchal patterns from images without human intervention. Objective: To enhance the performance of deep learning-based segmentation models, we integrate textual data from diagnostic reports with SPECT bone scans, aiming to develop an automated analysis method that outperforms purely unimodal data-driven segmentation models. Methods: We propose a dual-path segmentation framework to extract features from bone scan images and diagnostic reports separately. In the first path, an encoder-decoder network is employed to learn hierarchical representations of features from SPECT bone scan images. In the second path, the Chinese version of the MacBERT model is utilized to develop a text encoder for extracting features from diagnostic reports. The extracted textual features are then fused with image features during the decoding stage in the first path, enhancing the overall segmentation performance. Results: Experimental evaluation conducted on real-world clinical data demonstrated the superior performance of the proposed segmentation model. Our model achieved a 0.0209 increase in the DSC (Dice Similarity Coefficient) score compared to the well-known U-Net model. Conclusions: The proposed multimodal data-driven method effectively identifies and isolates metastasis lesions in SPECT bone scans, outperforming existing classical deep learning models. This study demonstrates the value of incorporating textual data in the deep learning-based segmentation of lowresolution SPECT bone scans.

Sequence Segmentation of Nematodes in Atlantic Cod with Multispectral Imaging Data.

Nematodes pose significant challenges for the fish processing industry, particularly in white fish. Despite technological advances, the industry still depends on manual labor for the detection and extraction of nematodes. This study addresses the initial steps of automatic nematode detection and differentiation from other common defects in fish fillets, such as skin remnants and blood spots. VideometerLab 4, an advanced Multispectral Imaging (MSI) System, was used to acquire 270 images of 50 Atlantic cod fillets under controlled conditions. In total, 173 nematodes were labeled using the Segment Anything Model (SAM), which is trained to automatically segment objects of interest from only few representative pixels. With the acquired dataset, we study the potential of identifying nematodes through their spectral signature. We incorporated normalized Canonical Discriminant Analysis (nCDA) to develop segmentation models trained to distinguish between different components within the fish fillets. By incorporating multiple segmentation models, we aimed to achieve a satisfactory balance between false negatives and false positives. This resulted in 88% precision and 79% recall for our annotated test data. This approach could improve process control by accurately identifying fillets with nematodes. Using MSI minimizes unnecessary inspection of fillets in good condition and concurrently boosts product safety and quality.

Analysis and comparison of retinal vascular parameters under different glucose metabolic status based on deep learning.

To develop a deep learning-based model for automatic retinal vascular segmentation, analyzing and comparing parameters under diverse glucose metabolic status (normal, prediabetes, diabetes) and to assess the potential of artificial intelligence (AI) in image segmentation and retinal vascular parameters for predicting prediabetes and diabetes. Retinal fundus photos from 200 normal individuals, 200 prediabetic patients, and 200 diabetic patients (600 eyes in total) were used. The U-Net network served as the foundational architecture for retinal artery-vein segmentation. An automatic segmentation and evaluation system for retinal vascular parameters was trained, encompassing 26 parameters. Significant differences were found in retinal vascular parameters across normal, prediabetes, and diabetes groups, including artery diameter (P=0.008), fractal dimension (P=0.000), vein curvature (P=0.003), C-zone artery branching vessel count (P=0.049), C-zone vein branching vessel count (P=0.041), artery branching angle (P=0.005), vein branching angle (P=0.001), artery angle asymmetry degree (P=0.003), vessel length density (P=0.000), and vessel area density (P=0.000), totaling 10 parameters. The deep learning-based model facilitates retinal vascular parameter identification and quantification, revealing significant differences. These parameters exhibit potential as biomarkers for prediabetes and diabetes.

Automated quantification of crack length and width in asphalt pavements

AbstractRapid, accurate, and fully automated estimation of both length and width of asphalt pavement cracks, essential for achieving a proactive asset management, presents a significant challenge, primarily due to limitations in the effectiveness of automatic image segmentation and the accuracy of crack width and length estimation algorithms. To address this challenge, this paper introduces the Branch Growing (BG) algorithm, specifically designed for crack length estimation in asphalt pavements, along with an optimized OrthoBoundary algorithm tailored for crack width estimation. Leveraging four widely adopted deep learning models for asphalt pavement crack segmentation, four distinct sets of image segmentation results have been produced. Subsequently, a comprehensive evaluation has been conducted to assess the effectiveness of both crack dimensions estimation algorithms. The findings demonstrate that the integration of the BG algorithm, the optimized OrthoBoundary algorithm, and the fully convolutional network with the HRNet backbone achieve a prediction accuracy of 80.21% for crack length estimation and 84.32% for average width estimation. Moreover, the image processing speed, at a resolution of 3024 × 3024, can be maintained at approximately 5 s, with average width estimation observed to be up to 9.1‐fold faster than the unoptimized OrthoBoundary algorithm. These results signify advancements in automated crack quantification methodologies, with implications for enhancing civil infrastructure maintenance practices.

Camouflaged cotton bollworm instance segmentation based on PVT and Mask R-CNN

Many pests change their appearance color to seamlessly blend with the surrounding environment in agricultural ecosystems, thereby rendering themselves virtually invisible. When the pest’s color and texture resemble the background, accurately identifying and detecting it becomes challenging. In this study, we construct a new dataset focusing on the cotton bollworm and conduct in-depth optimization and improvement of the instance segmentation model based on the Pyramid Vision Transformer (PVT) and Mask R-CNN. To better capture the features of camouflaged organisms, the proposed model utilizes the PVT as a feature extraction network and Mask-RCNN for instance segmentation. We also introduce overlapping image embedding patch structure and further incorporate a feed-forward network with depthwise separable convolution. These improvements enhance the PVT’s capability to capture global and intricate features and significantly boost the accuracy of instance segmentation. Considering the computational efficiency demands in real-time agricultural applications, we introduce a linear spatial-reduction attention mechanism that effectively reduces computational complexity. The experimental results show that the model achieves the detection accuracy of 89.7% and the segmentation accuracy of 89.2% for camouflaged cotton bollworms.

Driveway Detection for Weed Management in Cassava Plantation Fields in Thailand Using Ground Imagery Datasets and Deep Learning Models

Weeds reduce cassava root yields and infest furrow areas quickly. The use of mechanical weeders has been introduced in Thailand; however, manually aligning the weeders with each planting row and at headland turns is still challenging. It is critical to clear weeds on furrow slopes and driveways via mechanical weeders. Automation can support this difficult work for weed management via driveway detection. In this context, deep learning algorithms have the potential to train models to detect driveways through furrow image segmentation. Therefore, the purpose of this research was to develop an image segmentation model for automated weed control operations in cassava plantation fields. To achieve this, image datasets were obtained from various fields to aid weed detection models in automated weed management. Three models—Mask R-CNN, YOLACT, and YOLOv8n-seg—were used to construct the image segmentation model, and they were evaluated according to their precision, recall, and FPS. The results show that YOLOv8n-seg achieved the highest accuracy and FPS (114.94 FPS); however, it experienced issues with frame segmentation during video testing. In contrast, YOLACT had no segmentation issues in the video tests (23.45 FPS), indicating its potential for driveway segmentation in cassava plantations. In summary, image segmentation for detecting driveways can improve weed management in cassava fields, and the further automation of low-cost mechanical weeders in tropical climates can be performed based on the YOLACT algorithm.

Automated segmentation of midbrain nuclei using deep learning and multisequence MRI: a longitudinal study on iron accumulation with age

Abstract Elevated levels of brain iron, particularly within the basal ganglia, have been associated with cognitive and motor impairment in normal aging and neurodegenerative conditions. The subthalamic nucleus (STN), substantia nigra (SN), and red nucleus (RN), despite their high iron content and contribution to motor and cognitive processes, are less frequently studied. This oversight can largely be attributed to the challenges posed by in-vivo assessments of these small, deep-seated midbrain structures. We developed and validated an automated tool for the segmentation of the STN, SN, and RN. Multi-sequence magnetic resonance imaging (MRI) data including T1-weighted, FLAIR, Quantitative Susceptibility Mapping (QSM) and R2* alongside manual delineation on QSM images of 40 individuals, was used to train segmentation models based on nnU-Net deep-learning framework. A combination of QSM and FLAIR sequences was found to be optimal for structure segmentation (mean Dice scores of 0.84, 0.91 and 0.94 for STN, SN and RN, respectively). We next applied the automated segmentation method to an independent 3-year longitudinal dataset including 175 healthy adults (age range at baseline: 20-79 years old). Structural equation modelling was used to assess iron accumulation over time using age, sex, baseline iron and regional volume as factors of interest. Cross-sectionally, older age was linearly associated with higher iron load in SN and STN; the association was non-linear in RN. Longitudinally, results indicated significant iron accumulation in the STN (Mean increase = 0.02, p = 0.005) and SN (Mean increase = 0.035, p = 0.001), but not in the RN (Mean increase = 0.015, p = 0.2). Our findings demonstrated high performance of nnU-Net in automated segmentation, and advanced our understanding of iron accumulation in midbrain nuclei in aging.

A New Instance Segmentation Model for High-Resolution Remote Sensing Images Based on Edge Processing

With the goal of addressing the challenges of small, densely packed targets in remote sensing images, we propose a high-resolution instance segmentation model named QuadTransPointRend Net (QTPR-Net). This model significantly enhances instance segmentation performance in remote sensing images. The model consists of two main modules: preliminary edge feature extraction (PEFE) and edge point feature refinement (EPFR). We also created a specific approach and strategy named TransQTA for edge uncertainty point selection and feature processing in high-resolution remote sensing images. Multi-scale feature fusion and transformer technologies are used in QTPR-Net to refine rough masks and fine-grained features for selected edge uncertainty points while balancing model size and accuracy. Based on experiments performed on three public datasets: NWPU VHR-10, SSDD, and iSAID, we demonstrate the superiority of QTPR-Net over existing approaches.

MT-U2Net: Lightweight detection network for high-precision magnetic tile surface defect localization

Automated surface defect localization is closely related to product quality control. Although many image segmentation models have achieved excellent results in industrial defect localization, the wide-ranging defects in magnetic tile materials and the similarity between textures and background textures make it difficult to balance detection model performance and model size. Therefore, we propose a new magnetic tile defect localization model called MT-U2Net. The model consists of three parts: Dres-Ublock, U2 structure, and attention mechanism. Dres-Ublock is used to enhance the model's feature extraction capability and reduce its size; the U2-Net structure is utilized to learn cross-scale defect information; the attention mechanism is employed to eliminate redundant information. MT-U2Net improves detection performance across various defect scales. In the test set, MT-U2Net achieves an F1 Score of 0.840 and an MAE of 0.024 and compresses the model size to 3.39 M. Extensive experimental results show that MT-U2Net outperforms existing state-of-the-art algorithms in magnetic tile defect detection performance and model size.

Latent Diffusion Models to Enhance the Performance of Visual Defect Segmentation Networks in Steel Surface Inspection.

This paper explores the use of state-of-the-art latent diffusion models, specifically stable diffusion, to generate synthetic images for improving the robustness of visual defect segmentation in manufacturing components. Given the scarcity and imbalance of real-world defect data, synthetic data generation offers a promising solution for training deep learning models. We fine-tuned stable diffusion using the LoRA technique on the NEU-seg dataset and evaluated the impact of different ratios of synthetic to real images on the training set of DeepLabV3+ and FPN segmentation models. Our results demonstrated a significant improvement in mean Intersection over Union (mIoU) when the training dataset was augmented with synthetic images. This study highlights the potential of diffusion models for enhancing the quality and diversity of training data in industrial defect detection, leading to more accurate and reliable segmentation results. The proposed approach achieved improvements of 5.95% and 6.85% in mIoU of defect segmentation on each model over the original dataset.

Accurate identification of salt domes using deep learning techniques: Transformers, generative artificial intelligence and liquid state machines

AbstractAcross various global regions abundant in oil and natural gas reserves, the presence of substantial sub‐surface salt deposits holds significant relevance. Accurate identification of salt domes becomes crucial for enterprises engaged in oil and gas exploration. Our research introduces a precise method for the automatic detection of salt domes, leveraging advanced deep learning architectures such as U‐net, transformers, artificial intelligence generative models and liquid state machines. In comparison with state‐of‐the‐art techniques, our model demonstrates superior performance, achieving a stable and validated intersection over the union metric, indicating high accuracy and robustness. Furthermore, the Dice similarity coefficient attaining underscores the model's proficiency in closely aligning with ground truth across diverse scenarios. This evaluation, conducted on 1000 seismic images, reveals that our proposed architecture is not only comparable but often surpasses existing segmentation models in effectiveness and reliability.

Artificial Intelligence Tools in Pediatric Urology: A Comprehensive Review of Recent Advances.

Artificial intelligence (AI) is providing novel answers to long-standing clinical problems, and it is quickly changing pediatric urology. This thorough analysis focuses on current developments in AI technologies that improve pediatric urology diagnosis, treatment planning, and surgery results. Deep learning algorithms help detect problems with previously unheard-of precision in disorders including hydronephrosis, pyeloplasty, and vesicoureteral reflux, where AI-powered prediction models have demonstrated promising outcomes in boosting diagnostic accuracy. AI-enhanced image processing methods have significantly improved the quality and interpretation of medical images. Examples of these methods are deep-learning-based segmentation and contrast limited adaptive histogram equalization (CLAHE). These methods guarantee higher precision in the identification and classification of pediatric urological disorders, and AI-driven ground truth construction approaches aid in the standardization of and improvement in training data, resulting in more resilient and consistent segmentation models. AI is being used for surgical support as well. AI-assisted navigation devices help with difficult operations like pyeloplasty by decreasing complications and increasing surgical accuracy. AI also helps with long-term patient monitoring, predictive analytics, and customized treatment strategies, all of which improve results for younger patients. However, there are practical, ethical, and legal issues with AI integration in pediatric urology that need to be carefully navigated. To close knowledge gaps, more investigation is required, especially in the areas of AI-driven surgical methods and standardized ground truth datasets for pediatric radiologic image segmentation. In the end, AI has the potential to completely transform pediatric urology by enhancing patient care, increasing the effectiveness of treatments, and spurring more advancements in this exciting area.

Mask R-CNN and Centroid Tracking Algorithm to Process UAV Based Thermal-RGB Video for Drylot Cattle Heat Stress Monitoring

This study developed and evaluated an algorithm for processing thermal-RGB video feeds captured by an unmanned aerial vehicle (UAV) to automate heat stress monitoring in cattle housed in the drylots. The body surface temperature (BST) of individual cows was used as an indicator of heat stress. UAV data were collected using RGB and thermal infrared imagers, respectively, at 2 and 6.67 cm per pixel spatial resolution in Spring 2023 (dataset-1) and Summer 2024 (dataset-2). Study sites were two commercial drylots in Washington State. The custom algorithms were developed to: (1) detect and localize individual cows using a Mask R-CNN-based instance segmentation model combined with centroid tracking; and (2) extract BST by averaging the thermal-imagery pixels for each of the segmented cows. The algorithm showed higher detection accuracy with RGB images as input (F1 score: 0.89) compared to thermal (F1 score: 0.64). BST extraction with combined RGB and thermal imaging approach required corrections for alignment problems associated with differences in optics, imaging field of view, resolution, and lens properties. Consequently, thermal imaging-only approach was adopted for assessing real-time cow localization and BST estimation. Operating at one frame per second, algorithm successfully detected 72.4% and 81.65% of total cows in video frames from dataset-1 (38 s) and -2 (48 s), respectively. The mean absolute difference between algorithm output and ground truth (BSTGT) was 2.1 °C (dataset-1) and 3.3 °C (dataset-2), demonstrating satisfactory performance. With further refinements, this approach could be a viable tool for real-time heat stress monitoring in large-scale drylot production systems.

Vital Characteristics Cellular Neural Network (VCeNN) for Melanoma Lesion Segmentation: A Biologically Inspired Deep Learning Approach.

Cutaneous melanoma is a highly lethal form of cancer. Developing a medical image segmentation model capable of accurately delineating melanoma lesions with high robustness and generalization presents a formidable challenge. This study draws inspiration from cellular functional characteristics and natural selection, proposing a novel medical segmentation model named the vital characteristics cellular neural network. This model incorporates vital characteristics observed in multicellular organisms, including memory, adaptation, apoptosis, and division. Memory module enables the network to rapidly adapt to input data during the early stages of training, accelerating model convergence. Adaptation module allows neurons to select the appropriate activation function based on varying environmental conditions. Apoptosis module reduces the risk of overfitting by pruning neurons with low activation values. Division module enhances the network's learning capacity by duplicating neurons with high activation values. Experimental evaluations demonstrate the efficacy of this model in enhancing the performance of neural networks for medical image segmentation. The proposed method achieves outstanding results across numerous publicly available datasets, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery. The proposed method achieves outstanding results across numerous publicly available datasets, with an F1 score of 0.901, Intersection over Union of 0.841, and Dice coefficient of 0.913, indicating its potential to contribute significantly to the field of medical image analysis and facilitating accurate and efficient segmentation of medical imagery.

Automated Detection of Hillforts in Remote Sensing Imagery With Deep Multimodal Segmentation

ABSTRACTRecent advancements in remote sensing and artificial intelligence can potentially revolutionize the automated detection of archaeological sites. However, the challenging task of interpreting remote sensing imagery combined with the intricate shapes of archaeological sites can hinder the performance of computer vision systems. This work presents a computer vision system trained for efficient hillfort detection in remote sensing imagery. Equipped with an adapted multimodal semantic segmentation model, the system integrates LiDAR‐derived LRM images and aerial orthoimages for feature fusion, generating a binary mask pinpointing detected hillforts. Post‐processing includes margin and area filters to remove edge inferences and smaller anomalies. The resulting inferences are subjected to hard positive and negative mining, where expert archaeologists classify them to populate the training data with new samples for retraining the segmentation model. As the computer vision system is far more likely to encounter background images during its search, the training data are intentionally biased towards negative examples. This approach aims to reduce the number of false positives, typically seen when applying machine learning solutions to remote sensing imagery. Northwest Iberia experiments witnessed a drastic reduction in false positives, from 5678 to 40 after a single hard positive and negative mining iteration, yielding a 99.3% reduction, with a resulting F1 score of 66%. In England experiments, the system achieved a 59% F1 score when fine‐tuned and deployed countrywide. Its scalability to diverse archaeological sites is demonstrated by successfully detecting hillforts and other types of enclosures despite their typical complex and varied shapes. Future work will explore archaeological predictive modelling to identify regions with higher archaeological potential to focus the search, addressing processing time challenges.

External validation of a deep learning model for automatic segmentation of skeletal muscle and adipose tissue on abdominal computed tomography images.

Body composition assessment using computed tomography (CT) images at the L3-level is increasingly applied in cancer research. Robust high-throughput automated segmentation is key to assess large patient cohorts and to support implementation of body composition analysis into routine clinical practice. We trained and externally validated a deep learning neural network (DLNN) to automatically segment L3-CT images. Expert-drawn segmentations of visceral and subcutaneous adipose tissue (VAT/SAT) and skeletal muscle (SM) of L3-CT-images of 3,187 patients undergoing abdominal surgery were used to train a DLNN. The external validation cohort was comprised of 2,535 patients with abdominal cancer. DLNN performance was evaluated with (geometric) Dice Similarity (DS) and Lin's Concordance Correlation Coefficient. There was a strong concordance between automatic and manual segmentations with median DS for SM, VAT, and SAT of 0.97 (interquartile range, IQR: 0.95-0.98), 0.98 (IQR: 0.95-0.98), and 0.95 (IQR: 0.92-0.97), respectively. Concordance correlations were excellent: SM 0.964 (0.959-0.968), VAT 0.998 (0.998-0.998), and SAT 0.992 (0.991-0.993). Bland-Altman metrics indicated only small and clinically insignificant systematic offsets; SM radiodensity: 0.23 hounsfield units (0.5%), SM: 1.26 cm2.m-2 (2.8%), VAT: -1.02 cm2.m-2 (1.7%), and SAT: 3.24 cm2.m-2 (4.6%). A robustly-performing and independently externally validated DLNN for automated body composition analysis was developed. CT-based body composition analysis is highly prognostic for long-term overall survival in oncology. This DLNN was succesfully trained and externally validated on several large patient cohorts and will therefore enable large scale population studies and implementation of body composition analysis into clinical practice.