Automatic Segmentation Research Articles

Deep learning technology in vascular image segmentation and disease diagnosis

AbstractBlood vessel segmentation is a crucial aspect of medical image processing, aiding medical professionals in more accurate disease analysis and diagnosis. Manual blood vessel segmentation methods are time‐consuming and cumbersome, making the development of automatic segmentation methods essential. The rapid advancements in deep learning technology have introduced new tools and methods for vascular image segmentation. In this review, we provide a comprehensive overview of deep learning‐based blood vessel segmentation methods across various fields, including retinal vessel segmentation, cerebrovascular segmentation, and pulmonary vessel segmentation. Several prevalent diseases, such as retinal vascular diseases, cerebrovascular diseases, pulmonary vascular diseases, and tumors, have posed significant health challenges globally. This review also discusses the application of deep learning technology in disease diagnosis within these contexts. Finally, considering the current research landscape, we discuss existing challenges and potential future developments in blood vessel segmentation. We aim to assist researchers in gaining a comprehensive understanding and designing effective blood vessel segmentation models, ultimately offering opportunities for early disease diagnosis and treatment.

CardiacField: computational echocardiography for automated heart function estimation using two-dimensional echocardiography probes

Abstract Aims Accurate heart function estimation is vital for detecting and monitoring cardiovascular diseases. While two-dimensional echocardiography (2DE) is widely accessible and used, it requires specialized training, is prone to inter-observer variability, and lacks comprehensive three-dimensional (3D) information. We introduce CardiacField, a computational echocardiography system using a 2DE probe for precise, automated left ventricular (LV) and right ventricular (RV) ejection fraction (EF) estimations, which is especially easy to use for non-cardiovascular healthcare practitioners. We assess the system’s usability among novice users and evaluate its performance against expert interpretations and advanced deep learning (DL) tools. Methods and results We developed an implicit neural representation network to reconstruct a 3D cardiac volume from sequential multi-view 2DE images, followed by automatic segmentation of LV and RV areas to calculate volume sizes and EF values. Our study involved 127 patients to assess EF estimation accuracy against expert readings and two-dimensional (2D) video-based DL models. A subset of 56 patients was utilized to evaluate image quality and 3D accuracy and another 50 to test usability by novice users and across various ultrasound machines. CardiacField generated a 3D heart from 2D echocardiograms with <2 min processing time. The LVEF predicted by our method had a mean absolute error (MAE) of 2.48%, while the RVEF had an MAE of 2.65%. Conclusion Employing a straightforward apical ring scan with a cost-effective 2DE probe, our method achieves a level of EF accuracy for assessing LV and RV function that is comparable to that of three-dimensional echocardiography probes.

The Diachronic Annotated Corpus of Newar

Abstract This article presents the first Diachronic Annotated Corpus of Newar (DACON), an ongoing project to annotate Newar texts ranging from the 12th to the 19th centuries. The primary purpose of this corpus is to facilitate both synchronic and diachronic as well as comparative linguistic research. We also present our annotation pipeline with detailed information on our workflow and the results of the automatic segmentation and POS-tagging tools.

PSFHS challenge report: Pubic symphysis and fetal head segmentation from intrapartum ultrasound images

Segmentation of the fetal and maternal structures, particularly intrapartum ultrasound imaging as advocated by the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) for monitoring labor progression, is a crucial first step for quantitative diagnosis and clinical decision-making. This requires specialized analysis by obstetrics professionals, in a task that i) is highly time- and cost-consuming and ii) often yields inconsistent results. The utility of automatic segmentation algorithms for biometry has been proven, though existing results remain suboptimal. To push forward advancements in this area, the Grand Challenge on Pubic Symphysis-Fetal Head Segmentation (PSFHS) was held alongside the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). This challenge aimed to enhance the development of automatic segmentation algorithms at an international scale, providing the largest dataset to date with 5,101 intrapartum ultrasound images collected from two ultrasound machines across three hospitals from two institutions. The scientific community's enthusiastic participation led to the selection of the top 8 out of 179 entries from 193 registrants in the initial phase to proceed to the competition's second stage. These algorithms have elevated the state-of-the-art in automatic PSFHS from intrapartum ultrasound images. A thorough analysis of the results pinpointed ongoing challenges in the field and outlined recommendations for future work. The top solutions and the complete dataset remain publicly available, fostering further advancements in automatic segmentation and biometry for intrapartum ultrasound imaging.

A Multi-Task Model for Pulmonary Nodule Segmentation and Classification.

In the computer-aided diagnosis of lung cancer, the automatic segmentation of pulmonary nodules and the classification of benign and malignant tumors are two fundamental tasks. However, deep learning models often overlook the potential benefits of task correlations in improving their respective performances, as they are typically designed for a single task only. Therefore, we propose a multi-task network (MT-Net) that integrates shared backbone architecture and a prediction distillation structure for the simultaneous segmentation and classification of pulmonary nodules. The model comprises a coarse segmentation subnetwork (Coarse Seg-net), a cooperative classification subnetwork (Class-net), and a cooperative segmentation subnetwork (Fine Seg-net). Coarse Seg-net and Fine Seg-net share identical structure, where Coarse Seg-net provides prior location information for the subsequent Fine Seg-net and Class-net, thereby boosting pulmonary nodule segmentation and classification performance. We quantitatively and qualitatively analyzed the performance of the model by using the public dataset LIDC-IDRI. Our results show that the model achieves a Dice similarity coefficient (DI) index of 83.2% for pulmonary nodule segmentation, as well as an accuracy (ACC) of 91.9% for benign and malignant pulmonary nodule classification, which is competitive with other state-of-the-art methods. The experimental results demonstrate that the performance of pulmonary nodule segmentation and classification can be improved by a unified model that leverages the potential correlation between tasks.

A multi-constraint optimal puncture path planning algorithm for percutaneous interventional radiofrequency thermal fusion of the L5/S1 segments

To minimize variations in treatment outcomes of L5/S1 percutaneous intervertebral radiofrequency thermocoagulation (PIRFT) arising from physician proficiency and achieve precise quantitative risk assessment of the puncture paths. We used a self-developed deep neural network DWT-UNet for automatic segmentation of the magnetic resonance (MR) images of the L5/S1 segments into 7 key structures: L5, S1, Ilium, Disc, N5, Dura mater, and Skin, based on which a needle insertion path planning environment was modeled. Six hard constraints and 6 soft constraints were proposed based on clinical criteria for needle insertion, and the physician's experience was quantified into weights using the analytic hierarchy process and incorporated into the risk function for needle insertion paths to enhance individual case adaptability. By leveraging the proposed skin entry point sampling sub-algorithm and Kambin's triangle projection area sub-algorithm in conjunction with the analytic hierarchy process, and employing various technologies such as ray tracing, CPU multi-threading, and GPU parallel computing, a puncture path was calculated that not only met clinical hard constraints but also optimized the overall soft constraints. A surgical team conducted a subjective evaluation of the 21 needle puncture paths planned by the algorithm, and all the paths met the clinical requirements, with 95.24% of them rated excellent or good. Compared with the physician's planning results, the plans generated by the algorithm showed inferior DIlium, DS1, and Depth (P < 0.05) but much better DDura, DL5, DN5, and AKambin (P < 0.05). In the 21 cases, the planning time of the algorithm averaged 7.97±3.73 s, much shorter than that by the physicians (typically beyond 10 min). The multi-constraint optimal puncture path planning algorithm offers an efficient automated solution for PIRFT of the L5/S1 segments with great potentials for clinical application.

3D whole body preclinical micro-CT database of subcutaneous tumors in mice with annotations from 3 annotators

A pivotal animal model for development of anticancer molecules is mice with subcutaneous tumors, grown by injection of xenografted tumor cells, where micro-Computed Tomography (µCT) of the mice is used to analyze the efficacy of the anticancer molecule. Manual delineation of the tumor region is necessary for the analysis, which is time-consuming and inconsistent, highlighting the need for automatic segmentation (AS) tools. This study introduces a preclinical µCT database, comprising 452 whole-body scans from 223 individual mice with subcutaneous tumors, spanning ten diverse µCT datasets conducted between 2014 and 2020 on a preclinical PET/CT scanner, making it the hitherto largest dataset of its kind. Each tumor is annotated manually by three expert annotators, allowing for robust model development. Inter-annotator agreement was analyzed, and we report an overall annotation agreement of 0.903 ± 0.046 (mean ± std) Fleiss’ Kappa and a mean deviation in volume estimation of 0.015 ± 0.010 cm3 (6.9% ± 4.7), which establishes a human baseline accuracy for delineation of subcutaneous tumors, while showing good inter-annotator agreement.

Analysis and comparison of retinal vascular parameters under different glucose metabolic status based on deep learning.

To develop a deep learning-based model for automatic retinal vascular segmentation, analyzing and comparing parameters under diverse glucose metabolic status (normal, prediabetes, diabetes) and to assess the potential of artificial intelligence (AI) in image segmentation and retinal vascular parameters for predicting prediabetes and diabetes. Retinal fundus photos from 200 normal individuals, 200 prediabetic patients, and 200 diabetic patients (600 eyes in total) were used. The U-Net network served as the foundational architecture for retinal artery-vein segmentation. An automatic segmentation and evaluation system for retinal vascular parameters was trained, encompassing 26 parameters. Significant differences were found in retinal vascular parameters across normal, prediabetes, and diabetes groups, including artery diameter (P=0.008), fractal dimension (P=0.000), vein curvature (P=0.003), C-zone artery branching vessel count (P=0.049), C-zone vein branching vessel count (P=0.041), artery branching angle (P=0.005), vein branching angle (P=0.001), artery angle asymmetry degree (P=0.003), vessel length density (P=0.000), and vessel area density (P=0.000), totaling 10 parameters. The deep learning-based model facilitates retinal vascular parameter identification and quantification, revealing significant differences. These parameters exhibit potential as biomarkers for prediabetes and diabetes.

Multi-center external validation of an automated method segmenting and differentiating atypical lipomatous tumors from lipomas using radiomics and deep-learning on MRI

As differentiating between lipomas and atypical lipomatous tumors (ALTs) based on imaging is challenging and requires biopsies, radiomics has been proposed to aid the diagnosis. This study aimed to externally and prospectively validate a radiomics model differentiating between lipomas and ALTs on MRI in three large, multi-center cohorts, and extend it with automatic and minimally interactive segmentation methods to increase clinical feasibility. Three study cohorts were formed, two for external validation containing data from medical centers in the United States (US) collected from 2008 until 2018 and the United Kingdom (UK) collected from 2011 until 2017, and one for prospective validation consisting of data collected from 2020 until 2021 in the Netherlands. Patient characteristics, MDM2 amplification status, and MRI scans were collected. An automatic segmentation method was developed to segment all tumors on T1-weighted MRI scans of the validation cohorts. Segmentations were subsequently quality scored. In case of insufficient quality, an interactive segmentation method was used. Radiomics performance was evaluated for all cohorts and compared to two radiologists. The validation cohorts included 150 (54% ALT), 208 (37% ALT), and 86 patients (28% ALT) from the US, UK and NL. Of the 444 cases, 78% were automatically segmented. For 22%, interactive segmentation was necessary due to insufficient quality, with only 3% of all patients requiring manual adjustment. External validation resulted in an AUC of 0.74 (95% CI: 0.66, 0.82) in US data and 0.86 (0.80, 0.92) in UK data. Prospective validation resulted in an AUC of 0.89 (0.83, 0.96). The radiomics model performed similar to the two radiologists (US: 0.79 and 0.76, UK: 0.86 and 0.86, NL: 0.82 and 0.85). The radiomics model extended with automatic and minimally interactive segmentation methods accurately differentiated between lipomas and ALTs in two large, multi-center external cohorts, and in prospective validation, performing similar to expert radiologists, possibly limiting the need for invasive diagnostics. Hanarth fonds.

SeqSeg: Learning Local Segments for Automatic Vascular Model Construction.

Computational modeling of cardiovascular function has become a critical part of diagnosing, treating and understanding cardiovascular disease. Most strategies involve constructing anatomically accurate computer models of cardiovascular structures, which is a multistep, time-consuming process. To improve the model generation process, we herein present SeqSeg (sequential segmentation): a novel deep learning-based automatic tracing and segmentation algorithm for constructing image-based vascular models. SeqSeg leverages local U-Net-based inference to sequentially segment vascular structures from medical image volumes. We tested SeqSeg on CT and MR images of aortic and aortofemoral models and compared the predictions to those of benchmark 2D and 3D global nnU-Net models, which have previously shown excellent accuracy for medical image segmentation. We demonstrate that SeqSeg is able to segment more complete vasculature and is able to generalize to vascular structures not annotated in the training data.

Automated quantification of crack length and width in asphalt pavements

AbstractRapid, accurate, and fully automated estimation of both length and width of asphalt pavement cracks, essential for achieving a proactive asset management, presents a significant challenge, primarily due to limitations in the effectiveness of automatic image segmentation and the accuracy of crack width and length estimation algorithms. To address this challenge, this paper introduces the Branch Growing (BG) algorithm, specifically designed for crack length estimation in asphalt pavements, along with an optimized OrthoBoundary algorithm tailored for crack width estimation. Leveraging four widely adopted deep learning models for asphalt pavement crack segmentation, four distinct sets of image segmentation results have been produced. Subsequently, a comprehensive evaluation has been conducted to assess the effectiveness of both crack dimensions estimation algorithms. The findings demonstrate that the integration of the BG algorithm, the optimized OrthoBoundary algorithm, and the fully convolutional network with the HRNet backbone achieve a prediction accuracy of 80.21% for crack length estimation and 84.32% for average width estimation. Moreover, the image processing speed, at a resolution of 3024 × 3024, can be maintained at approximately 5 s, with average width estimation observed to be up to 9.1‐fold faster than the unoptimized OrthoBoundary algorithm. These results signify advancements in automated crack quantification methodologies, with implications for enhancing civil infrastructure maintenance practices.

Evaluating the Impact of Filtering Techniques on Deep Learning-Based Brain Tumour Segmentation

Gliomas are a common and aggressive kind of brain tumour that is difficult to diagnose due to their infiltrative development, variable clinical presentation, and complex behaviour, making them an important focus in neuro-oncology. Segmentation of brain tumour images is critical for improving diagnosis, prognosis, and treatment options. Manually segmenting brain tumours is time-consuming and challenging. Automatic segmentation algorithms can significantly improve the accuracy and efficiency of tumour identification, thus improving treatment planning and outcomes. Deep learning-based segmentation tumours have shown significant advances in the last few years. This study evaluates the impact of four denoising filters, namely median, Gaussian, anisotropic diffusion, and bilateral, on tumour detection and segmentation. The U-Net architecture is applied for the segmentation of 3064 contrast-enhanced magnetic resonance images from 233 patients diagnosed with meningiomas, gliomas, and pituitary tumours. The results of this work demonstrate that bilateral filtering yields superior outcomes, proving to be a robust and computationally efficient approach in brain tumour segmentation. This method reduces the processing time by 12 epochs, which in turn contributes to lowering greenhouse gas emissions by optimizing computational resources and minimizing energy consumption.

Dataset-level color augmentation and multi-scale exploration methods for polyp segmentation

Automatic segmentation of polyps from colonoscopy images plays a critical role in early screening and treatment of colorectal cancer. Although deep learning methods have made significant progress, precise polyp segmentation faces two challenges: (1) the imbalance of color appearances in the limited training dataset hinders the generalization of the model, and (2) polyps have the diverse scales, locations and shapes with blurred boundary. To address the issues, Dataset-Level Color Augmentation (DLCA) and Convolutional Multi-scale Attention Module (CMAM) are proposed. DLCA employs the dataset-level color knowledge and generate new color appearances, to avoid the model learning false associations with colors. Meanwhile, CMAM simultaneously explores the region and boundary clues and model multi-scale context, which improves the accuracy of polyp localization and fine-grained segmentation. We conduct comprehensive experiments and compare our network with state-of-the-art methods. The proposed model is superior on multiple polyp datasets, especially on ETIS, where mDice and mIoU reach 0.839 and 0.766. Furthermore, it is validated that DLCA can be widely applied to most polyp segmentation methods, and CMAM is practical and plug-and-play. Finally, the model demonstrates promising generalization to breast ultrasound and skin lesion segmentation.

External validation of a deep learning model for automatic segmentation of skeletal muscle and adipose tissue on abdominal computed tomography images.

Body composition assessment using computed tomography (CT) images at the L3-level is increasingly applied in cancer research. Robust high-throughput automated segmentation is key to assess large patient cohorts and to support implementation of body composition analysis into routine clinical practice. We trained and externally validated a deep learning neural network (DLNN) to automatically segment L3-CT images. Expert-drawn segmentations of visceral and subcutaneous adipose tissue (VAT/SAT) and skeletal muscle (SM) of L3-CT-images of 3,187 patients undergoing abdominal surgery were used to train a DLNN. The external validation cohort was comprised of 2,535 patients with abdominal cancer. DLNN performance was evaluated with (geometric) Dice Similarity (DS) and Lin's Concordance Correlation Coefficient. There was a strong concordance between automatic and manual segmentations with median DS for SM, VAT, and SAT of 0.97 (interquartile range, IQR: 0.95-0.98), 0.98 (IQR: 0.95-0.98), and 0.95 (IQR: 0.92-0.97), respectively. Concordance correlations were excellent: SM 0.964 (0.959-0.968), VAT 0.998 (0.998-0.998), and SAT 0.992 (0.991-0.993). Bland-Altman metrics indicated only small and clinically insignificant systematic offsets; SM radiodensity: 0.23 hounsfield units (0.5%), SM: 1.26 cm2.m-2 (2.8%), VAT: -1.02 cm2.m-2 (1.7%), and SAT: 3.24 cm2.m-2 (4.6%). A robustly-performing and independently externally validated DLNN for automated body composition analysis was developed. CT-based body composition analysis is highly prognostic for long-term overall survival in oncology. This DLNN was succesfully trained and externally validated on several large patient cohorts and will therefore enable large scale population studies and implementation of body composition analysis into clinical practice.

End-to-End Fully Automated Lung Cancer Volume Estimation System

The volume of the tumor plays a very crucial role in deciding the stage of lung cancer which in turn helps in deciding the best treatment and its schedule. Currently used computer-based volume estimation techniques are semi-automatic with limited accuracy. For any automatic lung cancer segmentation system, lung CT scans of hundreds of patients are required along with their corresponding annotated segmentation masks. It is difficult to get accurately annotated data as cancer segmentation of CT scans done by the radiologists, is a time-consuming manual process. Also, it is subjective and prone to intra and inter-observer variability. Further, owing to the irregular shape of the cancerous tumor, accurate volume estimation becomes a challenge with regular convolution models. This paper proposes an end-to-end automatic tumor volume estimation model that estimates volume using the GPR (Gaussian Process Regression) interpolation method. The proposed modified cancer segmentation model uses deformable convolutions. This modification offers a higher segmentation accuracy in terms of IoU (Intersection over Union) and clearly defined nodule boundaries with correct retention of the nodule shape. The research was undertaken in collaboration with Nanavati Hospital, Mumbai, and all the models were validated on a real dataset obtained from the hospital. The proposed model gives a mean segmentation IoU (Intersection over Union) of 0.9035 and a volume estimation accuracy of 93.13% which are almost 5% and 3% higher than 0.8548 and 90.51% which are the corresponding results obtained using a standard U-net++ algorithm.

Morphometry of medial temporal lobe subregions using high-resolution T2-weighted MRI in ADNI3: Why, how, and what's next?

This paper for the 20th anniversary of the Alzheimer's Disease Neuroimaging Initiative (ADNI) provides an overview of magnetic resonance imaging (MRI) of medial temporal lobe (MTL) subregions in ADNI using a dedicated high-resolution T2-weighted sequence. A review of the work that supported the inclusion of this imaging modality into ADNI Phase 3 is followed by a brief description of the ADNI MTL imaging and analysis protocols and a summary of studies that have used these data. This review is supplemented by a new study that uses novel surface-based tools to characterize MTL neurodegeneration across biomarker-defined AD stages. This analysis reveals a pattern of spreading cortical thinning associated with increasing levels of tau pathology in the presence of elevated amyloid beta, with apparent epicenters in the transentorhinal region and inferior hippocampal subfields. The paper concludes with an outlook for high-resolution imaging of the MTL in ADNI Phase 4. HIGHLIGHTS: As of Phase 3, the Alzheimer's Disease Neuroimaging Initiative (ADNI) magnetic resonance imaging (MRI) protocol includes a high-resolution T2-weighted MRI scan optimized for imaging hippocampal subfields and medial temporal lobe (MTL) subregions. These scans are processed by the ADNI core to obtain automatic segmentations of MTL subregions and to derive morphologic measurements. More detailed granular examination of MTL neurodegeneration in response to disease progression is achieved by applying surface-based modeling techniques. Surface-based analysis of gray matter loss in MTL subregions reveals increasing and spatially expanding patterns of neurodegeneration with advancing stages of Alzheimer's disease (AD), as defined based on amyloid and tau positron emission tomography biomarkers in accordance with recently proposed criteria. These patterns closely align with post mortem literature on spread of pathological tau in AD, supporting the role of tau pathology in the presence of elevated levels of amyloid beta as the driver of neurodegeneration.

Medical image analysis using improved SAM-Med2D: segmentation and classification perspectives

Recently emerged SAM-Med2D represents a state-of-the-art advancement in medical image segmentation. Through fine-tuning the Large Visual Model, Segment Anything Model (SAM), on extensive medical datasets, it has achieved impressive results in cross-modal medical image segmentation. However, its reliance on interactive prompts may restrict its applicability under specific conditions. To address this limitation, we introduce SAM-AutoMed, which achieves automatic segmentation of medical images by replacing the original prompt encoder with an improved MobileNet v3 backbone. The performance on multiple datasets surpasses both SAM and SAM-Med2D. Current enhancements on the Large Visual Model SAM lack applications in the field of medical image classification. Therefore, we introduce SAM-MedCls, which combines the encoder of SAM-Med2D with our designed attention modules to construct an end-to-end medical image classification model. It performs well on datasets of various modalities, even achieving state-of-the-art results, indicating its potential to become a universal model for medical image classification.

Anatomo-functional changes in neural substrates of cognitive memory in developmental amnesia: Insights from automated and manual Magnetic Resonance Imaging examinations.

Despite bilateral hippocampal damage dating to the perinatal or early childhood period and severely impaired episodic memory, patients with developmental amnesia continue to exhibit well-developed semantic memory across the developmental trajectory. Detailed information on the extent and focality of brain damage in these patients is needed to hypothesize about the neural substrate that supports their remarkable capacity for encoding and retrieval of semantic memory. In particular, we need to assess whether the residual hippocampal tissue is involved in this preservation, or whether the surrounding cortical areas reorganize to rescue aspects of these critical cognitive memory processes after early injury. We used voxel-based morphometry (VBM) analysis, automatic (FreeSurfer) and manual segmentation to characterize structural changes in the brain of an exceptionally large cohort of 23 patients with developmental amnesia in comparison with 32 control subjects. Both the VBM and the FreeSurfer analyses revealed severe structural alterations in the hippocampus and thalamus of patients with developmental amnesia. Milder damage was found in the amygdala, caudate, and parahippocampal gyrus. Manual segmentation demonstrated differences in the degree of atrophy of the hippocampal subregions in patients. The level of atrophy in CA-DG subregions and subicular complex was more than 40%, while the atrophy of the uncus was moderate (-24%). Anatomo-functional correlations were observed between the volumes of residual hippocampal subregions in patients and selective aspects of their cognitive performance, viz, intelligence, working memory, and verbal and visuospatial recall. Our findings suggest that in patients with developmental amnesia, cognitive processing is compromised as a function of the extent of atrophy in hippocampal subregions. More severe hippocampal damage may be more likely to promote structural and/or functional reorganization in areas connected to the hippocampus. In this hypothesis, different levels of hippocampal function may be rescued following this variable reorganization. Our findings document not only the extent, but also the limits of circuit reorganization occurring in the young brain after early bilateral hippocampal damage.

Zero-shot image segmentation for monitoring thermal conditions of individual cage-free laying hens

Body temperature is a critical indicator of the health and productivity of egg-laying chickens and other domesticated animals. Recent advancements in thermography allow for precise surface temperature measurement without physical contact with animals, reducing animal stress from human handling. Gold standard temperature analysis via thermography requires manual selection of limited points for an object of interest, which could be time-consuming and inadequate for representing the comprehensive thermal profile of a chicken’s body. The objective of this study was to leverage and optimize a zero-shot artificial intelligence technology for the automatic segmentation of individual cage-free laying hens within thermal images, providing insights into their overall thermal conditions. A zero-shot image segmentation model (Segment Anything, “SAM”) was modified by replacing manual selections of target points with automatic selection of the initial point using pre-processing techniques (e.g., thresholding) in each thermal image. The model was also incorporated with post-processing techniques integrated with a machine learning classifier to improve segmentation accuracy. Three versions of modified SAM models (i.e., SAM, FastSAM, and MobileSAM), two common instance segmentation algorithms (i.e., YOLOv8 and Mask R-CNN), and two foundation segmentation models (i.e., U2-Net and ISNet) were comparatively evaluated to determine the optimal one for bird segmentation from thermal images. A total of 1,917 thermal images were collected from cage-free laying hens (Hy-Line W-36) at 77–80 weeks of age. The image dataset exhibited considerable variations such as feathers, bird movement, body gestures, and the specific conditions of cage-free facilities. The experimental results demonstrate that the modified SAM did not only surpass the six zero-shot models—YOLOv8, Mask R-CNN, FastSAM, MobileSAM, U2Net, and ISNet—but also outperformed other modified SAM-based models (Modified FastSAM and Modified MobileSAM) in terms of hen detection performance, achieving a success rate of 84.4 %, and in segmentation performance, with an intersection over union of 85.5 %, recall of 91.0 %, and an F1 score of 92.3 %. The optimal model, modified SAM, was pipelined to extract statistics including the averages (°C) of mean (27.03, 27.04, 28.53, 26.68), median (26.27, 26.84, 28.28, 26.78), 25th percentile (25.33, 25.61, 27.26, 25.53), and 75th percentile (28.04, 27.95, 29.22, 27.55) of surface body temperature of individual laying hens in thermal images for each week. More statistics of hen body surface temperature can be extracted based on the segmentation results. The developed pipeline is a useful tool for automatically evaluating the thermal conditions of individual birds.

Automatic segmentation of echocardiographic images using a shifted windows vision transformer architecture

Echocardiography is one the most commonly used imaging modalities for the diagnosis of congenital heart disease. Echocardiographic image analysis is crucial to obtaining accurate cardiac anatomy information. Semantic segmentation models can be used to precisely delimit the borders of the left ventricle, and allow an accurate and automatic identification of the region of interest, which can be extremely useful for cardiologists. In the field of computer vision, convolutional neural network (CNN) architectures remain dominant. Existing CNN approaches have proved highly efficient for the segmentation of various medical images over the past decade. However, these solutions usually struggle to capture long-range dependencies, especially when it comes to images with objects of different scales and complex structures. In this study, we present an efficient method for semantic segmentation of echocardiographic images that overcomes these challenges by leveraging the self-attention mechanism of the Transformer architecture. The proposed solution extracts long-range dependencies and efficiently processes objects at different scales, improving performance in a variety of tasks. We introduce Shifted Windows Transformer models (Swin Transformers), which encode both the content of anatomical structures and the relationship between them. Our solution combines the Swin Transformer and U-Net architectures, producing a U-shaped variant. The validation of the proposed method is performed with the EchoNet-Dynamic dataset used to train our model. The results show an accuracy of 0.97, a Dice coefficient of 0.87, and an Intersection over union (IoU) of 0.78. Swin Transformer models are promising for semantically segmenting echocardiographic images and may help assist cardiologists in automatically analyzing and measuring complex echocardiographic images.