Average Intersection Over Union Research Articles

Using an Ensemble of Segmentation Methods to Detect Vertebral Bodies on Radiographs.

Vertebral compression fractures may indicate osteoporosis but are underdiagnosed and underreported by radiologists. We have developed an ensemble of vertebral body (VB) segmentation models for lateral radiographs as a critical component of an automated, opportunistic screening tool. Our goal is to detect the approximate location of thoracic and lumbar VBs, including fractured vertebra, on lateral radiographs. The Osteoporotic Fractures in Men Study (MrOS) data set includes spine radiographs of 5994 men aged ≥65 years from 6 clinical centers. Two segmentation models, U-Net and Mask-RCNN (Region-based Convolutional Neural Network), were independently trained on the MrOS data set retrospectively, and an ensemble was created by combining them. Primary performance metrics for VB detection success included precision, recall, and F1 score for object detection on a held-out test set. Intersection over union (IoU) and Dice coefficient were also calculated as secondary metrics of performance for the test set. A separate external data set from a quaternary health care enterprise was acquired to test generalizability, comprising diagnostic clinical radiographs from men and women aged ≥65 years. The trained models achieved F1 score of U-Net = 83.42%, Mask-RCNN = 86.30%, and ensemble = 88.34% in detecting all VBs, and F1 score of U-Net = 87.88%, Mask-RCNN = 92.31%, and ensemble = 97.14% in detecting severely fractured vertebrae. The trained models achieved an average IoU per VB of 0.759 for U-Net and 0.709 for Mask-RCNN. The trained models achieved F1 score of U-Net = 81.11%, Mask-RCNN = 79.24%, and ensemble = 87.72% in detecting all VBs in the external data set. An ensemble model combining predictions from U-Net and Mask-RCNN resulted in the best performance in detecting VBs on lateral radiographs and generalized well to an external data set. This model could be a key component of a pipeline to detect fractures on all vertebrae in a radiograph in an automated, opportunistic screening tool under development.

Unveiling the human-like similarities of automatic facial expression recognition: An empirical exploration through explainable ai

AbstractFacial expression recognition is vital for human behavior analysis, and deep learning has enabled models that can outperform humans. However, it is unclear how closely they mimic human processing. This study aims to explore the similarity between deep neural networks and human perception by comparing twelve different networks, including both general object classifiers and FER-specific models. We employ an innovative global explainable AI method to generate heatmaps, revealing crucial facial regions for the twelve networks trained on six facial expressions. We assess these results both quantitatively and qualitatively, comparing them to ground truth masks based on Friesen and Ekman’s description and among them. We use Intersection over Union (IoU) and normalized correlation coefficients for comparisons. We generate 72 heatmaps to highlight critical regions for each expression and architecture. Qualitatively, models with pre-trained weights show more similarity in heatmaps compared to those without pre-training. Specifically, eye and nose areas influence certain facial expressions, while the mouth is consistently important across all models and expressions. Quantitatively, we find low average IoU values (avg. 0.2702) across all expressions and architectures. The best-performing architecture averages 0.3269, while the worst-performing one averages 0.2066. Dendrograms, built with the normalized correlation coefficient, reveal two main clusters for most expressions: models with pre-training and models without pre-training. Findings suggest limited alignment between human and AI facial expression recognition, with network architectures influencing the similarity, as similar architectures prioritize similar facial regions.

Chondroitin/dermatan sulphate proteoglycan, desmosealin, showing affinity to desmosomes.

Desmosomes are the most prominent interkeratinocyte junctions. The correct barrier function of stratified epithelia such as epidermis depends on their expression. During epidermal differentiation, the molecular composition of desmosomes evolves and so do their physical and chemical properties. Desquamation of corneocytes at the surface of the stratum corneum depends on an orderly degradation of desmosomes by endogenous enzymes. This process may be regulated by glycosylated molecules. We focused on the detection and characterization of potentially implicated players bearing ‘sugar’ characteristics. Using an original monoclonal antibody and biochemical methods, we partially characterized a proteoglycan of the exclusively chondroitin/dermatan sulphate type, secreted into the interkeratinocyte spaces, that is incorporated into the extracellular parts of desmosomes in quantities proportional to the degree of cell differentiation, as visualized with immuno-electron microscopy. This antigen, that we named desmosealin, displays biochemical and immunocytochemical characteristics that clearly differentiate it from known desmosomal elements. Unlike so far described epidermal proteoglycans, which belong to the heparan sulphate family, desmosealin displays chondroitin/dermatan sulphate glycosaminoglycan chains. It can be detected within the extracellular ‘cores’ of desmosomes in the upper viable epidermal layers and in corneodesmosomes from the lowermost part of the stratum corneum. Extensive integration of proteoglycans into the extracellular parts of desmosomes at the late stages of keratinocyte maturation is likely of functional importance. Given its biochemical profile, its pattern of expression in the epidermis and its desmosomal localization, desmosealin may emerge as a key element in the regulation of desmosome processing, epidermal cohesion and formation of a functional epidermal barrier.

205 Detection of anemic sheep using ocular conjunctiva images and deep learning technology

Abstract In sheep, severe anemia often results from gastrointestinal nematodes infections, commonly caused by Haemonchus contortus, a blood-sucking nematode. The objective of the study was to develop a model to predict packed cell volume (PCV) in sheep through ocular conjunctiva images as a real-time anemia diagnosis approach. We collected 3,291 ocular conjunctiva images from 392 sheep on three different farms using an iPhone 8 (12-megapixel). To identify the region of interest in the images (ocular conjunctiva), we annotated 480 images using the Segment Anything Model (SAM). Subsequently, we employed an image segmentation algorithm based on U-net, utilizing the original images and annotations obtained from SAM. We then cropped the segmented images to retain only the region containing the ocular conjunctiva. These cropped and segmented images were used as input data, with PCV as the target variable, in both classification and regression models, while exploring three different deep neural network (DNN) architectures: VGG19, Inception v3, and Xception. We set a threshold of 27% (anemic < 27%, non-anemic ≥ 27%) to convert PCV into a binary variable for classification models. The dataset was split in train, validation and test sets using random by sheep strategy to perform the analysis of the segmentation and prediction models. The segmentation was tested using intersection over union (IoU) as an evaluation metric. To assess and compare predictive quality in the test set, we calculated accuracy, precision, recall, and F1 score for classification models, and R2 for regression models. The U-net segmentation model demonstrated good localization accuracy and reliable segmentation ability, with an average IoU of 0.93, 0.84 and 0.68 in the train, validation and test set, respectively. VGG19 outperformed the other models in classifying individuals as anemic or non-anemic, achieving an F1 score of 0.60, indicating its moderate ability to distinguish between these two classes. For regression, Xception provided the best performance with an R2 of 0.29, suggesting that 29% of the variance in PCV can be explained by the model’s predictions. This innovative approach not only expands the possibilities for integrated high-throughput phenotyping through computer vision but also aids in identifying anemic animals. The results suggest that using ocular conjunctiva images and DNN technology can contribute to support farm-level management decisions, and potentially reduce economic losses due to parasitic infections like H. contortus. Furthermore, this approach reduces the cost of blood tests for anemia identification while allowing for real-time anemia diagnosis.

A cascading learning method with SegFormer for radiographic measurement of periodontal bone loss

ObjectiveMarginal alveolar bone loss is one of the key features of periodontitis and can be observed via panoramic radiographs. This study aimed to establish a cascading learning method with deep learning (DL) for precise radiographic bone loss (RBL) measurements at specific tooth positions.Materials and methodsThrough the design of two tasks for tooth position recognition and tooth semantic segmentation using the SegFormer model, specific tooth’s crown, intrabony portion, and suprabony portion of the roots were obtained. The RBL was subsequently measured by length through these three areas using the principal component analysis (PCA) principal axis.ResultsThe average intersection over union (IoU) for the tooth position recognition task was 0.8906, with an F1-score of 0.9338. The average IoU for the tooth semantic segmentation task was 0.8465, with an F1-score of 0.9138. When the two tasks were combined, the average IoU was 0.7889, with an F1-score of 0.8674. The correlation coefficient between the RBL prediction results based on the PCA principal axis and the clinicians’ measurements exceeded 0.85. Compared to those of the other two methods, the average precision of the predicted RBL was 0.7722, the average sensitivity was 0.7416, and the average F1-score was 0.7444.ConclusionsThe method for predicting RBL using DL and PCA produced promising results, offering rapid and reliable auxiliary information for future periodontal disease diagnosis.Clinical relevancePrecise RBL measurements are important for periodontal diagnosis. The proposed RBL-SF can measure RBL at specific tooth positions and assign the bone loss stage. The ability of the RBL-SF to measure RBL at specific tooth positions can guide clinicians to a certain extent in the accurate diagnosis of periodontitis.

U-Net-Based Automatic Segmentation of Sphenoid Sinus Fluid in Drowning Cases Using Postmortem CT Images: A Feasibility Study

Detecting sphenoid sinus fluid (SSF) is an additional finding in autopsies for diagnosing drowning. SSF can provide additional forensic evidence through laboratory tests such as diatom and electrolyte analyses. If drowning is suspected, accurately assessing the presence and volume of SSF during an autopsy is crucial. Utilizing postmortem computed tomography (PMCT) images could aid in accurately sampling SSF. Accurately segmenting the region of interest is essential for volume analysis using computed tomography images. However, manual segmentation techniques are labor-intensive and time-consuming, and their success depends on the experience of the observer. Therefore, this study aimed to develop a U-Net–based deep learning model for the automatic segmentation of SSF in drowning cases using PMCT images and to evaluate the performance of the model. We retrospectively reviewed 34 drowning cases in which both PMCT scans and forensic autopsies were performed at our institution. The U-Net architecture of deep learning was used for automatic segmentation. The proposed model achieved the Dice similarity coefficient (DSC) and Intersection over Union (IoU) of a maximum of 95.85% and 92.03%, a minimum of 0% and 0%, and an average of 77.15% and 67.18%, respectively. Although the average DSC and IoU did not show high similarity, this study showed that PMCT images can be used for automatic segmentation of SSF in drowning cases, which could improve the performance with sufficient dataset acquisition and further model training.

Evaluating surgical expertise with AI-based automated instrument recognition for robotic distal gastrectomy.

Complexities of robotic distal gastrectomy (RDG) give reason to assess physician's surgical skill. Varying levels in surgical skill affect patient outcomes. We aim to investigate how a novel artificial intelligence (AI) model can be used to evaluate surgical skill in RDG by recognizing surgical instruments. Fifty-five consecutive robotic surgical videos of RDG for gastric cancer were analyzed. We used Deeplab, a multi-stage temporal convolutional network, and it trained on 1234 manually annotated images. The model was then tested on 149 annotated images for accuracy. Deep learning metrics such as Intersection over Union (IoU) and accuracy were assessed, and the comparison between experienced and non-experienced surgeons based on usage of instruments during infrapyloric lymph node dissection was performed. We annotated 540 Cadiere forceps, 898 Fenestrated bipolars, 359 Suction tubes, 307 Maryland bipolars, 688 Harmonic scalpels, 400 Staplers, and 59 Large clips. The average IoU and accuracy were 0.82 ± 0.12 and 87.2 ± 11.9% respectively. Moreover, the percentage of each instrument's usage to overall infrapyloric lymphadenectomy duration predicted by AI were compared. The use of Stapler and Large clip were significantly shorter in the experienced group compared to the non-experienced group. This study is the first to report that surgical skill can be successfully and accurately determined by an AI model for RDG. Our AI gives us a way to recognize and automatically generate instance segmentation of the surgical instruments present in this procedure. Use of this technology allows unbiased, more accessible RDG surgical skill.

Intelligent overheating fault diagnosis for overhead transmission line using semantic segmentation

AbstractThe strain clamps and leading wires are important components that connect conductors on overhead transmission lines and conduct current. During operation, poor contact between these components can cause abnormal overheating, leading to electric failures and threatening power system reliability. Recently, the use of unmanned aerial vehicles equipped with infrared thermal imagers for strain clamp and leading wire maintenance has become increasingly popular. Deep learning‐based image recognition shows promising prospects for intelligent fault diagnosis of overheating faults. A pre‐treatment method is proposed based on dynamic histogram equalisation to enhance the contrast of infrared images. The DeepLab v3+ network, loss function, and existing networks with different backbones are compared. The DeepLab v3+ network with ResNet101 and convolutional block attention module added, and the Focal Loss function achieved the highest performance with an average pixel accuracy of 0.614, an average intersection over union (AIoU) of 0.567, an F1 score of 0.644, and a frequency weighted intersection over union of 0.594 on the test set. The optimised Atrous rates has increased the AIoU by 12.91%. Moreover, an intelligent diagnosis scheme for evaluating the defect state of the strain clamps and leading wires is proposed and which achieves a diagnostic accuracy of 91.0%.

Convolutional Neural Network Approaches in Median Nerve Morphological Assessment from Ultrasound Images.

Ultrasound imaging has been used to investigate compression of the median nerve in carpal tunnel syndrome patients. Ultrasound imaging and the extraction of median nerve parameters from ultrasound images are crucial and are usually performed manually by experts. The manual annotation of ultrasound images relies on experience, and intra- and interrater reliability may vary among studies. In this study, two types of convolutional neural networks (CNNs), U-Net and SegNet, were used to extract the median nerve morphology. To the best of our knowledge, the application of these methods to ultrasound imaging of the median nerve has not yet been investigated. Spearman's correlation and Bland-Altman analyses were performed to investigate the correlation and agreement between manual annotation and CNN estimation, namely, the cross-sectional area, circumference, and diameter of the median nerve. The results showed that the intersection over union (IoU) of U-Net (0.717) was greater than that of SegNet (0.625). A few images in SegNet had an IoU below 0.6, decreasing the average IoU. In both models, the IoU decreased when the median nerve was elongated longitudinally with a blurred outline. The Bland-Altman analysis revealed that, in general, both the U-Net- and SegNet-estimated measurements showed 95% limits of agreement with manual annotation. These results show that these CNN models are promising tools for median nerve ultrasound imaging analysis.

Safety Helmet Detection on Field Project Worker Using Detection Transformer

There have been many cases of work accidents caused by not complying with safety standards at work, especially in the use of safety helmets. This study is able to make regular observations in identifying project personnel using safety helmets at work, this aims to reduce the risk of accidents at work, namely in the use of helmet attributes at work. Some previous studies, have proposed the use of image detection-based models using the Detection Transformer (DeTr) method for obtaining object detection, group prediction, and combining methods, using the Intersection over Union (IoU) method for obtaining object detection results, to achieve the best performance, namely to get convergence results. Based on the combination of these two methods, the results value of average IoU is 0.50 from 500 identified project personnel data were obtained.

Multi-class Tissue Segmentation of CT images using an Ensemble Deep Learning method.

Microwave ablation (MWA) therapy is a well-known technique for locally destroying lung tumors with the help of computed tomography (CT) images. However, tumor recurrence occurs because of insufficient ablation of the tumor. In order to perform an accurate treatment of lung cancer, there is a demand to determine the tumor area precisely. To address the problem at hand, which involves accurately segmenting organs and tumors in CT images obtained during MWA therapy, physicians could benefit from a semantic segmentation method. However, such a method typically requires a large number of images to achieve optimal results through deep learning techniques. To overcome this challenge, our team developed four different (multiple) U-Net based semantic segmentation models that work in conjunction with one another to produce a more precise segmented image, even when working with a relatively small dataset. By combining the highest weight value of segmentation from multiple methods into a single output, we can achieve a more reliable and accurate segmentation outcome. Our approach proved successful in segmenting four different tissue structures, including lungs, lung tumors, and ablated tissues in CT medical images. The Intersection over Union (IoU) is employed to quantitatively evaluate the proposed method. The method shows the highest average IoU, with 0.99 for the background, 0.98 for the lung, 0.77 for the ablated, and 0.54 for the tumor tissue. The results show that employing multiple DL methods is superior to that of individual base-learner models for all four different tissue structures, even in the presence of the relatively small dataset.Clinical relevance- An essential issue of tumor ablation therapy is to know when the entire tumor tissue has completely been destroyed. However, as it is difficult to distinguish between destroyed and living tumor, this is hardly reliable in clinical practice during MWA therapy, especially when working with a small dataset. Improved AI segmentation methods can help to improve performance to reduce recurrence.

Comparison of an Optimised Multiresolution Segmentation Approach with Deep Neural Networks for Delineating Agricultural Fields from Sentinel-2 Images

Effective monitoring of agricultural lands requires accurate spatial information about the locations and boundaries of agricultural fields. Through satellite imagery, such information can be mapped on a large scale at a high temporal frequency. Various methods exist in the literature for segmenting agricultural fields from satellite images. Edge-based, region-based, or hybrid segmentation methods are traditional methods that have widely been used for segmenting agricultural fields. Lately, the use of deep neural networks (DNNs) for various tasks in remote sensing has been gaining traction. Therefore, to identify the optimal method for segmenting agricultural fields from satellite images, we evaluated three state-of-the-art DNNs, namely Mask R-CNN, U-Net, and FracTAL ResUNet against the multiresolution segmentation (MRS) algorithm, which is a region-based and a more traditional segmentation method. Given that the DNNs are supervised methods, we used an optimised version of the MRS algorithm based on supervised Bayesian optimisation. Monotemporal Sentinel-2 (S2) images acquired in Lower Saxony, Germany were used in this study. Based on the agricultural parcels declared by farmers within the European Common Agricultural Policy (CAP) framework, the segmentation results of each method were evaluated using the F-score and intersection over union (IoU) metrics. The respective average F-score and IoU obtained by each method are 0.682 and 0.524 for Mask R-CNN, 0.781 and 0.646 for U-Net, 0.808 and 0.683 for FracTAL ResUNet, and 0.805 and 0.678 for the optimised MRS approach. This study shows that DNNs, particularly FracTAL ResUNet, can be effectively used for large-scale segmentation of agricultural fields from satellite images.

An FPGA-based online reconfigurable CNN edge computing device for object detection

Edge devices offer advantages such as low computation latency and high data security for executing convolutional neural networks (CNNs). However, deploying CNNs on resource-constrained devices is challenging due to the high computational intensity of CNNs and limited hardware on-chip resources. This hinders the application of deep learning techniques on edge devices. To address this issue, this paper proposes a reconfigurable CNN edge computing system based on Field-Programmable Gate Array (FPGA) for target detection tasks. The system utilizes the pipeline structure of FPGAs to speed up network computation and employs off-chip memory to store network models. Thus, the need for tiling techniques with high intermediate cache requirements was circumvented in this system. Additionally, we developed a parallel data scheduling model to reduce storage access cost delay on network computation efficiency. Our model achieves comparable efficiency with on-chip storage-based works when using off-chip storage. Online reconfigurable design enables the system to configure network structure and parameters at runtime to achieve different target recognition. This provides greater flexibility for use cases with frequently changing requirements. The proposed system was implemented on a Spartan-6 XC6SLX150 FPGA platform and was applied to pedestrian and vehicle classification tasks. To evaluate performance, speed, power consumption, and average intersection over union (IoU) were separately measured. Our system achieved a detection speed of 16 frames per second (FPS) on the Spartan-6, with a power consumption rate of 0.79 W and an average IoU of 41.2%. Remarkably, the system demonstrated a 178% speed increase and a 60% power consumption reduction compared to the FPGA-based FitNN implementation, while classification accuracy was reduced by only 2.52%.

Extraction and Mapping of Cropland Parcels in Typical Regions of Southern China Using Unmanned Aerial Vehicle Multispectral Images and Deep Learning

The accurate extraction of cropland distribution is an important issue for precision agriculture and food security worldwide. The complex characteristics in southern China pose great challenges to the extraction. In this study, for the objective of accurate extraction and mapping of cropland parcels in multiple crop growth stages in southern China, we explored a method based on unmanned aerial vehicle (UAV) data and deep learning algorithms. Our method considered cropland size, cultivation patterns, spectral characteristics, and the terrain of the study area. From two aspects—model architecture of deep learning and the data form of UAV—four groups of experiments are performed to explore the optimal method for the extraction of cropland parcels in southern China. The optimal result obtained in October 2021 demonstrated an overall accuracy (OA) of 95.9%, a Kappa coefficient of 89.2%, and an Intersection-over-Union (IoU) of 95.7%. The optimal method also showed remarkable results in the maps of cropland distribution in multiple crop growth stages, with an average OA of 96.9%, an average Kappa coefficient of 89.5%, and an average IoU of 96.7% in August, November, and December of the same year. This study provides a valuable reference for the extraction of cropland parcels in multiple crop growth stages in southern China or regions with similar characteristics.

Artificial intelligence-aided lytic spinal bone metastasis classification on CT scans.

Spinal bone metastases directly affect quality of life, and patients with lytic-dominant lesions are at high risk for neurological symptoms and fractures. To detect and classify lytic spinal bone metastasis using routine computed tomography (CT) scans, we developed a deep learning (DL)-based computer-aided detection (CAD) system. We retrospectively analyzed 2125 diagnostic and radiotherapeutic CT images of 79 patients. Images annotated as tumor (positive) or not (negative) were randomized into training (1782 images) and test (343 images) datasets. YOLOv5m architecture was used to detect vertebra on whole CT scans. InceptionV3 architecture with the transfer-learning technique was used to classify the presence/absence of lytic lesions on CT images showing the presence of vertebra. The DL models were evaluated via fivefold cross-validation. For vertebra detection, bounding box accuracy was estimated using intersection over union (IoU). We evaluated the area under the curve (AUC) of a receiver operating characteristic curve to classify lesions. Moreover, we determined the accuracy, precision, recall, and F1 score. We used the gradient-weighted class activation mapping (Grad-CAM) technique for visual interpretation. The computation time was 0.44s per image. The average IoU value of the predicted vertebra was 0.923 ± 0.052 (0.684-1.000) for test datasets. In the binary classification task, the accuracy, precision, recall, F1-score, and AUC value for test datasets were 0.872, 0.948, 0.741, 0.832, and 0.941, respectively. Heat maps constructed using the Grad-CAM technique were consistent with the location of lytic lesions. Our artificial intelligence-aided CAD system using two DL models could rapidly identify vertebra bone from whole CT images and detect lytic spinal bone metastasis, although further evaluation of diagnostic accuracy is required with a larger sample size.

Multiple Defects Inspection of Dam Spillway Surface Using Deep Learning and 3D Reconstruction Techniques

After a lengthy period of scouring, the reinforced concrete surface of the dam spillway (i.e., drift spillways and flood discharge spillways) will suffer from deterioration and damage. Regular manual inspection is time-consuming and dangerous. This paper presents a robotic solution to detect automatically, count defect instance numbers, and reconstruct the surface of dam spillways by incorporating the deep learning method with a visual 3D reconstruction method. The lack of a real dam defect dataset and incomplete registration of minor defects on the 3D mesh model in fusion step are two challenges addressed in the paper. We created a multi-class semantic segmentation dataset of 1711 images (with resolutions of 848 × 480 and 1280 × 720 pixels) acquired by a wall-climbing robot, including cracks, erosion, spots, patched areas, and power safety cable. Then, the architecture of the U-net is modified with pixel-adaptive convolution (PAC) and conditional random field (CRF) to segment different scales of defects, trained, validated, and tested using this dataset. The reconstruction and recovery of minor defect instances in the flow surface and sidewall are facilitated using a keyframe back-projection method. By generating an instance adjacency matrix within the class, the intersection over union (IoU) of 3D voxels is calculated to fuse multiple instances. Our segmentation model achieves an average IoU of 60% for five defect class. For the surface model’s semantic recovery and instance statistics, our method achieves accurate statistics of patched area and erosion instances in an environment of 200 m2, and the average absolute error of the number of spots and cracks has reduced from the original 13.5 to 3.5.

Automated MRI Field of View Prescription from Region of Interest Prediction by Intra-Stack Attention Neural Network.

Manual prescription of the field of view (FOV) by MRI technologists is variable and prolongs the scanning process. Often, the FOV is too large or crops critical anatomy. We propose a deep learning framework, trained by radiologists' supervision, for automating FOV prescription. An intra-stack shared feature extraction network and an attention network are used to process a stack of 2D image inputs to generate scalars defining the location of a rectangular region of interest (ROI). The attention mechanism is used to make the model focus on a small number of informative slices in a stack. Then, the smallest FOV that makes the neural network predicted ROI free of aliasing is calculated by an algebraic operation derived from MR sampling theory. The framework's performance is examined quantitatively with intersection over union (IoU) and pixel error on position and qualitatively with a reader study. The proposed model achieves an average IoU of 0.867 and an average ROI position error of 9.06 out of 512 pixels on 80 test cases, significantly better than two baseline models and not significantly different from a radiologist. Finally, the FOV given by the proposed framework achieves an acceptance rate of 92% from an experienced radiologist.

Segmentation of trabecular bone microdamage in Xray microCT images using a two-step deep learning method

IntroductionOne of the current approaches to improve our understanding of osteoporosis is to study the development of bone microdamage under mechanical loading. The current practice for evaluating bone microdamage is to quantify damage volume from images of bone samples stained with a contrast agent, often composed of toxic heavy metals and requiring long tissue preparation. This work aims to evaluate the potential of linear microcracks detection and segmentation in trabecular bone samples using well-known deep learning models, namely YOLOv4 and Unet, applied on microCT images. MethodsSix trabecular bovine bone cylinders underwent compression until ultimate stress and were subsequently imaged with a microCT at a resolution of 1.95 μm. Two of these samples (samples 1 and 2) were then stained using barium sulfate (BaSO4) and imaged again. The unstained samples (samples 3–6) were used to train two neural networks YOLOv4 to detect regions with microdamage further combined with Unet to segment the microdamage at the pixel level in the detected regions. Four different model versions of YOLOv4 were compared using the average Intersection over Union (IoU) and the mean average precision (mAP). The performance of Unet was also measured using two segmentation metrics, the Dice Score and the Intersection over Union (IoU). A qualitative comparison was finally done between the deep learning and the contrast agent approaches. ResultsAmong the four versions of YOLOv4, the YOLOv4p5 model resulted in the best performance with an average IoU of 45,32% and 51,12% and a mAP of 28.79% and 46.22%, respectively for samples 1 and 2. The segmentation performance of Unet provided better IoU and DICE score on sample 2 compared to sample 1. The poorer performance of the test on sample 1 could be explained by its poorer contrast to noise ratio (CNR). Indeed, sample 1 resulted in a CNR of 7,96, which was worse than the average CNR in the training samples, while sample 2 resulted in a CNR of 10,08. The qualitative comparison between the contrast agent and the deep learning segmentation showed that two different regions were segmented by the two techniques. Deep learning is segmenting the region inside the cracks while the contrast agent segments the region around it or even regions with no visible damage. ConclusionThe combination of YOLOv4 for microdamage detection with Unet for damage segmentation showed a potential for the detection and segmentation of microdamage in trabecular bone. The accuracy of both neural networks achieved in this work is acceptable considering it is their first application in this specific field and the amount of data was limited. Even if the errors from both neural networks are accumulated, the two-steps approach is faster than the semantic segmentation of the whole volume.

Intelligent Segmentation of Intima-Media and Plaque Recognition in Carotid Artery Ultrasound Images.

Ultrasound imaging has been established as an effective method for measuring the thickness of the intima-media, the thickening of which, along with carotid plaque, is an indicator of cerebrovascular diseases. Here, a 2-D V-Net model that can automatically segment the intima-media in carotid artery ultrasound images is proposed. Moreover, a plaque recognition algorithm that automatically identifies plaque-affected areas is described. Performance tests to determine the average accuracy of the intima-media segmentation yielded the following results (expressed as lumen-intima boundary/media-adventitia boundary): intersection over union (IOU) of 0.752/0.813, pixel accuracy of 0.813/0.885 and Dice loss of 0.858/0.897. Finally, average IOU of 0.785, pixel accuracy of 0.825 and Dice loss of 0.866 were obtained for plaque recognition. These results satisfy the threshold for clinical application and indicate that the proposed model can assist doctors in making more efficient and accurate diagnoses.

Layered deep learning for automatic mandibular segmentation in cone-beam computed tomography

ObjectiveTo develop and validate a layered deep learning algorithm which automatically creates three-dimensional (3D) surface models of the human mandible out of cone-beam computed tomography (CBCT) imaging. Materials & methodsTwo convolutional networks using a 3D U-Net architecture were combined and deployed in a cloud-based artificial intelligence (AI) model. The AI model was trained in two phases and iteratively improved to optimize the segmentation result using 160 anonymized full skull CBCT scans of orthognathic surgery patients (70 preoperative scans and 90 postoperative scans). Finally, the final AI model was tested by assessing timing, consistency, and accuracy on a separate testing dataset of 15 pre- and 15 postoperative full skull CBCT scans. The AI model was compared to user refined AI segmentations (RAI) and to semi-automatic segmentation (SA), which is the current clinical standard. The time needed for segmentation was measured in seconds. Intra- and inter-operator consistency were assessed to check if the segmentation protocols delivered reproducible results. The following consistency metrics were used: intersection over union (IoU), dice similarity coefficient (DSC), Hausdorff distance (HD), absolute volume difference and root mean square (RMS) distance. To evaluate the match of the AI and RAI results to those of the SA method, their accuracy was measured using IoU, DSC, HD, absolute volume difference and RMS distance. ResultsOn average, SA took 1218.4s. RAI showed a significant drop (p<0.0001) in timing to 456.5s (2.7-fold decrease). The AI method only took 17s (71.3-fold decrease). The average intra-operator IoU for RAI was 99.5% compared to 96.9% for SA. For inter-operator consistency, RAI scored an IoU of 99.6% compared to 94.6% for SA. The AI method was always consistent by default. In both the intra- and inter-operator consistency assessments, RAI outperformed SA on all metrics indicative of better consistency. With SA as the ground truth, AI and RAI scored an IoU of 94.6% and 94.4%, respectively. All accuracy metrics were similar for AI and RAI, meaning that both methods produce 3D models that closely match those produced by SA. ConclusionA layered 3D U-Net architecture deep learning algorithm, with and without additional user refinements, improves time-efficiency, reduces operator error, and provides excellent accuracy when benchmarked against the clinical standard. Clinical significanceSemi-automatic segmentation in CBCT imaging is time-consuming and allows user-induced errors. Layered convolutional neural networks using a 3D U-Net architecture allow direct segmentation of high-resolution CBCT images. This approach creates 3D mandibular models in a more time-efficient and consistent way. It is accurate when benchmarked to semi-automatic segmentation.