Ground Truth Annotations Research Articles

Structuring Information from Plant Morphological Descriptions using Open Information Extraction

Structuring Information from Plant Morphological Descriptions using Open Information Extraction

Positive-unlabeled learning for coronary artery segmentation in CCTA images

Accurate three-dimensional (3D) segmentation of the coronary artery is an essential step in the quantitative analysis of the coronary arteries. However, due to the small size and complex morphology of the coronary arteries, voxel-by-voxel labeling of the complete coronary artery in 3D computed coronary tomography angiography images is both difficult and laborious. To alleviate the workload of annotating, it is possible to randomly label only a fraction of the positive samples and leave all remaining instances unlabeled, known as the positive-unlabeled (PU) learning problem. Due to the presence of coronary artery-like structures and the absence of negative annotations, we propose a novel sample-selection-based PU learning method for coronary artery segmentation. Specifically, only pseudo-negative labels (PNLs) are generated during the self-training process, and all data are further exploited implicitly using the teacher–student (TS) framework. To address the difficulty of detecting tiny coronary artery branches, we propose a post-processing method by exploiting the variance of multi-scale features in the inference stage. Extensive experiments were conducted on a self-constructed dataset and the publicly available ASOCA dataset. The results demonstrate that our proposed method performs better than baseline supervised and state-of-the-art PU learning methods. Notably, even in extreme cases where more than 80% of annotations are missing, our method still achieves significant gains. When the proportion of missing annotations is relatively low, our method even outperforms the backbone trained with ground truth annotations.

Multicamera edge-computing system for persons indoor location and tracking

This paper presents an indoor person localization and tracking system that uses multiple smart cameras equipped with artificial intelligence (AI) accelerators serving as edge-computing nodes. Our main contributions are as follows: (a) the development of a new multicamera tracking system for indoor scenarios; (b) the release of a multitarget multicamera tracking dataset; and (c) the development of an annotation mechanism based on waypoints. The system can simultaneously track several individuals while preserving their privacy and anonymity, because no images or sensitive data are transmitted outside the edge nodes. Only the position and appearance of each person were transmitted to the central server. In addition, a multitarget multicamera tracking dataset was released. The dataset contains recordings from five cameras in an indoor scenario and is annotated with the real-world coordinates of individuals. Ground-truth annotations were semiautomatically generated using a mechanism in which people equipped with mobile phones followed specific paths with predefined waypoints. Software related to the ground-truth annotation mechanism has also been released as open source.

Real-Time Joint-Stem Prediction for Agricultural Robots in Grasslands Using Multi-Task Learning

Autonomous weeding robots need to accurately detect the joint stem of grassland weeds in order to control those weeds in an effective and energy-efficient manner. In this work, keypoints on joint stems and bounding boxes around weeds in grasslands are detected jointly using multi-task learning. We compare a two-stage, heatmap-based architecture to a single-stage, regression-based architecture—both based on the popular YOLOv5 object detector. Our results show that introducing joint-stem detection as a second task boosts the individual weed detection performance in both architectures. Furthermore, the single-stage architecture clearly outperforms its competitors with an OKS of 56.3 in joint-stem detection while also achieving real-time performance of 12.2 FPS on Nvidia Jetson NX, suitable for agricultural robots. Finally, we make the newly created joint-stem ground-truth annotations publicly available for the relevant research community.

Intra-oral scan segmentation using deep learning

ObjectiveIntra-oral scans and gypsum cast scans (OS) are widely used in orthodontics, prosthetics, implantology, and orthognathic surgery to plan patient-specific treatments, which require teeth segmentations with high accuracy and resolution. Manual teeth segmentation, the gold standard up until now, is time-consuming, tedious, and observer-dependent. This study aims to develop an automated teeth segmentation and labeling system using deep learning.Material and methodsAs a reference, 1750 OS were manually segmented and labeled. A deep-learning approach based on PointCNN and 3D U-net in combination with a rule-based heuristic algorithm and a combinatorial search algorithm was trained and validated on 1400 OS. Subsequently, the trained algorithm was applied to a test set consisting of 350 OS. The intersection over union (IoU), as a measure of accuracy, was calculated to quantify the degree of similarity between the annotated ground truth and the model predictions.ResultsThe model achieved accurate teeth segmentations with a mean IoU score of 0.915. The FDI labels of the teeth were predicted with a mean accuracy of 0.894. The optical inspection showed excellent position agreements between the automatically and manually segmented teeth components. Minor flaws were mostly seen at the edges.ConclusionThe proposed method forms a promising foundation for time-effective and observer-independent teeth segmentation and labeling on intra-oral scans.Clinical significanceDeep learning may assist clinicians in virtual treatment planning in orthodontics, prosthetics, implantology, and orthognathic surgery. The impact of using such models in clinical practice should be explored.

Potential of ChatGPT and GPT-4 for Data Mining of Free-Text CT Reports on Lung Cancer.

Background The latest large language models (LLMs) solve unseen problems via user-defined text prompts without the need for retraining, offering potentially more efficient information extraction from free-text medical records than manual annotation. Purpose To compare the performance of the LLMs ChatGPT and GPT-4 in data mining and labeling oncologic phenotypes from free-text CT reports on lung cancer by using user-defined prompts. Materials and Methods This retrospective study included patients who underwent lung cancer follow-up CT between September 2021 and March 2023. A subset of 25 reports was reserved for prompt engineering to instruct the LLMs in extracting lesion diameters, labeling metastatic disease, and assessing oncologic progression. This output was fed into a rule-based natural language processing pipeline to match ground truth annotations from four radiologists and derive performance metrics. The oncologic reasoning of LLMs was rated on a five-point Likert scale for factual correctness and accuracy. The occurrence of confabulations was recorded. Statistical analyses included Wilcoxon signed rank and McNemar tests. Results On 424 CT reports from 424 patients (mean age, 65 years ± 11 [SD]; 265 male), GPT-4 outperformed ChatGPT in extracting lesion parameters (98.6% vs 84.0%, P < .001), resulting in 96% correctly mined reports (vs 67% for ChatGPT, P < .001). GPT-4 achieved higher accuracy in identification of metastatic disease (98.1% [95% CI: 97.7, 98.5] vs 90.3% [95% CI: 89.4, 91.0]) and higher performance in generating correct labels for oncologic progression (F1 score, 0.96 [95% CI: 0.94, 0.98] vs 0.91 [95% CI: 0.89, 0.94]) (both P < .001). In oncologic reasoning, GPT-4 had higher Likert scale scores for factual correctness (4.3 vs 3.9) and accuracy (4.4 vs 3.3), with a lower rate of confabulation (1.7% vs 13.7%) than ChatGPT (all P < .001). Conclusion When using user-defined prompts, GPT-4 outperformed ChatGPT in extracting oncologic phenotypes from free-text CT reports on lung cancer and demonstrated better oncologic reasoning with fewer confabulations. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Hafezi-Nejad and Trivedi in this issue.

ParallelEye Pipeline: An Effective Method to Synthesize Images for Improving the Visual Intelligence of Intelligent Vehicles

Virtual simulated scenes are becoming a critical part of autonomous driving. In the context of knowledge automation and machine learning, simulated images are widely used for visual environmental perception. However, even the most inspirational applications have not fully exploited the potential of simulated images in solving real-world problems. In this article, we propose a novel framework “ParallelEye Pipeline”, which uses image-to-image translation and simulated images to automatically generate realistic synthetic images with multiple ground-truth annotations. Specifically, this method has three steps: first, we use Unity3D software to simulate driving scenarios and generate simulated image pairs (including raw images and six ground-truth labels) from the simulated scenes; second, advanced image-to-image translation algorithms can generate realistic and high-resolution synthetic images from simulated image pairs; third, we exploit publicly datasets, simulated images, and synthetic images to conduct experiments for visual perception. The experimental results suggest: 1) synthetic images and simulated images can improve the performance of detectors in real autonomous driving scenarios and 2) image-to-image translation algorithms can be affected by occlusion condition.

A proposed methodology for detecting the malignant potential of pulmonary nodules in sarcoma using computed tomographic imaging and artificial intelligence-based models.

The presence of lung metastases in patients with primary malignancies is an important criterion for treatment management and prognostication. Computed tomography (CT) of the chest is the preferred method to detect lung metastasis. However, CT has limited efficacy in differentiating metastatic nodules from benign nodules (e.g., granulomas due to tuberculosis) especially at early stages (<5mm). There is also a significant subjectivity associated in making this distinction, leading to frequent CT follow-ups and additional radiation exposure along with financial and emotional burden to the patients and family. Even 18F-fluoro-deoxyglucose positron emission technology-computed tomography (18F-FDG PET-CT) is not always confirmatory for this clinical problem. While pathological biopsy is the gold standard to demonstrate malignancy, invasive sampling of small lung nodules is often not clinically feasible. Currently, there is no non-invasive imaging technique that can reliably characterize lung metastases. The lung is one of the favored sites of metastasis in sarcomas. Hence, patients with sarcomas, especially from tuberculosis prevalent developing countries, can provide an ideal platform to develop a model to differentiate lung metastases from benign nodules. To overcome the lack of optimal specificity of CT scan in detecting pulmonary metastasis, a novel artificial intelligence (AI)-based protocol is proposed utilizing a combination of radiological and clinical biomarkers to identify lung nodules and characterize it as benign or metastasis. This protocol includes a retrospective cohort of nearly 2,000-2,250 sample nodules (from at least 450 patients) for training and testing and an ambispective cohort of nearly 500 nodules (from 100 patients; 50 patients each from the retrospective and prospective cohort) for validation. Ground-truth annotation of lung nodules will be performed using an in-house-built segmentation tool. Ground-truth labeling of lung nodules (metastatic/benign) will be performed based on histopathological results or baseline and/or follow-up radiological findings along with clinical outcome of the patient. Optimal methods for data handling and statistical analysis are included to develop a robust protocol for early detection and classification of pulmonary metastasis at baseline and at follow-up and identification of associated potential clinical and radiological markers.

3D Breast Cancer Segmentation in DCE-MRI Using Deep Learning With Weak Annotation.

Deep learning models require large-scale training to perform confidently, but obtaining annotated datasets in medical imaging is challenging. Weak annotation has emerged as a way to save time and effort. To develop a deep learning model for 3D breast cancer segmentation in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using weak annotation with reliable performance. Retrospective. Seven hundred and thirty-six women with breast cancer from a single institution, divided into the development (N = 544) and test dataset (N = 192). 3.0-T, 3D fat-saturated gradient-echo axial T1-weighted flash 3D volumetric interpolated brain examination (VIBE) sequences. Two radiologists performed a weak annotation of the ground truth using bounding boxes. Based on this, the ground truth annotation was completed through autonomic and manual correction. The deep learning model using 3D U-Net transformer (UNETR) was trained with this annotated dataset. The segmentation results of the test set were analyzed by quantitative and qualitative methods, and the regions were divided into whole breast and region of interest (ROI) within the bounding box. As a quantitative method, we used the Dice similarity coefficient to evaluate the segmentation result. The volume correlation with the ground truth was evaluated with the Spearman correlation coefficient. Qualitatively, three readers independently evaluated the visual score in four scales. A P-value <0.05 was considered statistically significant. The deep learning model we developed achieved a median Dice similarity score of 0.75 and 0.89 for the whole breast and ROI, respectively. The volume correlation coefficient with respect to the ground truth volume was 0.82 and 0.86 for the whole breast and ROI, respectively. The mean visual score, as evaluated by three readers, was 3.4. The proposed deep learning model with weak annotation may show good performance for 3D segmentations of breast cancer using DCE-MRI. 3 TECHNICAL EFFICACY: Stage 2.

SPSNet: Boosting 3D point-based object detectors with stable point sampling

Currently, the primary focus of 3D point-based detectors is to enhance their performance by primarily increasing the recall rate of foreground points at the semantic level. However, this approach is suboptimal. On one hand, ground-truth annotations of 3D bounding boxes are inherently ambiguous due to occlusions, signal distortion, or manual annotation errors, which can obscure genuine semantic information. On the other hand, the saliency of each point varies. In this paper, we propose SPSNet based on Bayes theorem to explore the mapping relationship between the sampled point and the corresponding bounding box, which we call point stability. This is a plug-and-play module and used as the basis for downsampling instead of relying on semantic information obtained through hard supervision of 3D bounding boxes. We incorporate the proposed method into the most popular point-based detector, IA-SSD, called SPSNet-IA. On the challenging KITTI validation set, with only changes to the downsampling strategy, it obtains 1.18, 2.78 and 2.96 gains for the metric of AP3D on the easy, moderate and hard levels, respectively. In addition, SPSNet-IA outperforms all published point-based approaches by a large margin and ranks 1st among single-modal methods. We also design qualitative and quantitative experiments to explain the meaning of stability learned from SPSNet. These additional experiments demonstrate that SPSNet is not only a performance-enhancing model but also a saliency analysis model. The code is available at https://github.com/AlanLiangC/SPSNet.git.

REALITY-ENHANCED SYNTHETIC IMAGE TRAINING DATASET FOR COMPUTER VISION CONSTRUCTION MONITORING

Recent growth in the availability of aerial and ground-level images from construction sites has created a surge in computer vision-based techniques for construction monitoring applications. To address the need for large volumes of visual data with ground truth for training the underlying machine learning models, the application of synthetic data, particularly from 3D/4D BIM, has gained traction in the research community. However, the existing domain gap between real and synthetic images, such as rendering noise from repeated material texture patterns, randomized positional camera parameters that collect the ground truth of BIM elements, and the poorly visibility of BIM elements to the camera viewpoints within these environments, has negatively impacted the potential use of these synthetic datasets as scale. To address these limitations, this paper presents a new synthetic image generation pipeline that optimizes and integrates camera extrinsic parameters with a novel synthetic image appearance enhancement technique to generate high volume of quality synthetic data with respective ground-truth. The use of synthetic datasets for training progress monitoring models is validated through several real-data segmentation cases that incorporate: 1) the automatic collection of synthetic images and ground-truth annotations from high-LoD BIM model disciplines, 2) optimization of positional camera parameters using element visibility metrics, and 3) the enhancement of realism of synthetic images using a patch-based generative approach. The benefits and the current limitations to automated progress monitoring, as well as robotic path automation and optimization for progress motoring real data collection are discussed.

FedUSC: Collaborative Unsupervised Representation Learning From Decentralized Data for Internet of Things

Federated Learning (FL) lately has shown much promise in improving the shared model and preserving data privacy. However, these existing methods are only of limited utility in the Internet of Things (IoT) scenarios, as they either heavily depend on high-quality labeled data or only perform well under idealized conditions, which typically cannot be found in practical applications. In this paper, we propose a novel federated unsupervised learning method for image classification without the use of any ground truth annotations. In IoT scenarios, a big challenge is that decentralized data among multiple clients is normally non-IID, leading to performance degradation. To address this issue, we further propose a dynamic update mechanism that can decide how to update the local model based on weights divergence. Extensive experiments show that our method outperforms all baseline methods by large margins, including +6.67% on CIFAR-10, +5.15% on STL-10, and +8.44% on SVHN in terms of classification accuracy. In particular, we obtain promising results on Mini-ImageNet and COVID-19 datasets and outperform several federated unsupervised learning methods under non-IID settings.

Towards reliable hepatocytic anatomy segmentation in laparoscopic cholecystectomy using U-Net with Auto-Encoder.

Most bile duct (BDI) injuries during laparoscopic cholecystectomy (LC) occur due to visual misperception leading to the misinterpretation of anatomy. Deep learning (DL) models for surgical video analysis could, therefore, support visual tasks such as identifying critical view of safety (CVS). This study aims to develop a prediction model of CVS during LC. This aim is accomplished using a deep neural network integrated with a segmentation model that is capable of highlighting hepatocytic anatomy. Still images from LC videos were annotated with four hepatocystic landmarks of anatomy segmentation. A deep autoencoder neural network with U-Net to investigate accurate medical image segmentation was trained and tested using fivefold cross-validation. Accuracy, Loss, Intersection over Union (IoU), Precision, Recall, and Hausdorff Distance were computed to evaluate the model performance versus the annotated ground truth. A total of 1550 images from 200 LC videos were annotated. Mean IoU for segmentation was 74.65%. The proposed approach performed well for automatic hepatocytic landmarks identification with 92% accuracy and 93.9% precision and can segment challenging cases. DL, can potentially provide an intraoperative model for surgical video analysis and can be trained to guide surgeons toward reliable hepatocytic anatomy segmentation and produce selective video documentation of this safety step of LC.

Stability of Multi-Parametric Prostate MRI Radiomic Features to Variations in Segmentation.

Stability analysis remains a fundamental step in developing a successful imaging biomarker to personalize oncological strategies. This study proposes an in silico contour generation method for simulating segmentation variations to identify stable radiomic features. Ground-truth annotation provided for the whole prostate gland on the multi-parametric MRI sequences (T2w, ADC, and SUB-DCE) were perturbed to mimic segmentation differences observed among human annotators. In total, we generated 15 synthetic contours for a given image-segmentation pair. One thousand two hundred twenty-four unfiltered/filtered radiomic features were extracted applying Pyradiomics, followed by stability assessment using ICC(1,1). Stable features identified in the internal population were then compared with an external population to discover and report robust features. Finally, we also investigated the impact of a wide range of filtering strategies on the stability of features. The percentage of unfiltered (filtered) features that remained robust subjected to segmentation variations were T2w-36% (81%), ADC-36% (94%), and SUB-43% (93%). Our findings suggest that segmentation variations can significantly impact radiomic feature stability but can be mitigated by including pre-filtering strategies as part of the feature extraction pipeline.

Persistent animal identification leveraging non-visual markers

Our objective is to locate and provide a unique identifier for each mouse in a cluttered home-cage environment through time, as a precursor to automated behaviour recognition for biological research. This is a very challenging problem due to (i) the lack of distinguishing visual features for each mouse, and (ii) the close confines of the scene with constant occlusion, making standard visual tracking approaches unusable. However, a coarse estimate of each mouse’s location is available from a unique RFID implant, so there is the potential to optimally combine information from (weak) tracking with coarse information on identity. To achieve our objective, we make the following key contributions: (a) the formulation of the object identification problem as an assignment problem (solved using Integer Linear Programming), (b) a novel probabilistic model of the affinity between tracklets and RFID data, and (c) a curated dataset with per-frame BB and regularly spaced ground-truth annotations for evaluating the models. The latter is a crucial part of the model, as it provides a principled probabilistic treatment of object detections given coarse localisation. Our approach achieves 77% accuracy on this animal identification problem, and is able to reject spurious detections when the animals are hidden.

DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion

Unsupervised depth completion aims to recover dense depth from the sparse one without using the ground-truth annotation. Although depth measurement obtained from LiDAR is usually sparse, it contains valid and real distance information, i.e., scale-consistent absolute depth values. Meanwhile, scale-agnostic counterparts seek to estimate relative depth and have achieved impressive performance. To leverage both the inherent characteristics, we thus suggest to model scale-consistent depth upon unsupervised scale-agnostic frameworks. Specifically, we propose the decomposed scale-consistent learning (DSCL) strategy, which disintegrates the absolute depth into relative depth prediction and global scale estimation, contributing to individual learning benefits. But unfortunately, most existing unsupervised scale-agnostic frameworks heavily suffer from depth holes due to the extremely sparse depth input and weak supervisory signal. To tackle this issue, we introduce the global depth guidance (GDG) module, which attentively propagates dense depth reference into the sparse target via novel dense-to-sparse attention. Extensive experiments show the superiority of our method on outdoor KITTI, ranking 1st and outperforming the best KBNet more than 12% in RMSE. Additionally, our approach achieves state-of-the-art performance on indoor NYUv2 benchmark as well.

MEMbrain: an interactive deep learning MATLAB tool for connectomic segmentation on commodity desktops.

Connectomics is fundamental in propelling our understanding of the nervous system's organization, unearthing cells and wiring diagrams reconstructed from volume electron microscopy (EM) datasets. Such reconstructions, on the one hand, have benefited from ever more precise automatic segmentation methods, which leverage sophisticated deep learning architectures and advanced machine learning algorithms. On the other hand, the field of neuroscience at large, and of image processing in particular, has manifested a need for user-friendly and open source tools which enable the community to carry out advanced analyses. In line with this second vein, here we propose mEMbrain, an interactive MATLAB-based software which wraps algorithms and functions that enable labeling and segmentation of electron microscopy datasets in a user-friendly user interface compatible with Linux and Windows. Through its integration as an API to the volume annotation and segmentation tool VAST, mEMbrain encompasses functions for ground truth generation, image preprocessing, training of deep neural networks, and on-the-fly predictions for proofreading and evaluation. The final goals of our tool are to expedite manual labeling efforts and to harness MATLAB users with an array of semi-automatic approaches for instance segmentation. We tested our tool on a variety of datasets that span different species at various scales, regions of the nervous system and developmental stages. To further expedite research in connectomics, we provide an EM resource of ground truth annotation from four different animals and five datasets, amounting to around 180 h of expert annotations, yielding more than 1.2 GB of annotated EM images. In addition, we provide a set of four pre-trained networks for said datasets. All tools are available from https://lichtman.rc.fas.harvard.edu/mEMbrain/. With our software, our hope is to provide a solution for lab-based neural reconstructions which does not require coding by the user, thus paving the way to affordable connectomics.

DEEP LEARNING FOR AUTOMATIC SEGMENTATION OF CLINICAL PHOTOGRAPHS OF ORAL PREMALIGNANT AND MALIGNANT LESIONS

Objective To explore the usage of Deep Learning (DL) induced models for automatic segmentation of premalignant and incipient malignant lesions on photographic images. Study Design A dataset of 308 clinical images from three institutions was used to design, train, and evaluate DL-based models. For each image, ground-truth annotation was performed by three experts and combined via union of the labelled areas, thus minimizing false negatives. The dataset was split into two subsets, with 246 training and 62 test images, with 10-fold cross-validation being applied to the first subset. The experimental results were evaluated using mean pixel-wise Intersection Over Union (IoU). Preliminary Results The best performing model was a U-Net architecture with a 224 × 224 input. The downstack section of the U-Net was a VGG16 CNN pre-trained with the ImageNet dataset, fine-tuned with the training subset. The training used random horizontal and vertical flips as data augmentation. A mean IoU of 0.675 (±0.030 std) and mean accuracy of 0.865 (±0.020 std) were obtained. Conclusion These preliminary results demonstrate the feasibility of DL-powered models for automatic segmentation of premalignant and incipient malignant lesions. The induced model can be a reliable, fast, and non-invasive screening tool for cancer detection. To explore the usage of Deep Learning (DL) induced models for automatic segmentation of premalignant and incipient malignant lesions on photographic images. A dataset of 308 clinical images from three institutions was used to design, train, and evaluate DL-based models. For each image, ground-truth annotation was performed by three experts and combined via union of the labelled areas, thus minimizing false negatives. The dataset was split into two subsets, with 246 training and 62 test images, with 10-fold cross-validation being applied to the first subset. The experimental results were evaluated using mean pixel-wise Intersection Over Union (IoU). The best performing model was a U-Net architecture with a 224 × 224 input. The downstack section of the U-Net was a VGG16 CNN pre-trained with the ImageNet dataset, fine-tuned with the training subset. The training used random horizontal and vertical flips as data augmentation. A mean IoU of 0.675 (±0.030 std) and mean accuracy of 0.865 (±0.020 std) were obtained. These preliminary results demonstrate the feasibility of DL-powered models for automatic segmentation of premalignant and incipient malignant lesions. The induced model can be a reliable, fast, and non-invasive screening tool for cancer detection.

MDB-43. GRADIENT ENTROPY-BASED RADIOMIC RISK-SCORE FROM T1-WEIGHTED PRE-TREATMENT MRI SCANS CAN PREDICT SURVIVAL IN PEDIATRIC MEDULLOBLASTOMA

Abstract BACKGROUND Medulloblastoma (MB) is the most frequent malignant brain tumor in children with an inadequate 5-year survival rate of 70-75%. A key determinate in MB treatment (consisting of radiation, chemotherapy) is the accurate risk-stratification of patients into low- and high-risk. The current stratification approaches (Chang’s classification, molecular categorization) are sub-optimal in predicting individual patient outcomes. Reliable risk-assessment approaches can significantly impact treatment management. Our group previously developed a new radiomics feature COLLAGE that measures the orientation of local intensity gradients as an estimate of lesion heterogeneity. We hypothesize that the disorder (i.e., heterogeneity) in orientation of local per-voxel intensity measured by COLLAGE, will tease-out differences in patients with low - versus high-risk MB tumors. METHODS T1-weighted MRI scans of 49 pediatric MB patients were retrospectively collected from Children’s Hospital Los Angeles and Cincinnati Children’s Hospital Medical Center. Registration of the scans was performed via age-specific pediatric atlases, followed by skull stripping and bias correction. Ground truth annotations for tumor sub-compartments; enhancing tumor, edema, and non-enhancing tumor, were generated by two experienced radiologists. Then, 52 Collage features were extracted from each of these sub-compartments, followed by conducting a Lasso model and a cox-proportional hazards model to identify the top features for risk-stratification. A continuous risk score was created for every patient and a risk threshold was obtained from the fitted cox model to stratify patients into low- and high-risk. RESULTS Our risk-score consisted of 13 top COLLAGE features from edema and 14 from non-enhancing tumor sub-compartments, respectively, and demonstrated significant differences between low- and high-risk patients (p = .017 for edema, p =0.0027 for non-enhancing tumor). CONCLUSION Our preliminary analysis suggests that gradient-entropy radiomic risk score maybe able to predict overall survival in pediatric MB tumors. In future, we plan to include additional patients to perform independent multi-site analysis.

Preserving privacy in surgical video analysis using a deep learning classifier to identify out-of-body scenes in endoscopic videos

Surgical video analysis facilitates education and research. However, video recordings of endoscopic surgeries can contain privacy-sensitive information, especially if the endoscopic camera is moved out of the body of patients and out-of-body scenes are recorded. Therefore, identification of out-of-body scenes in endoscopic videos is of major importance to preserve the privacy of patients and operating room staff. This study developed and validated a deep learning model for the identification of out-of-body images in endoscopic videos. The model was trained and evaluated on an internal dataset of 12 different types of laparoscopic and robotic surgeries and was externally validated on two independent multicentric test datasets of laparoscopic gastric bypass and cholecystectomy surgeries. Model performance was evaluated compared to human ground truth annotations measuring the receiver operating characteristic area under the curve (ROC AUC). The internal dataset consisting of 356,267 images from 48 videos and the two multicentric test datasets consisting of 54,385 and 58,349 images from 10 and 20 videos, respectively, were annotated. The model identified out-of-body images with 99.97% ROC AUC on the internal test dataset. Mean ± standard deviation ROC AUC on the multicentric gastric bypass dataset was 99.94 ± 0.07% and 99.71 ± 0.40% on the multicentric cholecystectomy dataset, respectively. The model can reliably identify out-of-body images in endoscopic videos and is publicly shared. This facilitates privacy preservation in surgical video analysis.