Held-out Dataset Research Articles

Exoplanet Detection Using Machine Learning : A Comparative Study Using Kepler Mission Data

The search for habitable planets outside our solar system has captivated scientists throughout the centuries. Discovery and characterization of exoplanets have been one of the most important endeavors of modern astronomy. With various space missions, we have significantly expanded our observational capacity, resulting in an abundance of information about the universe. The influx of more data necessitates the development of techniques that can aid astronomers in processing all the information more efficiently and in an automated manner. Machine learning in recent years has become an indispensable paradigm to automate complex tasks that are possible only by humans. This work explores the application of machine learning to detect exoplanets from NASA’s Kepler mission. Our dataset comprises Kepler Objects of Interest (KOIs), encompassing their characteristic features and confirmed exoplanet status. We experiment with multiple supervised classification techniques including classical, tree-based, and neural methods. The best-performing model Histogram Gradient Boosting achieves a strong performance of 94.6% precision and 94.1% recall on a held-out dataset demonstrating the strong potential of integrating machine learning techniques into astronomy, potentially leading to new insights into planetary systems outside the solar system.

Development of a deep-learning phenotyping tool for analyzing image-based strawberry phenotypes.

In strawberry farming, phenotypic traits (such as crown diameter, petiole length, plant height, flower, leaf, and fruit size) measurement is essential as it serves as a decision-making tool for plant monitoring and management. To date, strawberry plant phenotyping has relied on traditional approaches. In this study, an image-based Strawberry Phenotyping Tool (SPT) was developed using two deep-learning (DL) architectures, namely "YOLOv4" and "U-net" integrated into a single system. We aimed to create the most suitable DL-based tool with enhanced robustness to facilitate digital strawberry plant phenotyping directly at the natural scene or indirectly using captured and stored images. Our SPT was developed primarily through two steps (subsequently called versions) using image data with different backgrounds captured with simple smartphone cameras. The two versions (V1 and V2) were developed using the same DL networks but differed by the amount of image data and annotation method used during their development. For V1, 7,116 images were annotated using the single-target non-labeling method, whereas for V2, 7,850 images were annotated using the multitarget labeling method. The results of the held-out dataset revealed that the developed SPT facilitates strawberry phenotype measurements. By increasing the dataset size combined with multitarget labeling annotation, the detection accuracy of our system changed from 60.24% in V1 to 82.28% in V2. During the validation process, the system was evaluated using 70 images per phenotype and their corresponding actual values. The correlation coefficients and detection frequencies were higher for V2 than for V1, confirming the superiority of V2. Furthermore, an image-based regression model was developed to predict the fresh weight of strawberries based on the fruit size (R2 = 0.92). The results demonstrate the efficiency of our system in recognizing the aforementioned six strawberry phenotypic traits regardless of the complex scenario of the environment of the strawberry plant. This tool could help farmers and researchers make accurate and efficient decisions related to strawberry plant management, possibly causing increased productivity and yield potential.

Multi-scale relational graph convolutional network for multiple instance learning in histopathology images

Graph convolutional neural networks have shown significant potential in natural and histopathology images. However, their use has only been studied in a single magnification or multi-magnification with either homogeneous graphs or only different node types. In order to leverage the multi-magnification information and improve message passing with graph convolutional networks, we handle different embedding spaces at each magnification by introducing the Multi-Scale Relational Graph Convolutional Network (MS-RGCN) as a multiple instance learning method. We model histopathology image patches and their relation with neighboring patches and patches at other scales (i.e., magnifications) as a graph. We define separate message-passing neural networks based on node and edge types to pass the information between different magnification embedding spaces. We experiment on prostate cancer histopathology images to predict the grade groups based on the extracted features from patches. We also compare our MS-RGCN with multiple state-of-the-art methods with evaluations on several source and held-out datasets. Our method outperforms the state-of-the-art on all of the datasets and image types consisting of tissue microarrays, whole-mount slide regions, and whole-slide images. Through an ablation study, we test and show the value of the pertinent design features of the MS-RGCN.

A Pilot Predictive Model for Indirect Assessment of Suicidal Ideation

IntroductionIn recent years, there has been a concerning increase in suicidal thoughts and, in some countries, completed suicides, amplified by the COVID-19 pandemic. Screening for suicidal ideation (SI) in the general population is limited due to ethical, effectiveness, and feasibility concerns. Identifying individuals at risk of suicide remains a complex challenge. Our study aimed to develop a predictive model using COVID-19 data, gathering psychometric information from 1790 respondents in Slovenia via an online survey conducted between July 2020 and December 2020, with a second wave of data (ne=1200) collected from January 2022 to February 2022.ObjectivesWith 9.7% of respondents reporting recent SI in the first wave of data, our primary goal was to estimate SI indirectly using SIDAS. We examined changes in habits, demographics, coping strategies, and satisfaction in key life aspects to discreetly identify potential risk factors.MethodsWe employed four machine learning algorithms (logistic regression, random forest, XGBoost, and support vector machines) and assessed model performance using the area under the receiver operating characteristic curve (AUC). Initial assessment used a held-out dataset, followed by validation with a new cohort of 1,200 users from the late COVID-19 period.ResultsLogistic regression, random forest, and XGBoost achieved comparable AUCs, reaching 0.83 on unseen data. Our analysis revealed significant associations between Brief-COPE subscales and SI. Self-Blame emerged as a strong SI indicator, followed by increased Substance Use, reduced Positive Reframing, Behavioral Disengagement, dissatisfaction with relationships, and younger age, in both 2020 and 2022 models. The model consistently performed well, even with varying population characteristics.Conclusions These results suggest that SI presence can be reasonably estimated using selected indicators, offering promise for developing an indirect screening tool without explicit questioning about suicidal thoughts. However, individuals flagged as at-risk should undergo clinical examination, as this model serves as an initial step in identifying SI risk factors in the context of the stressful event’s (COVID-19 pandemic) impact on mental health.Disclosure of InterestNone Declared

Analysis of systems’ performance in natural language processing competitions

Collaborative competitions have gained popularity in the scientific and technological fields. These competitions involve defining tasks, selecting evaluation scores, and devising result verification methods. In the standard scenario, participants receive a training set and are expected to provide a solution for a held-out dataset kept by organizers. An essential challenge for organizers arises when comparing algorithms’ performance, assessing multiple participants, and ranking them. Statistical tools are often used for this purpose; however, traditional statistical methods often fail to capture decisive differences between systems’ performance. This manuscript describes an evaluation methodology for statistically analyzing competition results and competition. The methodology is designed to be universally applicable; however, it is illustrated using eight natural language competitions as case studies involving classification and regression problems. The proposed methodology offers several advantages, including off-the-shell comparisons with correction mechanisms and the inclusion of confidence intervals. Furthermore, we introduce metrics that allow organizers to assess the difficulty of competitions. Our analysis shows the potential usefulness of our methodology for effectively evaluating competition results.

DeepLocRNA: an interpretable deep learning model for predicting RNA subcellular localization with domain-specific transfer-learning.

Accurate prediction of RNA subcellular localization plays an important role in understanding cellular processes and functions. Although post-transcriptional processes are governed by trans-acting RNA binding proteins (RBPs) through interaction with cis-regulatory RNA motifs, current methods do not incorporate RBP-binding information. In this article, we propose DeepLocRNA, an interpretable deep-learning model that leverages a pre-trained multi-task RBP-binding prediction model to predict the subcellular localization of RNA molecules via fine-tuning. We constructed DeepLocRNA using a comprehensive dataset with variant RNA types and evaluated it on the held-out dataset. Our model achieved state-of-the-art performance in predicting RNA subcellular localization in mRNA and miRNA. It has also demonstrated great generalization capabilities, performing well on both human and mouse RNA. Additionally, a motif analysis was performed to enhance the interpretability of the model, highlighting signal factors that contributed to the predictions. The proposed model provides general and powerful prediction abilities for different RNA types and species, offering valuable insights into the localization patterns of RNA molecules and contributing to our understanding of cellular processes at the molecular level. A user-friendly web server is available at: https://biolib.com/KU/DeepLocRNA/.

Holdout predictive checks for Bayesian model criticism

Abstract Bayesian modelling helps applied researchers to articulate assumptions about their data and develop models tailored for specific applications. Thanks to good methods for approximate posterior inference, researchers can now easily build, use, and revise complicated Bayesian models for large and rich data. These capabilities, however, bring into focus the problem of model criticism. Researchers need tools to diagnose the fitness of their models, to understand where they fall short, and to guide their revision. In this paper, we develop a new method for Bayesian model criticism, the holdout predictive check (HPC). Holdout predictive check are built on posterior predictive check (PPC), a seminal method that checks a model by assessing the posterior predictive distribution on the observed data. However, PPC use the data twice—both to calculate the posterior predictive and to evaluate it—which can lead to uncalibrated p-values. Holdout predictive check, in contrast, compare the posterior predictive distribution to a draw from the population distribution, a heldout dataset. This method blends Bayesian modelling with frequentist assessment. Unlike the PPC, we prove that the HPC is properly calibrated. Empirically, we study HPC on classical regression, a hierarchical model of text data, and factor analysis.

Classification of T lymphocyte motility behaviors using a machine learning approach.

T lymphocytes migrate into organs and interact with local cells to perform their functions. How human T lymphocytes communicate with organ-specific cells and participate in pathobiological processes remains unresolved. Brain infiltration of T lymphocytes is associated with multiple neurological disorders. Thus, to characterize the behavior of human T lymphocytes reaching the human brain, we performed time-lapse microscopy on human CD8+ T lymphocytes co-cultured with either primary human astrocytes or neurons. Using traditional manual and visual assessment of microscopy data, we identified distinct CD8+ T lymphocyte motility behaviors. However, such characterization is time and labor-intensive. In this work, we trained and validated a machine-learning model for the automated classification of behaviors of CD8+ T lymphocytes interacting with astrocytes and neurons. A balanced random forest was trained for the binary classification of established classes of cell behaviors (synapse vs. kinapse) as well as visually identified behaviors (scanning, dancing, and poking). Feature selection was performed during 3-fold cross-validation using the minimum redundancy maximum relevance algorithm. Results show promising performances when tested on a held-out dataset of CD8+ T lymphocytes interacting with astrocytes with a new experimenter and a held-out independent dataset of CD8+ T lymphocytes interacting with neurons. When tested on the independent CD8+ T cell-neuron dataset, the final model achieved a binary classification accuracy of 0.82 and a 3-class accuracy of 0.79. This novel automated classification approach could significantly reduce the time required to label cell motility behaviors while facilitating the identification of interactions of T lymphocytes with multiple cell types.

MRI-Based Deep Learning Method for Classification of IDH Mutation Status.

Isocitrate dehydrogenase (IDH) mutation status has emerged as an important prognostic marker in gliomas. This study sought to develop deep learning networks for non-invasive IDH classification using T2w MR images while comparing their performance to a multi-contrast network. Methods: Multi-contrast brain tumor MRI and genomic data were obtained from The Cancer Imaging Archive (TCIA) and The Erasmus Glioma Database (EGD). Two separate 2D networks were developed using nnU-Net, a T2w-image-only network (T2-net) and a multi-contrast network (MC-net). Each network was separately trained using TCIA (227 subjects) or TCIA + EGD data (683 subjects combined). The networks were trained to classify IDH mutation status and implement single-label tumor segmentation simultaneously. The trained networks were tested on over 1100 held-out datasets including 360 cases from UT Southwestern Medical Center, 136 cases from New York University, 175 cases from the University of Wisconsin-Madison, 456 cases from EGD (for the TCIA-trained network), and 495 cases from the University of California, San Francisco public database. A receiver operating characteristic curve (ROC) was drawn to calculate the AUC value to determine classifier performance. Results: T2-net trained on TCIA and TCIA + EGD datasets achieved an overall accuracy of 85.4% and 87.6% with AUCs of 0.86 and 0.89, respectively. MC-net trained on TCIA and TCIA + EGD datasets achieved an overall accuracy of 91.0% and 92.8% with AUCs of 0.94 and 0.96, respectively. We developed reliable, high-performing deep learning algorithms for IDH classification using both a T2-image-only and a multi-contrast approach. The networks were tested on more than 1100 subjects from diverse databases, making this the largest study on image-based IDH classification to date.

Deep learning-based model detects atrial septal defects from electrocardiography: a cross-sectional multicenter hospital-based study

Atrial septal defect (ASD) increases the risk of adverse cardiovascular outcomes. Despite the potential for risk mitigation through minimally invasive percutaneous closure, ASD remains underdiagnosed due to subtle symptoms and examination findings. To bridge this diagnostic gap, we propose a novel screening strategy aimed at early detection and enhanced diagnosis through the implementation of a convolutional neural network (CNN) to identify ASD from 12-lead electrocardiography (ECG). ECGs were collected from patients with at least one recorded echocardiogram at 3 hospitals from 2 continents (Keio University Hospital from July 2011 to December 2020, Brigham and Women's Hospital from January 2015 to December 2020, and Dokkyo Medical University Saitama Medical Center from January 2010 and December 2021). ECGs from patients with a diagnosis of ASD were labeled as positive cases while the remainder were labeled as negative. ECGs after the closure of ASD were excluded. After randomly splitting the ECGs into 3 datasets (50% derivation, 20% validation, and 30% test) with no patient overlap, a CNN-based model was trained using the derivation datasets from 2 hospitals and was tested on held-out datasets along with an external validation on the 3rd hospital. All eligible ECGs were used for derivation and validation whereas the earliest ECG for each patient was used for the test and external validation. The discrimination of ASD was assessed by the area under the receiver operating characteristic curve (AUROC). Multiple subgroups were examined to identify any heterogeneity. A total of 671,201 ECGs from 80,947 patients were collected from the 3 institutions. The AUROC for detecting ASD was 0.85-0.90 across the 3 hospitals. The subgroup analysis showed excellent performance across various characteristics Screening simulation using the model greatly increased sensitivity from 80.6% to 93.7% at specificity 33.6% when compared to using overt ECG abnormalities. A CNN-based model using 12-lead ECG successfully identified the presence of ASD with excellent generalizability across institutions from 2 separate continents. This work was supported by research grants from JST (JPMJPF2101), JSR corporation, Taiju Life Social Welfare Foundation, Kondou Kinen Medical Foundation, Research fund of Mitsukoshi health and welfare foundation, Tokai University School of Medicine Project Research and Internal Medicine Project Research, Secom Science and Technology Foundation, and Grants from AMED (JP23hma922012 and JP23ym0126813). This work was partially supported by One Brave Idea, co-funded by the American Heart Association and Verily with significant support from AstraZeneca and pillar support from Quest Diagnostics.

Neurochemistry-enriched dynamic causal models of magnetoencephalography, using magnetic resonance spectroscopy

We present a hierarchical empirical Bayesian framework for testing hypotheses about neurotransmitters’ concertation as empirical prior for synaptic physiology using ultra-high field magnetic resonance spectroscopy (7T-MRS) and magnetoencephalography data (MEG). A first level dynamic causal modelling of cortical microcircuits is used to infer the connectivity parameters of a generative model of individuals’ neurophysiological observations. At the second level, individuals’ 7T-MRS estimates of regional neurotransmitter concentration supply empirical priors on synaptic connectivity. We compare the group-wise evidence for alternative empirical priors, defined by monotonic functions of spectroscopic estimates, on subsets of synaptic connections. For efficiency and reproducibility, we used Bayesian model reduction (BMR), parametric empirical Bayes and variational Bayesian inversion. In particular, we used Bayesian model reduction to compare alternative model evidence of how spectroscopic neurotransmitter measures inform estimates of synaptic connectivity. This identifies the subset of synaptic connections that are influenced by individual differences in neurotransmitter levels, as measured by 7T-MRS. We demonstrate the method using resting-state MEG (i.e., task-free recording) and 7T-MRS data from healthy adults. Our results confirm the hypotheses that GABA concentration influences local recurrent inhibitory intrinsic connectivity in deep and superficial cortical layers, while glutamate influences the excitatory connections between superficial and deep layers and connections from superficial to inhibitory interneurons. Using within-subject split-sampling of the MEG dataset (i.e., validation by means of a held-out dataset), we show that model comparison for hypothesis testing can be highly reliable. The method is suitable for applications with magnetoencephalography or electroencephalography, and is well-suited to reveal the mechanisms of neurological and psychiatric disorders, including responses to psychopharmacological interventions.

Abstract 1970: Protecting against algorithmic bias of AI-based clinical decision making tools in oncology

Abstract MOTIVATION: AI-based decision-making tools hold the potential to reduce patient suffering and burden. However, healthcare data reflects inequity, and datasets often lack representation of underprivileged groups. Without systems and methods to protect all patients, we risk making dangerous biases automatic and invisible. We can only ensure protection against bias if performance is monitored in production, continuously and with meaningful measures. Notably, a lack of representation and bias in the validation of diverse ancestry in genetics data results in genetic misdiagnoses and potential health disparities. Similarly, monitoring calibration in addition to receiver operating characteristic curve area under curve (ROC-AUC) in subgroups could have prevented bias predictions leading to underestimation of the health needs of sicker patients. Here, we describe methods and results implemented in an AI-based clinical decision-making tool to protect against bias. METHODS: Our model uses proven techniques and metrics to identify and monitor bias and quantify the magnitude of its impact. Our algorithm compares all demographic subgroups to that of a top performer based on overall model performance (measured by both ROC-AUC and Brier score). To determine if the top-performing subgroup is receiving statistically significant preferential treatment, we perform t-tests for each other subgroup against the top performer (p-value > 0.05 indicates that performance is statistically significant) using sample-wise performance metrics (log-loss in this case). We then quantify the magnitude of the difference by computing the ratio of the top-performer to that of the affected subgroup. The bias detection algorithm is continuously run as our model generates risk predictions based on up-to-date patient data. If the algorithm identifies a bias, it immediately notifies data scientists and engineers to begin rapidly restoring balanced model performance for clinical use by investigating algorithms that optimize for overall and balanced performance by subgroup. RESULTS: We trained and validated our machine learning model to predict 30-day emergency department visits using a data set with over 28,000 oncology patients representing diverse cancer types, races, ethnicities, ages, genders, and socio-economic statuses. Our model performance on a held-out validation data set was exceptional overall (AUC 0.8, Brier score 0.07) and across all cancer type and demographic subgroups (AUC 0.74-0.82, Brier score 0.06-0.1). We continuously monitored our live model in production over three months to ensure consistent and fair performance. Our overall and subgroup performance in production never dropped below AUC 0.75 or above Brier score 0.1. CONCLUSIONS: We demonstrate an effective framework for continuously and automatically detecting and combating algorithmic bias in oncology AI decision-making tools. Citation Format: Renee D. George, Benjamin H. Ellis, Chris J. Sidey-Gibbons, Christine Swisher. Protecting against algorithmic bias of AI-based clinical decision making tools in oncology [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 1970.

508 Deep Learning Segmentation of the Nucleus Basalis of Meynert for Deep Brain Stimulation Surgical Planning

INTRODUCTION: The nucleus basalis of Meynert (NBM) is a subcortical structure involved in arousal and cognition being explored as a deep brain stimulation (DBS) target for Alzheimer’s disease and other disorders. Given the small size of the NBM and variability between patients, accurate patient-specific targeting for DBS is needed. Accurate patient-specific manual segmentation is possible but requires high resolution 7T MRI. Therefore, the current standard is a non-patient-specific probabilistic atlas (Zaborszky 2008). METHODS: Paired 3T and 7T MRI datasets of 21 healthy subjects were obtained and the NBM was expertly segmented on 7T MRI. The 7T NBM segmentation was then used on the 3T MRI. To increase generalizability, we augmented the dataset to a total of 210 images. An external dataset of 14 patients with temporal lobe epilepsy (TLE) was used to validate the network’s performance on brains with pathological changes (Alvim 2016). A 3D-Unet convolutional neural network was constructed, and a 5-fold cross-validation was performed for model selection. The model was evaluated on healthy subjects using the held-out test dataset and on the external dataset of 14 TLE patients. RESULTS: When tested on the held-out dataset, the deep learning network demonstrated significantly improved dice coefficient compared against the probabilistic atlas for both healthy subjects (0.68 ± 0.08, 0.47 ± 0.06) and patients with TLE (0.63 ± 0.08, 0.38 ± 0.19). Additionally, the centroid distance was significantly decreased when using the network in patients with TLE (1.22 ± 0.33 mm, 3.25 ± 2.57 mm). CONCLUSIONS: We developed the first network, to our knowledge, for automatic and accurate patient-specific segmentation of the NBM using deep learning. The proposed network has less error than accepted targeting margins. This segmentation strategy allows accurate patient-specific targeting of the NBM for DBS.

ICatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.

Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.

316 Neurosurgical Federated Learning: A Multi-Center Collaboration to Detect Intracranial Hemorrhage Without Directly Sharing Data

INTRODUCTION: The development of accurate and generalizable machine learning algorithms requires sufficient quantities of diverse data. This poses a challenge in healthcare due to the sensitive and siloed nature of biomedical information. Decentralized algorithms through a federated learning (FL) paradigm avoid the need for data aggregation by instead distributing algorithms to the data itself before centrally updating one global model. METHODS: Five academic neurosurgery departments across the US collaborated to establish a federated network using computed tomography (CT) scans for prototyping. We trained a convolutional neural network to detect the presence of ICH through FL and benchmarked this against a standard, centralized training approach. Models were validated on each site’s data and tested on a held-out dataset to determine the area under the ROC curve (AUROC). RESULTS: A federated network of practicing neurosurgeon-scientists was successfully initiated and used to train a model for predicting ICH across the US. The federated model achieved an average AUROC of 0.9487 at predicting all types of ICH compared to a benchmark non-FL model AUROC of 0.9753, although performance varied by hemorrhage subtype. Subdural bleeds had the lowest AUROC (0.9257) while intraventricular bleeds had the highest (0.9751) on hold-out data. The global FL model consistently achieved top-three performance (of five) when validated on data from each site, suggesting improved generalizability. A qualitative survey of participants revealed that a majority found the process of connecting to the federated network easy. CONCLUSIONS: This study demonstrates the feasibility of implementing a federated network for multi-institutional collaboration amongst clinicians and using FL to conduct machine learning research without the transfer of data between sites, thereby opening up a new paradigm for neurosurgical collaboration.

Diagnosis of Parkinson’s disease based on 3D ResNet: The frontal lobe is crucial

Parkinson’s disease (PD) is a serious neurological disease. Many studies have preseted regions of interest such as substantia nigra (SN) for PD detection from magnetic resonance imaging (MRI). However, the SN is not the only region with remarkable tissue changes in PD MRIs. Patients with Prodromal Parkinson’s Disease usually present with non-motor symptoms, and the associated brain regions may show varying degrees of damage on imaging. Therefore, exploring PD-related regions from whole-brain MRI is essential. In this study, we proposed an interpretable PD detection framework, including PD classification and feature region visualization. Specifically, we constructed a 3D ResNet model that could detect PD from whole-brain MRIs and discover other brain regions related to PD through 3D Gradient-weighted Class Activation Mapping (Grad-CAM) and Unified Parkinson’s Disease Rating Scale (UPDRS). We obtained T1-Weighted MRIs from the Parkinson’s Progression Markers Initiative (PPMI) database. The average classification accuracy of the 5-fold cross-validation and held-out dataset reached 96.1% and 94.5%, respectively. In addition, we used the 3D Grad-CAM framework to extract the weight of the feature map and obtain visual interpretation. The heat map highlighted the regions that were crucial for PD classification and found significant differences between PD and HC in frontal lobe related to linguistic semantic disorders. The UPDRS scores of PD and HC on the linguistic semantic function items were also remarkably different. Combined with previous studies, this work verified the significance of the frontal lobe and proved that the correlation between the frontal lobe and the pathogenesis of PD was explanatory.

An Algorithm to Classify Real-World Ambulatory Status From a Wearable Device Using Multimodal and Demographically Diverse Data: Validation Study.

Measuring the amount of physical activity and its patterns using wearable sensor technology in real-world settings can provide critical insights into health status. This study's aim was to develop and evaluate the analytical validity and transdemographic generalizability of an algorithm that classifies binary ambulatory status (yes or no) on the accelerometer signal from wrist-worn biometric monitoring technology. Biometric monitoring technology algorithm validation traditionally relies on large numbers of self-reported labels or on periods of high-resolution monitoring with reference devices. We used both methods on data collected from 2 distinct studies for algorithm training and testing, one with precise ground-truth labels from a reference device (n=75) and the second with participant-reported ground-truth labels from a more diverse, larger sample (n=1691); in total, we collected data from 16.7 million 10-second epochs. We trained a neural network on a combined data set and measured performance in multiple held-out testing data sets, overall and in demographically stratified subgroups. The algorithm was accurate at classifying ambulatory status in 10-second epochs (area under the curve 0.938; 95% CI 0.921-0.958) and on daily aggregate metrics (daily mean absolute percentage error 18%; 95% CI 15%-20%) without significant performance differences across subgroups. Our algorithm can accurately classify ambulatory status with a wrist-worn device in real-world settings with generalizability across demographic subgroups. The validated algorithm can effectively quantify users' walking activity and help researchers gain insights on users' health status.

Training machine learning algorithms for automatic facial coding: The role of emotional facial expressions' prototypicality.

Automatic facial coding (AFC) is a promising new research tool to efficiently analyze emotional facial expressions. AFC is based on machine learning procedures to infer emotion categorization from facial movements (i.e., Action Units). State-of-the-art AFC accurately classifies intense and prototypical facial expressions, whereas it is less accurate for non-prototypical and less intense facial expressions. A potential reason might be that AFC is typically trained with standardized and prototypical facial expression inventories. Because AFC would be useful to analyze less prototypical research material as well, we set out to determine the role of prototypicality in the training material. We trained established machine learning algorithms either with standardized expressions from widely used research inventories or with unstandardized emotional facial expressions obtained in a typical laboratory setting and tested them on identical or cross-over material. All machine learning models' accuracies were comparable when trained and tested with held-out dataset from the same dataset (acc. = [83.4% to 92.5%]). Strikingly, we found a substantial drop in accuracies for models trained with the highly prototypical standardized dataset when tested in the unstandardized dataset (acc. = [52.8%; 69.8%]). However, when they were trained with unstandardized expressions and tested with standardized datasets, accuracies held up (acc. = [82.7%; 92.5%]). These findings demonstrate a strong impact of the training material's prototypicality on AFC's ability to classify emotional faces. Because AFC would be useful for analyzing emotional facial expressions in research or even naturalistic scenarios, future developments should include more naturalistic facial expressions for training. This approach will improve the generalizability of AFC to encode more naturalistic facial expressions and increase robustness for future applications of this promising technology.

BMI-adjusted adipose tissue volumes exhibit depot-specific and divergent associations with cardiometabolic diseases

For any given body mass index (BMI), individuals vary substantially in fat distribution, and this variation may have important implications for cardiometabolic risk. Here, we study disease associations with BMI-independent variation in visceral (VAT), abdominal subcutaneous (ASAT), and gluteofemoral (GFAT) fat depots in 40,032 individuals of the UK Biobank with body MRI. We apply deep learning models based on two-dimensional body MRI projections to enable near-perfect estimation of fat depot volumes (R2 in heldout dataset = 0.978-0.991 for VAT, ASAT, and GFAT). Next, we derive BMI-adjusted metrics for each fat depot (e.g. VAT adjusted for BMI, VATadjBMI) to quantify local adiposity burden. VATadjBMI is associated with increased risk of type 2 diabetes and coronary artery disease, ASATadjBMI is largely neutral, and GFATadjBMI is associated with reduced risk. These results – describing three metabolically distinct fat depots at scale – clarify the cardiometabolic impact of BMI-independent differences in body fat distribution.

Early prediction of hospital admission of emergency department patients.

The early prediction of hospital admission is important to ED patient management. Using available electronic data, we aimed to develop a predictive model for hospital admission. We analysed all presentations to the ED of a tertiary referral centre over 7 years. To our knowledge, our data set of nearly 600 000 presentations is the largest reported. Using demographic, clinical, socioeconomic, triage, vital signs, pathology data and keywords in electronic notes, we trained a machine learning (ML) model with presentations from 2015 to 2020 and evaluated it on a held-out data set from 2021 to mid-2022. We assessed electronic medical records (EMRs) data at patient arrival (baseline), 30, 60, 120 and 240 min after ED presentation. The training data set included 424 354 data points and the validation data set 53 403. We developed and trained a binary classifier to predict inpatient admission. On a held-out test data set of 121 258 data points, we predicted admission with 86% accuracy within 30 min of ED presentation with 94% discrimination. All models for different time points from ED presentation produced an area under the receiver operating characteristic curve (AUC) ≥0.93 for admission overall, with sensitivity/specificity/F1-scores of 0.83/0.90/0.84 for any inpatient admission at 30 min after presentation and 0.81/0.92/0.84 at baseline. The models retained lower but still high AUC levels when separated for short stay units or inpatient admissions. We combined available electronic data and ML technology to achieve excellent predictive performance for subsequent hospital admission. Such prediction may assist with patient flow.