Hold-out Test Research Articles

Diagnosing migraine from genome-wide genotype data: a machine learning analysis.

Migraine has an assumed polygenic basis, but the genetic risk variants identified in genome-wide association studies only explain a proportion of the heritability. We aimed to develop machine learning models, capturing non-additive and interactive effects, to address the missing heritability. This was a cross-sectional population-based study of participants in the second and third Trøndelag Health Study. Individuals underwent genome-wide genotyping and were phenotyped based on validated modified criteria of the International Classification of Headache Disorders. Four datasets of increasing number of genetic variants were created using different thresholds of linkage disequilibrium and univariate genome-wide associated p-values. A series of machine learning and deep learning methods were optimized and evaluated. The genotype tools PLINK and LDPred2 were used for polygenic risk scoring. Models were trained on a partition of the dataset and tested in a hold-out set. The area under the receiver operating characteristics curve was used as the primary scoring metric. Classification by machine learning was statistically compared to that of polygenic risk scoring. Finally, we explored the biological functions of the variants unique to the machine learning approach. 43,197 individuals (51% women), with a mean age of 54.6 years, were included in the modelling. A light gradient boosting machine performed best for the three smallest datasets (108, 7,771 and 7,840 variants), all with hold-out test set area under curve at 0.63. A multinomial naïve Bayes model performed best in the largest dataset (140,467 variants) with a hold-out test set area under curve of 0.62. The models were statistically significantly superior to polygenic risk scoring (area under curve 0.52 to 0.59) for all the datasets (p<0.001 to p=0.02). Machine learning identified many of the same genes and pathways identified in genome-wide association studies, but also several unique pathways, mainly related to signal transduction and neurological function. Interestingly, pathways related to botulinum toxins, and pathways related to the calcitonin gene-related peptide receptor also emerged. This study suggests that migraine may follow a non-additive and interactive genetic causal structure, potentially best captured by complex machine learning models. Such structure may be concealed where the data dimensionality (high number of genetic variants) is insufficiently supported by the scale of available data, leaving a misleading impression of purely additive effects. Future machine learning models using substantially larger sample sizes could harness both the additive and the interactive effects, enhancing precision and offering deeper understanding of genetic interactions underlying migraine.

Prediction of IUGR condition at birth by means of CTG recordings and a ResNet model.

Prediction of IUGR condition at birth by means of CTG recordings and a ResNet model.

A Machine Learning-Based Risk Assessment Model for Poor Postoperative Pain Outcome.

Postoperative pain is a relevant and unresolved problem in clinical practice. In order to reduce the occurrence of severe postoperative pain, preventive, multi-professional and target group-specific pain management should be implemented. Risk assessment models based on machine learning and artificial intelligence are a resource-efficient way to identify the target group. The aim of this study was to develop a risk assessment model for early predicting poor postoperative pain outcomes that achieves good results without the need of additional, non-routine data collection. The various machine learning-based models were developed by using electronic medical records from over 70.000 in- and outpatient cases and 807 modelling features. The GBM (gradient boost machine) algorithm performed best with an area under the receiver operating characteristic curve (AUROC) of 0.82 on hold-out test data. Despite the excellent result, further research is needed to determine the modelt's performance in clinical practice.

Abstract 7440: Natural language processing algorithms allow retrospective glioblastoma phenotyping by identifying true wild-type isocitrate dehydrogenase gliomas in electronic health records

Abstract Background: Glioblastoma (GBM), the most common primary brain tumor in adults with a devastating median overall survival time of 12 months, was reclassified by the World Health Organization in 2021 to only include gliomas with wild-type isocitrate dehydrogenase (IDHwt). This presents a challenge in studying retrospective cohorts of patients diagnosed with outdated definitions of GBM, which may include tumors with IDH mutations. To address this, we developed and validated natural language processing-based models in electronic health record (EHR) data designed to identify patients with IDHwt gliomas. We also developed models for identifying important prognostic markers in GBM, such as the methylation status of the O6-methylguanine-DNA-methyltransferase (MGMT) promoter, to further demonstrate the utility of these tools and EHR data in studying GBM. Methods: We manually adjudicated 1, 500 pathology documents from Northwestern Medicine (NM) containing the case-insensitive word, “glioblastoma”, for the presence of a diagnosis of IDHwt glioma as well as the methylation status of MGMT. In parallel, we implemented the National Library of Medicine’s MetaMap tool on these documents to identify biomedical concepts contained within the text. These biomedical concepts were used as features in training a series of regularized logistic regression models that identify the IDH mutation and MGMT methylation status. We next constructed Kaplan-Meier survival curves, stratifying patients by MGMT methylation. Finally, we evaluated the performance of these models at Loyola University Medical Center (LUMC) to determine their efficacy at a secondary healthcare institution. Results: The best performing model in identifying IDHwt gliomas had an F1-measure of 0.88 in the holdout test set of documents from NM. Comparing patients with hyper-methylated and unmethylated MGMT showed significant differences in median overall survival times regardless of the MGMT identification method used (p &amp;lt; 0.001). Finally, the best performing IDHwt glioma identification model had an F1-measure of 0.96 when implemented at LUMC. 15% of patients identified at LUMC were Hispanic or Latino, compared to only 5% at NM (p &amp;lt; 0.05). Discussion: Our results suggest that we successfully designed and implemented models to identify patients with true IDHwt gliomas. Importantly, these models displayed excellent performance at a secondary healthcare institution, demonstrating that they can be utilized to identify ethnically diverse and multi-site GBM cohorts. Furthermore, our characterization of the NM GBM cohort recapitulated known trends in survival patterns, serving as a secondary validation metric and demonstrating the utility of EHR data in studying GBM in clinical settings. Citation Format: Vijeeth Guggilla, Noah Forrest, April Bell, Khizar Nandoliya, Ethan Houskamp, Steven Tran, Rimas Lukas, Jodi Johnson, Ishan Roy, Derek Wainwright, Theresa Walunas. Natural language processing algorithms allow retrospective glioblastoma phenotyping by identifying true wild-type isocitrate dehydrogenase gliomas in electronic health records [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7440.

Abstract 1063: CellMap: a comprehensive human single cell gene expression reference for automated cell annotation and cancer cell-of-origin analysis

Abstract In single-cell transcriptomic analysis, accurate cell type annotation forms the essential basis for all downstream analysis and data interpretation. While time consuming manual annotation requires extensive prior knowledge, available automated cell annotation tools lack a unified, curated human cell reference to ensure successful identification of all cell types present in any given dataset. To fill in the gap we create “CellMap” a comprehensive single cell gene expression reference database of known cell types across most human tissues by compiling, curating and integrating single cell datasets from multiple sources including Human Protein Atlas, Tabula Sapiens, ArrayExpress and GEO. CellMap contains 151 different celltypes/states including 94 epithelial, 31 brain, 4 from soft tissues, 8 stromal and 15 major immune celltypes from 42 human tissues. Projecting pseudo-bulk profiles of the 151 celltypes into 2-dimensional UMAP revealed major clusters of cells from the nervous system, respiratory system, gastrointestinal tract, proximal digestive system, female tissues and endocrine tissues. Immune cells and stromal cells are generally clustered together regardless of their tissue origin. We also curated about 200 immune cell phenotypes from the ImmGen database for fine-tuning annotation of immune cell subpopulations. Importantly by combining CellMap and the SingleR algorithm, we successfully identified the cell lineage of the tumor and minor cell types originated from the biopsy site of a breast cancer metastatic sample; the latter were left unannotated with reference derived from breast tissue or misclassified using Human Primary Cell Atlas. In addition, CellMap allowed us to identify highly specific tissue markers and cell type markers. Based on 1241 identified marker genes, we trained a random forest model to predict cancer types using 80% of the TCGA tumor samples of each cancer type (n = 7160) and tested on a 20% hold-out testing set (n = 1774), achieving a median of 95.4% accuracy across cancer types. Cancer types with low accuracy include esophageal carcinoma (mostly mis-classified as head and neck squamous cell carcinoma) and rectum adenocarcinoma (mostly mis-classified as colon adenocarcinoma). The misclassification could be explained by the high similarity between these cancer types in terms of cell-of-origin. Finally a webportal for exploring celltypes in CellMap and an open-source R-package for automated cell annotation of single cell data and cell lineage prediction for bulk RNA-seq data were developed. This work provides valuable resources for conducting single-cell analysis and cancer cell-of-origin analysis. Additionally, it helps identify closely related cell types across tissue types, which is particularly useful since diseases originating from related cell types may share similar therapeutic targets. Citation Format: Yuping Zhang, Gabriel Cruz, Hanbyul Cho, Chandan Kumar-Sinha, Rahul Mannan, Jin Chen, Xuhong Cao, Saravana M. Dhanasekaran, Arul M. Chinnaiyan. CellMap: a comprehensive human single cell gene expression reference for automated cell annotation and cancer cell-of-origin analysis [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 1063.

Automating the Addiction Behaviors Checklist for Problematic Opioid Use Identification

Individuals whose chronic pain is managed with opioids are at high risk of developing an opioid use disorder. Electronic health records (EHR) allow large-scale studies to identify a continuum of problematic opioid use, including opioid use disorder. Traditionally, this is done through diagnostic codes, which are often unreliable and underused. To determine whether regular expressions, an interpretable natural language processing technique, could automate a validated clinical tool (Addiction Behaviors Checklist) to identify problematic opioid use. This cross-sectional study reports on a retrospective cohort with data analyzed from 2021 through 2023. The approach was evaluated against a blinded, manually reviewed holdout test set and validated against an independent test set at a separate institution. The study used data from Vanderbilt University Medical Center's Synthetic Derivative, a deidentified version of the EHR for research purposes. This cohort comprised 8063 individuals with chronic pain, defined by diagnostic codes on at least 2 days. The study team collected free-text notes, demographics, and diagnostic codes and performed an external validation with 100 individuals with chronic pain from Geisinger, recruited from an interventional pain clinic cohort. The primary outcome was the evaluation of the automated method in identifying patients demonstrating problematic opioid use and its comparison with manual medical record review and opioid use disorder diagnostic codes. Methods with F1 scores were evaluated (a single value that combines sensitivity and positive predictive value at a single threshold) and areas under the curve (a single value that combines sensitivity and specificity across multiple thresholds). Among the 8063 patients in the primary site (5081 female [63%] and 2982 male [37%]; mean [SD] age, 56 [16] years) and 100 patients in the validation site (57 female [57%] and 43 male [43%]; mean [SD] age, 54 [13] years), the automated approach outperformed diagnostic codes based on F1 scores (0.73; 95% CI, 0.62-0.83 vs 0.08; 95% CI, 0.00-0.19 at the primary site and 0.70; 95% CI, 0.50-0.85 vs 0.29; 95% CI, 0.07-0.50 at the validation site) and areas under the curve (0.82; 95% CI, 0.73-0.89 vs 0.52; 95% CI, 0.50-0.55 at the primary site and 0.86; 95% CI, 0.76-0.94 vs 0.59;95% CI, 0.50-0.67 at validation site). This automated data extraction technique may facilitate earlier identification of people at risk for and who are experiencing problematic opioid use, and create new opportunities for studying long-term sequelae of opioid pain management.

Tumor Bud Classification in Colorectal Cancer Using Attention-Based Deep Multiple Instance Learning and Domain-Specific Foundation Models.

Identifying tumor budding (TB) in colorectal cancer (CRC) is vital for better prognostic assessment as it may signify the initial stage of metastasis. Despite its importance, TB detection remains challenging due to subjectivity in manual evaluations. Identifying TBs remains difficult, especially at high magnification levels, leading to inconsistencies in prognosis. To address these issues, we propose an automated system for TB classification using deep learning. We trained a deep learning model to identify TBs through weakly supervised learning by aggregating positive and negative bags from the tumor invasive front. We assessed various foundation models for feature extraction and compared their performance. Attention heatmaps generated by attention-based multi-instance learning (ABMIL) were analyzed to verify alignment with TBs, providing insights into the interpretability of the features. The dataset includes 29 WSIs for training and 70 whole slide images (WSIs) for the hold-out test set. In six-fold cross-validation, Phikon-v2 achieved the highest average AUC (0.984 ± 0.003), precision (0.876 ± 0.004), and recall (0.947 ± 0.009). Phikon-v2 again achieved the highest AUC (0.979) and precision (0.980) on the external hold-out test set. Moreover, its recall rate (0.910) was still higher than that of UNI's (0.879). UNI exhibited a balanced performance on the hold-out test set, with an AUC of 0.960 and a precision of 0.968. CtransPath showed strong precision on the external hold-out test set (0.947) but had a slightly lower recall (0.911). The proposed technique enhances the accuracy of TB assessment, offering potential applications for CRC and other cancer types.

An artificial-intelligence interpretable tool to predict risk of deep vein thrombosis after endovenous thermal ablation.

An artificial-intelligence interpretable tool to predict risk of deep vein thrombosis after endovenous thermal ablation.

Assessing severe pneumonia risk in children via clinical prognostic model based on laboratory markers.

Assessing severe pneumonia risk in children via clinical prognostic model based on laboratory markers.

Benchmarking domain-specific pretrained language models to identify the best model for methodological rigor in clinical studies.

Benchmarking domain-specific pretrained language models to identify the best model for methodological rigor in clinical studies.

Prehospital triage of trauma patients: predicting major surgery using artificial intelligence as decision support.

Matching the necessary resources and facilities to attend to the needs of trauma patients is traditionally performed by clinicians using criteria-directed triage protocols. In the present study, it was hypothesized that an artificial intelligence (AI) model should be able to predict the need for major surgery based on data available at the scene. Prehospital and in-hospital electronic health record data were available for 4578 patients in the Danish Prehospital Trauma Data set. Data included demographics (age and sex), clinical scores (airway, breathing, circulation, disability (ABCD) and Glasgow Coma Scale scores), and sequential vital signs (heart rate, blood pressure, and oxygen saturation). The data from the first 5, 10, and 20 min of prehospital contact were used for predicting the need for surgery up to 12 h after hospital arrival. Surgeries were stratified into all major surgical procedures and specialty-specific procedures (neurosurgery, abdominal surgery, and vascular surgery). The data set was split into training (70%), validation (20%) and holdout test (10%) data sets. Three hybrid neural networks were trained and performance was evaluated on the holdout test data set using the area under the receiver operating characteristic curve (ROC-AUC). Overall, the model achieved an ROC-AUC of 0.80-0.86 for predicting the need for major surgery. For predicting the need for major neurosurgery the ROC-AUC was 0.90-0.95, for predicting the need for major vascular surgery the ROC-AUC was 0.69-0.88, and for predicting the need for major abdominal surgery the ROC-AUC was 0.77-0.84. Utilizing AI early in the prehospital phase of a trauma patient's trajectory can predict specialized surgical needs. This approach has the potential to aid the early triage of trauma patients.

Epileptic seizure detection using heart rate variability from ambulatory ECG: a pseudoprospective study

Objective.Seizure detection algorithms enable clinicians to accurately assess seizure burden for epilepsy diagnosis and long-term management. State-of-the-art algorithms rely on electroencephalography (EEG) data to identify electrographic seizures. Previous research that used non-EEG signals, such as electrocardiography (ECG) and wristband data, were collected in epilepsy monitoring units. We aimed to investigate the feasibility of ECG seizure detection in ambulatory settings.Approach.We developed a patient-independent, machine learning-based seizure detector using ambulatory long-term ECG monitoring data. The model was trained on long-term studies of 47 patients and evaluated pseudoprospectively using event detection on a hold-out test set of 18 patients.Main results.In the hold-out test set, the seizure detector performed better than chance for 14 out of 18 patients. The average sensitivity was 72% and the average specificity was 68% for the whole test cohort. Overall, across training and test sets, the performance was better for patients diagnosed with focal epilepsy and for patients who were identified as responders (had substantial heart rate changes during seizures).Significance.Key contributions of this study include the development of a patient-independent seizure detector using ambulatory data and the introduction of a pseudoprospective evaluation framework, which can benefit chronic ambulatory seizure monitoring.

Automatic sequence identification in multicentric prostate multiparametric MRI datasets for clinical machine-learning

ObjectivesTo present an accurate machine-learning (ML) method and knowledge-based heuristics for automatic sequence-type identification in multi-centric multiparametric MRI (mpMRI) datasets for prostate cancer (PCa) ML.MethodsRetrospective prostate mpMRI studies were classified into 5 series types—T2-weighted (T2W), diffusion-weighted images (DWI), apparent diffusion coefficients (ADC), dynamic contrast-enhanced (DCE) and other series types (others). Metadata was processed for all series and two models were trained (XGBoost after custom categorical tokenization and CatBoost with raw categorical data) using 5-fold cross-validation (CV) with different data fractions for learning curve analyses. For validation, two test sets—hold-out test set and temporal split—were used. A leave-one-group-out (LOGO) CV analysis was performed with centres as groups to understand the effect of dataset-specific data.Results4045 studies (31,053 series) and 1004 studies (7891 series) from 11 centres were used to train and test series identification models, respectively. Test F1-scores were consistently above 0.95 (CatBoost) and 0.97 (XGBoost). Learning curves demonstrate learning saturation, while temporal validation shows model remain capable of correctly identifying all T2W/DWI/ADC triplets. However, optimal performance requires centre-specific data—controlling for model and used feature sets when comparing CV with LOGOCV, F1-score dropped for T2W, DCE and others (−0.146, −0.181 and −0.179, respectively), with larger performance decreases for CatBoost (−0.265). Finally, we delineate heuristics to assist researchers in series classification for PCa mpMRI datasets.ConclusionsAutomatic series-type identification is feasible and can enable automated data curation. However, dataset-specific data should be included to achieve optimal performance.Critical relevance statementOrganising large collections of data is time-consuming but necessary to train clinical machine-learning models. To address this, we outline and validate an automatic series identification method that can facilitate this process. Finally, we outline a set of metadata-based heuristics that can be used to further automate series-type identification.Key PointsMulti-centric prostate MRI studies were used for sequence annotation model training.Automatic sequence annotation requires few instances and generalises temporally.Sequence annotation, necessary for clinical AI model training, can be performed automatically.Graphical

Assessing Cancer Presence in Prostate MRI Using Multi-Encoder Cross-Attention Networks.

Prostate cancer (PCa) is currently the second most prevalent cancer among men. Accurate diagnosis of PCa can provide effective treatment for patients and reduce mortality. Previous works have merely focused on either lesion detection or lesion classification of PCa from magnetic resonance imaging (MRI). In this work we focus on a critical, yet underexplored task of the PCa clinical workflow: distinguishing cases with cancer presence (pathologically confirmed PCa patients) from conditions with no suspicious PCa findings (no cancer presence). To this end, we conduct large-scale experiments for this task for the first time by adopting and processing the multi-centric ProstateNET Imaging Archive which contains more than 6 million image representations of PCa from more than 11,000 PCa cases, representing the largest collection of PCa MR images. Bi-parametric MR (bpMRI) images of 4504 patients alongside their clinical variables are used for training, while the architectures are evaluated on two hold-out test sets of 975 retrospective and 435 prospective patients. Our proposed multi-encoder-cross-attention-fusion architecture achieved a promising area under the receiver operating characteristic curve (AUC) of 0.91. This demonstrates our method's capability of fusing complex bi-parametric imaging modalities and enhancing model robustness, paving the way towards the clinical adoption of deep learning models for accurately determining the presence of PCa across patient populations.

The application of decision trees for estimating osteological sex from common measurements of the skull.

Skull measurements are commonly evaluated for osteological sex estimation in forensic anthropology, and decision tree-based classification models for the skull may improve accuracy compared to current metric methods. Additionally, decision trees can provide accurate sex classification with a limited number of measurements, which is valuable when analyzing fragmentary remains. Thus, the present study seeks to test the utility of decision trees for generating sex classification models from metric variables of the skull. Twenty-one skull measurements were evaluated for 403 adult males and females. Relative technical error of measurement was used to assess intraobserver error, and two-way ANOVAs and aligned rank transformation were used to examine the effects of sex, population, age, and temporal period on the measurements. The data set was split into 80% training and 20% holdout testing samples to assess the predictive accuracy of each tree. Trees were generated for the skull and cranium, with models for European Americans, African Americans, and the pooled population sample. Overall, the recommended trees for the cranium achieved higher accuracies (85.3-95.0%) compared to the skull trees (84.0-92.5%). Accuracies for the population-inclusive trees ranged from 84.0% to 85.3%, whereas the European American (92.5-95.0%) and African American (90.9%) trees achieved slightly higher accuracies. Improved accuracies were achieved compared to previous decision tree research as well as compared to current metric methods for the skull. These trees provide an additional option for estimating osteological sex, particularly when morphological methods do not yield adequate classification accuracies or cannot be assessed due to damage.

28 Using AI to predict molecular subtype from histopathology slides in endometrial cancer

Objectives/Goals: Endometrial cancer is one of the few cancers that has both a rising incidence and mortality rate. Molecular classification is becoming more important for the management of endometrial cancer but the ability to translate this into clinical practice remains constrained. Our goal is to use AI to predict the molecular subtype from histopathology slides. Methods/Study Population: We utilized the open source endometrial cancer datasets from The Cancer Genome Atlas (TCGA) (N = 387) and Cancer Proteomics Transcriptomic Tumor Analysis Consortium (CPTAC) (N = 135) to develop and train a vision transformer AI model. We used a proprietary cohort of patients (N = 548) for external validation. Whole slide images (WSI) and molecular subtype data were collected. Subtypes include POLE ultramutated (POLE), microsatellite instability (MSI-H), copy-number low (CNV-L), and copy-number high (CNV-H). WSI were preprocessed, and features were extracted. Modified STAMP protocol was used in training, utilizing a pretrained foundation transformer model (Virchow2). Cross-validation of the TCGA was used for initial training, followed by testing on the CPTAC dataset and validation on our proprietary cohort. Results/Anticipated Results: Fivefold cross-validation of the TCGA database (60% training, 20% testing, and 20% validation) developed a best overall model with a mean AUC of 0.74 (POLE 0.78, MSI-H 0.76, CNV-H 0.86, CNV-L 0.77). Overall precision 0.58, recall 0.55. CNV-H was the subtype with the most accurate prediction. CPTAC holdout testing revealed moderately high AUC (POLE 0.63, MSI-H 0.62, CNV-H 0.98, and CNV-L 0.76). Overall precision 0.54 and recall 0.58. Again, CNV-H was the most accurate prediction. Validation on our proprietary cohort revealed a drop in performance with overall mixed results by AUC (POLE 0.50, MSI-H 0.69, CNV-H 0.78, and CNV-L 0.61). Overall precision 0.57, recall 0.45. Again, CNV-H with the most accurate prediction but F1 score dropped from 0.77 in the CPTAC to 0.47 on validation. POLE was the least accurate prediction subtype. Discussion/Significance of Impact: The CNV-H subtype demonstrated robust performance, suggesting the model effectively captures the features associated with this subtype. CNV-L had moderate performance. MSI-H and POLE were notably lower. WSI-based AI models show translational potential for subtype prediction in the management of endometrial cancer but more work is necessary.

Student dropout prediction through machine learning optimization: insights from moodle log data

Student attrition and academic failure remain pervasive challenges in education, often occurring at substantial rates and posing considerable difficulties for timely identification and intervention. Learning management systems such as Moodle generate extensive datasets reflecting student interactions and enrollment patterns, presenting opportunities for predictive analytics. This study seeks to advance the field of dropout and failure prediction through the application of artificial intelligence with machine learning methodologies. In particular, we employed the CatBoost algorithm, trained on student activity logs from the Moodle platform. To mitigate the challenges posed by a limited and imbalanced dataset, we employed sophisticated data balancing techniques, such as Adaptive Synthetic Sampling, and conducted multi-objective hyperparameter optimization using the Non-dominated Sorting Genetic Algorithm II. We compared models trained on weekly log data against a single model trained on all weeks’ data. The proposed model trained with all weeks’ data demonstrated superior performance, showing significant improvements in F1-scores and recall, particularly for the minority class of at-risk students. For example, the model got an average F1-score across multiple weeks of approximately 0.8 in the holdout test. These findings underscore the potential of targeted machine learning approaches to facilitate early identification of at-risk students, thereby enabling timely interventions and improving educational outcomes.

Impact of hospital-specific domain adaptation on BERT-based models to classify neuroradiology reports.

To determine the effectiveness of hospital-specific domain adaptation through masked language modelling (MLM) on BERT-based models' performance in classifying neuroradiology reports, and to compare these models with open-source large language models (LLMs). This retrospective study (2008-2019) utilised 126,556 and 86,032 MRI brain reports from two tertiary hospitals-King's College Hospital (KCH) and Guys and St Thomas' Trust (GSTT). Various BERT-based models, including RoBERTa, BioBERT and RadBERT, underwent MLM on unlabelled reports from these centres. The downstream tasks were binary abnormality classification and multi-label classification. Performances of models with and without hospital-specific domain adaptation were compared against each other and LLMs on internal (KCH) and external (GSTT) hold-out test sets. Model performances for binary classification were compared using 2-way and 1-way ANOVA. All models that underwent hospital-specific domain adaptation performed better than their baseline counterparts (all p-values < 0.001). For binary classification, MLM on all available unlabelled reports (194,467 reports) yielded the highest balanced accuracies (KCH: mean 97.0 ± 0.4% (standard deviation), GSTT: 95.5 ± 1.0%), after which no differences between BERT-based models remained (1-way ANOVA, p-values > 0.05). There was a log-linear relationship between the number of reports and performance. LLama-3.0 70B was the best-performing LLM (KCH: 97.1%, GSTT: 94.0%). Multi-label classification demonstrated consistent performance improvements from MLM for all abnormality categories. Hospital-specific domain adaptation should be considered best practice when deploying BERT-based models in new clinical settings. When labelled data is scarce or unavailable, LLMs can serve as a viable alternative, assuming adequate computational power is accessible. Question BERT-based models can classify radiology reports, but it is unclear if there is any incremental benefit from additional hospital-specific domain adaptation. Findings Hospital-specific domain adaptation resulted in the highest BERT-based model accuracies and performance scaled log-linearly with the number of reports. Clinical relevance BERT-based models after hospital-specific domain adaptation achieve the best classification results provided sufficient high-quality training labels. When labelled data is scarce, LLMs such as Llama-3.0 70B are a viable alternative provided there are sufficient computational resources.

A Multiple Instance Learning Approach to Study Leaf Wilt in Soybean Plants

Recent years have seen significant technological advancements in precision farming and plant phenotyping. Remote sensing along with deep learning (DL) techniques can increase phenotyping efficiency and help on-farm decision making with rapid stress detection. In this work, we use these techniques to evaluate drought stress in soybean plants, a crop whose yield is significantly affected by water availability. Images were taken from a high vantage in the field at various times throughout the day. Each image is given a wilting score ranging from 0 to 4 by expert scorers. We implement a DL method called multiple instance learning (MIL) to perform wilt classification as well as generate heat maps that highlight wilt levels in specific regions of the image. Given the significant overlap between adjacent classes in our dataset, we were able to achieve an overall classification accuracy of 64% and a one-off accuracy of 96% on our holdout test set. Our model outperformed DenseNet121 in most metrics, and provided comparable performance to a vision transformer (ViT) while having fewer parameters overall, less complexity (useful for edge implementations), and some interpretability. Furthermore, we were able to show that our model outperformed expert human annotators by predicting more consistent and accurate wilt levels when considering single-image re-annotation. The results show that our proposed methodology can be a useful approach in detecting drought stress in soybean fields to facilitate efficient crop management and aid selection of drought-resilient varieties.

Preterm Birth Prediction Using an Explainable Machine Learning Approach

Pregnancy and childbirth are critical periods, with complications like preterm birth presenting serious risks to both mother and baby. Preterm birth is defined as delivery before 37 weeks of gestation. It poses serious health risks to newborns, including respiratory complications, developmental delays, and long-term disabilities. Early prediction of preterm birth can enable timely interventions to reduce these risks and improve maternal-fetal health outcomes. This paper proposes a machine learning-based framework for preterm birth prediction using a blended approach. This approach combines multiple classifiers into blended model to enhance the prediction accuracy. Feature selection techniques such as variance threshold, Pearson’s correlation, and mutual information are applied exclusively to the training set to enhance model performance. The proposed blended model consistently outperforms the standalone base models. It achieves the highest accuracy of 74.61%, precision of 71.83%, recall of 70.94%, F1 score of 71.38%, and an area under the curve (AUC) of 84.56% during 5-fold cross-validation. For holdout testing, it also maintains a superior performance with an accuracy of 73.82% and AUC of 83.91%. Additionally, explainable artificial intelligence techniques are applied to interpret the model’s predictions. It identifies that abnormal amniotic fluid, prenatal care, and household income have a substantial impact on predicting both preterm and normal birth outcomes. The proposed approach shows strong potential for real-world healthcare applications to offer clinicians valuable insights for early preterm birth intervention. Received: 10 October 2024 | Revised: 9 December 2024 | Accepted: 18 February 2025 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in Midwifery at https://doi.org/10.1016/j.midw.2020.102670, reference number [40]. Author Contribution Statement Ahmad Hassan: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review &amp; editing, Visualization, Supervision, Project administration. Saba Nawaz: Validation, Formal analysis, Investigation, Resources, Writing – original draft, Writing – review &amp; editing, Project administration. Sidra Tahira: Validation, Investigation, Data curation, Writing – review &amp; editing, Visualization, Project administration. Arslan Ahmed: Investigation, Data curation, Writing – review &amp; editing, Supervision, Project administration.