Performance Evaluation Model Research Articles

Artificial intelligence-based cardiac transthyretin amyloidosis detection and scoring in scintigraphy imaging: multi-tracer, multi-scanner, and multi-center development and evaluation study.

Providing tools for comprehensively evaluating scintigraphy images could enhance transthyretin amyloid cardiomyopathy (ATTR-CM) diagnosis. This study aims to automatically detect and score ATTR-CM in total body scintigraphy images using deep learning on multi-tracer, multi-scanner, and multi-center datasets. In the current study, we employed six datasets (from 12 cameras) for various tasks and purposes. Dataset #1 (93 patients, 99mTc-MDP) was used to develop the 2D-planar segmentation and localization models. Dataset #2 (216 patients, 99mTc-DPD) was used for the detection (grade 0 vs. grades 1, 2, and 3) and scoring (0 and 1 vs. grades 2 and 3) of ATTR-CM. Datasets #3 (41 patients, 99mTc-HDP), #4 (53 patients, 99mTc-PYP), and #5 (129 patients, 99mTc-DPD) were used as external centers. ATTR-CM detection and scouring were performed by two physicians in each center. Moreover, Dataset #6 consisting of 3215 patients without labels, was employed for retrospective model performance evaluation. Different regions of interest were cropped and fed into the classification model for the detection and scoring of ATTR-CM. Ensembling was performed on the outputs of different models to improve their performance. Model performance was measured by classification accuracy, sensitivity, specificity, and AUC. Grad-CAM and saliency maps were generated to explain the models' decision-making process. In the internal test set, all models for detection and scoring achieved an AUC of more than 0.95 and an F1 score of more than 0.90. For detection in the external dataset, AUCs of 0.93, 0.95, and 1 were achieved for datasets 3, 4, and 5, respectively. For the scoring task, AUCs of 0.95, 0.83, and 0.96 were achieved for these datasets, respectively. In dataset #6, we found ten cases flagged as ATTR-CM by the network. Out of these, four cases were confirmed by a nuclear medicine specialist as possibly having ATTR-CM. GradCam and saliency maps showed that the deep-learning models focused on clinically relevant cardiac areas. In the current study, we developed and evaluated a fully automated pipeline to detect and score ATTR-CM using large multi-tracer, multi-scanner, and multi-center datasets, achieving high performance on total body images. This fully automated pipeline could lead to more timely and accurate diagnoses, ultimately improving patient outcomes.

Harnessing Deep Learning and Technical Indicators for Enhanced Stock Predictions of Blue-Chip Stocks on the Indonesia Stock Exchange (IDX)

Given the limitations of existing models in accurately predicting stock prices, particularly in emerging markets such as Indonesia, this study aimed to evaluate the effectiveness of deep learning models in forecasting stock prices using blue-chip company shares traded on the Indonesia Stock Exchange (IDX). The main focus lies in combining historical stock data with a series of existing technical indicators, optimizing their integration to improve prediction accuracy. The accuracy of this method is reflected in a comprehensive evaluation of model performance using robust metrics, including R2, Mean Squared Error (MSE), and Root Mean Squared Error (RMSE). Empirical results show the superiority of models integrating technical indicators compared to models relying only on historical data. The LSTM model showed the most significant improvement, with R2 for ASII stock jumping by 14.59% after incorporating technical indicators. The prediction accuracy of the GRU model for BBCA shares increased significantly, as shown by a decrease of 45.16% in MSE. These findings underscore the critical role of feature selection in developing prediction models. Integrating technical indicators with historical stock data increases prediction accuracy and provides additional tools for informed decision-making.

Radiomics-based prediction of HCC response to atezolizumab/bevacizumab.

636 Background: Advanced hepatocellular carcinoma (HCC) treatment has evolved with the introduction of atezolizumab/bevacizumab, showing improved outcomes over sorafenib. However, the response varies among patients, particularly between viral and non-viral etiologies. This study aimed to develop and evaluate multimodal prediction models combining quantitative imaging and clinical markers to predict treatment response in HCC patients. Methods: From March 2020 to May 2023, patients with advanced HCC treated with atezolizumab/bevacizumab were retrospectively identified from six centers in Germany and Austria. Patients underwent baseline contrast-enhanced liver MRI and follow-up imaging to assess therapy response. Machine learning models, including RandomForestClassifier, were developed for radiomics, clinical, and combined datasets. Hyperparameter tuning was performed using RandomizedSearchCV, followed by cross-validation to evaluate model performance. Results: The study included 103 patients, with 70 achieving disease control (DC) and 33 experiencing disease progression (PD). Key findings included significant differences in treatment response and progression-free survival between DC and PD groups. The radiomics model, using 14 selected features, achieved 73.1% accuracy and a ROC AUC of 0.635 on the test set. The clinical model, with 4 selected features, achieved 73% accuracy and a ROC AUC of 0.649 on the test set. The combined model showed improved performance with 69% accuracy and a ROC AUC of 0.753 on the test set. Hyperparameter tuning further enhanced the combined model's accuracy to 80.1% and ROC AUC to 0.771 on the test set. Conclusions: The hybrid model combining clinical and radiological data outperformed individual models, providing better predictions of response to atezolizumab/bevacizumab in HCC patients.

Automated segmentation of child-clinician speech in naturalistic clinical contexts.

Computational approaches hold significant promise for enhancing diagnosis and therapy in child and adolescent clinical practice. Clinical procedures heavily depend n vocal exchanges and interpersonal dynamics conveyed through speech. Research highlights the importance of investigating acoustic features and dyadic interactions during child development. However, observational methods are labor-intensive, time-consuming, and suffer from limited objectivity and quantification, hindering translation to everyday care. We propose a novel AI-based system for fully automatic acoustic segmentation of clinical sessions with autistic preschool children. We focused on naturalistic and unconstrained clinical contexts, which are characterized by background noise and data scarcity. Our approach addresses key challenges in the field while remaining non-invasive. We carefully evaluated model performance and flexibility in diverse, challenging conditions by means of domain alignment. Results demonstrated promising outcomes in voice activity detection and speaker diarization. Notably, minimal annotation efforts -just 30 seconds of target data- significantly improved model performance across all tested conditions. Our models exhibit satisfying predictive performance and flexibility for deployment in everyday settings. Automating data annotation in real-world clinical scenarios can enable the widespread exploitation of advanced computational methods for downstream modeling, fostering precision approaches that bridge research and clinical practice.

Do machine learning methods solve the main pitfall of linear regression in dental age estimation?

Age estimation is crucial in forensic and anthropological fields. Teeth, are valued for their resilience to environmental factors and their preservation over time, making them essential for age estimation when other skeletal remains deteriorate. Recently, Machine Learning algorithms have been used in age estimation, demonstrating high levels of accuracy. However, their precision with respect to the trend of age estimation error, typical in some traditional methods like linear regression, has not been thoroughly investigated. To evaluate and compare the performance of frequently used Machine Learning-assisted methods against two traditional age estimation methods, linear regression and the Segmented Normal Bayesian Calibration model. Overall, 1.949 orthopantomographs from black and white South African children aged 5-14 years, with 49 % males, were evaluated. The performance of Random Forest, Support Vector Regression, K-Nearest Neighbors and the Gradient Boosting Method were compared against traditional linear regression and the Segmented Normal Bayesian Calibration model. The comparison was based on accuracy measures, including Mean Absolute Error and Root Mean Squared Error, and precision measures, including the Inter-Quartile Range of the error distribution and the slope of the estimated age error relative to chronological age. The Machine Learning methods outperformed linear regression and the Segmented Normal Bayesian Calibration models in terms of accuracy, although the differences were small. Gradient Boosting Method and Support Vector Regression achieved the highest levels of accuracy (Mean Absolute Error: 0.69 years, Root Mean Squared Error: 0.85 years). All Machine Learning methods and linear regression exhibited significant bias in residuals, whereas the Segmented Normal Bayesian Calibration model showed no significant bias. Gender-stratified analyses revealed similar results in terms of the accuracy and precision of all considered models. Although Machine Learning methods demonstrate high levels of accuracy, they may be prone to trends in error distribution when estimating dental age. Evaluating this error is crucial and should be an integral part of model performance evaluation. Future research should aim to improve accuracy while rigorously addressing systematic biases.

California annual grass phenology and allometry influence ecosystem dynamics and fire regime in a vegetation demography model.

Grass-dominated ecosystems cover wide areas of the land surface yet have received far less attention from the Earth System Model (ESM) community. This limits model projections of ecosystem dynamics in response to global change and coupled vegetation-climate dynamics. We used the Functionally Assembled Terrestrial Ecosystem Simulator (FATES), a dynamic vegetation demography model, to determine ecosystem sensitivity to alternate, observed grass allometries and biophysical traits, and evaluated model performance in capturing California C3 annual grasslands structure and fire regimes. Grass allometry, leaf physiology, plant phenology, and plant mortality all drove the seasonal variation in matter and energy exchange and fire dynamics in California annual grasslands. Allometry influenced grassland structure and function mainly through canopy architecture-mediated space and light competition instead of through carbon partitioning strategy. Regional variation in grassland annual burned area was driven by variation in ecosystem productivity. Our study advances the modeling of grassy ecosystems in ESMs by establishing the importance of grass allometry and plant phenology and mortality in driving C3 annual grassland seasonal dynamics and fire regime. The calibrated annual grass allometry and biophysical traits presented can be applied in future studies to project climate-vegetation-fire feedbacks in annual grass-dominant ecosystems under global change.

Hierarchical image classification using transfer learning to improve deep learning model performance for amazon parrots

Numerous studies have proven the potential of deep learning models for classifying wildlife. Such models can reduce the workload of experts by automating species classification to monitor wild populations and global trade. Although deep learning models typically perform better with more input data, the available wildlife data are ordinarily limited, specifically for rare or endangered species. Recently, citizen science programs have helped accumulate valuable wildlife data, but such data is still not enough to achieve the best performance of deep learning models compared to benchmark datasets. Recent studies have applied the hierarchical classification of a given wildlife dataset to improve model performance and classification accuracy. This study applied hierarchical classification by transfer learning for classifying Amazon parrot species. Specifically, a hierarchy was built based on diagnostic morphological features. Upon evaluating model performance, the hierarchical model outperformed the non-hierarchical model in detecting and classifying Amazon parrots. Notably, the hierarchical model achieved the mean Average Precision (mAP) of 0.944, surpassing the mAP of 0.908 achieved by the non-hierarchical model. Moreover, the hierarchical model improved classification accuracy between morphologically similar species. The outcomes of this study may facilitate the monitoring of wild populations and the global trade of Amazon parrots for conservation purposes.

Machine-Learning Predictions of Cochlear Implant Functional Outcomes: A Systematic Review.

Cochlear implant (CI) user functional outcomes are challenging to predict because of the variability in individual anatomy, neural health, CI device characteristics, and linguistic and listening experience. Machine learning (ML) techniques are uniquely poised for this predictive challenge because they can analyze nonlinear interactions using large amounts of multidimensional data. The objective of this article is to systematically review the literature regarding ML models that predict functional CI outcomes, defined as sound perception and production. We analyze the potential strengths and weaknesses of various ML models, identify important features for favorable outcomes, and suggest potential future directions of ML applications for CI-related clinical and research purposes. We conducted a systematic literature search with Web of Science, Scopus, MEDLINE, EMBASE, CENTRAL, and CINAHL from the date of inception through September 2024. We included studies with ML models predicting a CI functional outcome, defined as those pertaining to sound perception and production, and excluded simulation studies and those involving patients without CIs. Using Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we extracted participant population, CI characteristics, ML model, and performance data. Sixteen studies examining 5058 pediatric and adult CI users (range: 4 to 2489) were included from an initial 1442 publications. Studies predicted heterogeneous outcome measures pertaining to sound production (5 studies), sound perception (12 studies), and language (2 studies). ML models use a variety of prediction features, including demographic, audiological, imaging, and subjective measures. Some studies highlighted predictors beyond traditional CI audiometric outcomes, such as anatomical and imaging characteristics (e.g., vestibulocochlear nerve area, brain regions unaffected by auditory deprivation), health system factors (e.g., wait time to referral), and patient-reported measures (e.g., dizziness and tinnitus questionnaires). Used ML models were tree-based, kernel-based, instance-based, probabilistic, or neural networks, with validation and test methods most commonly being k-fold cross-validation and train-test split. Various statistical measures were used to evaluate model performance, however, for studies reporting accuracy, the best-performing models for each study ranged from 71.0% to 98.83%. ML models demonstrate high predictive performance and illuminate factors that contribute to CI user functional outcomes. While many models showed favorable evaluation statistics, the majority were not adequately reported with regard to dataset characteristics, model creation, and validation. Furthermore, the extent of overfitting in these models is unclear and will likely result in poor generalization to new data. This suggests the need for more robust validation procedures and standardization in reporting, with the ultimate hope that the iterative improvement of these models will allow for their adoption as a future clinical tool.

Enhancing Mass Transfer Coefficient Prediction from Field Emission Scanning Electron Microscope Images Through Convolutional Neural Networks and Data Augmentation Techniques

With the growing demand for drug products requiring lyophilization, it is essential to either expand aseptic drying capacity or improve the efficiency of existing capacity through process intensification, ensuring that resources are utilized to their full potential. In this regard, mathematical models are highly recommended to assist professionals in process optimization. To effectively utilise these models, it is also essential to develop robust techniques for determining key parameters, including the product resistance to vapour flow. Traditional experimental methods for evaluating this coefficient are time-intensive and/or require the insertion of probes into the product, which is not feasible at a manufacturing scale. This study addresses these challenges by introducing a novel deep learning framework designed to predict the mass transfer coefficient directly from Field Emission Scanning Electron Microscope images. This approach significantly streamlines the evaluation process, leveraging the high-resolution capabilities of Field Emission Scanning Electron Microscope for detailed analysis. In this work, we focus on advanced Field Emission Scanning Electron Microscope image processing, choice of strategic convolutional neural network configuration, and thorough model performance evaluation to predict the mass transfer coefficient. Given the frequent scarcity of datasets in this field, we have employed data augmentation techniques to enhance the robustness of our model. The results demonstrate good predictive accuracy (error on the interpolation test data lower than 5%), highlighting the potential of this framework to facilitate the assessment of mass transfer coefficients in freeze-dried products.

A Remote Operator’s Ship Collision Avoidance Performance Evaluation Model: Comparison Between Human and AI Decisions in the Remote Operation Simulation Training

This research develops a performance evaluation model of humans in avoiding ship collision situations compared to the AI ship collision avoidance system (CAS). A human, the remote operator (RO) of Maritime Autonomous Surface Ships (MASS), ought to make compact collision avoidance (CA) decisions to secure safety and efficiency during remote operations. Hence, evaluating RO’s CA performance is important; however, research on developing evaluation methods concentrates merely on evaluating trainees based on instructor’s guidelines, while artificial intelligence (AI) makes CA decisions through parameter-based calculation. Therefore, this research proposes a CA performance evaluation model for RO of MASS based on CAS intervening of RO trainee’s simulation training in ship collision avoidance. In each time interval, the RO’s decisions showed divergent behaviors under given situational conditions compared to the CAS’s behaviors in equivalent situations. Findings denote the benefits of CAS intervention methods and RO performance evaluation model based on the proposed performance features.

Predicting Post-Wildfire Stream Temperature and Turbidity: A Machine Learning Approach in Western U.S. Watersheds

Wildfires significantly impact water quality in the Western United States, posing challenges for water resource management. However, limited research quantifies post-wildfire stream temperature and turbidity changes across diverse climatic zones. This study addresses this gap by using Random Forest (RF) and Support Vector Regression (SVR) models to predict post-wildfire stream temperature and turbidity based on climate, streamflow, and fire data from the Clackamas and Russian River Watersheds. We selected Random Forest (RF) and Support Vector Regression (SVR) because they handle non-linear, high-dimensional data, balance accuracy with efficiency, and capture complex post-wildfire stream temperature and turbidity dynamics with minimal assumptions. The primary objectives were to evaluate model performance, conduct sensitivity analyses, and project mid-21st century water quality changes under Representative Concentration Pathway (RCP) 4.5 and 8.5 scenarios. Sensitivity analyses indicated that 7-day maximum air temperature and discharge were the most influential predictors. Results show that RF outperformed SVR, achieving an R2 of 0.98 and root mean square error of 0.88 °C for stream temperature predictions. Post-wildfire turbidity increased up to 70 NTU during storm events in highly burned subwatersheds. Under RCP 8.5, stream temperatures are projected to rise by 2.2 °C by 2050. RF’s ensemble approach captured non-linear relationships effectively, while SVR excelled in high-dimensional datasets but struggled with temporal variability. These findings underscore the importance of using machine learning for understanding complex post-fire hydrology. We recommend adaptive reservoir operations and targeted riparian restoration to mitigate warming trends. This research highlights machine learning’s utility for predicting post-wildfire impacts and informing climate-resilient water management strategies.

ANP-MEAT-based evaluation of the performance of rural infrastructure provision in Mainland China

PurposeIt is a complex and dynamic process to provide high-quality rural infrastructure. However, there lacks a holistic performance evaluation method for rural infrastructure provision that reflects changing rural social needs and takes a village as a whole. This study aims to develop a holistic and dynamic performance evaluation model for rural infrastructure in Mainland China.Design/methodology/approachThis study established an evaluation index system by combining the lifecycle theory and the economy, efficiency, effectiveness and equity (4E) theory. This study developed an evaluation model by using the analytic network process (ANP) and matter-element analysis theory (MEAT). The model was validated by two representative villages in Mainland China.FindingsThe developed model can reflect dynamic social needs and effectively evaluate the overall infrastructure provision performance of a village. The weight of indicators reflects the changes in Mainland China’s contemporary rural social needs, with particular emphasis on the impact and output performance. The evaluation result shows that the overall performance of the representative villages was excellent but had a tendency toward good. Although the output performance was excellent, different input, process and impact performances resulted in different downgrade trends.Originality/valueThis study provides a theoretical basis for disaggregating the complex issue of the performance of rural infrastructure provision. The results can be used by relevant authorities to make a holistic and dynamic evaluation of the performance of rural infrastructure provision and timely revise planning and management policies.

Performance evaluation of Neural Network models applied to the classification of mineral imagery from the Katanga mining region

Nowadays, the need for a powerful model in the task of accurate image classification is increasing. This paper focuses on evaluating and comparing the effectiveness of three main neural network models namely the multilayer perceptron model, convolutional neural networks and the transfer learning model for the classification of mineral and rock images in the mining region of Katanga. The main problem addressed is the identification of the most efficient and accurate machine learning techniques in the specific classification of mineral images, with a particular focus on the constitution of the dataset, the analysis of mineral image data and their associated labels. Parameters such as the burial lot, the number of cycles, the size of convolution filters, the precision and the loss have been taken into account. The results show that the transfer learning-based model significantly outperforms the multilayer perceptron models and convolutional neural networks in terms of accuracy and robustness, achieving a classification accuracy of 97.8% compared to 75% for the multilayer perceptron and 96% for the convolutional neural networks designed from scratch. These remarkable results demonstrate the importance of deep learning in processing complex images and open new perspectives for the use of these techniques in the mining sector of the Greater Katanga mining region in the identification of mineral resources. The broader implications of this study include an innovation in mining exploration strategies through faster and more accurate classification of minerals, thus influencing both economic decision-making and environmental policies associated with mining in the region.

Predicting the risk of heart failure after acute myocardial infarction using an interpretable machine learning model

Background Early prediction of heart failure (HF) after acute myocardial infarction (AMI) is essential for personalized treatment. We aimed to use interpretable machine learning (ML) methods to develop a risk prediction model for HF in AMI patients.MethodsWe retrospectively included patients initially with AMI who received percutaneous coronary intervention (PCI) in our hospital from November 2016 to February 2020. The primary endpoint was the occurrence of HF within 3 years after operation. For developing a predictive model for HF risk in AMI patients, the least absolute shrinkage and selection operator (LASSO) Regression was used to feature selection, and four ML algorithms including Random Forest (RF), Extreme Gradient Boost (XGBoost), Support Vector Machine (SVM), and Logistic Regression (LR) were employed to develop the model on the training set. The performance evaluation of the prediction model was carried out on the training set and the testing set, utilizing metrics including AUC (Area under the receiver operating characteristic curve), calibration plot, and decision curve analysis (DCA). In addition, we used the Shapley Additive Explanations (SHAP) value to determine the importance of the selected features and interpret the optimal model.ResultsA total of 1220 AMI patients were included and 244 (20%) patients developed HF during follow-up. Among the four evaluated ML models, the XGBoost model exhibited exceptional accuracy, with an AUC value of 0.922. The SHAP method showed that left ventricular ejection fraction (LVEF), left ventricular end-systolic diameter (LVDs) and lactate dehydrogenase (LDH) were identified as the three most important characteristics to predict HF risk in AMI patients. Individual risk assessment was performed using SHAP plots and waterfall plot analysis.ConclusionsOur research demonstrates the potential of ML methods in the early prediction of HF risk in AMI patients. Furthermore, it enhances the interpretability of the XGBoost model through SHAP analysis to guide clinical decision-making.

Estimating soil strength using ultra high-resolution seismic data: A case study from a shallow water wind farm

The increasing global reliance on offshore wind farms as a sustainable energy source necessitates precise soil assessment for foundation design due to their high foundation costs and complex integration into marine environments. This study explores the use of ultrahigh-resolution seismic (UHRS) data in conjunction with cone penetration test data for soil strength estimation in offshore wind farm development. We develop a convolutional neural network-based approach that leverages UHRS data for direct soil strength estimation, aiming to address the challenges of limited cone penetration test measurements and potential overfitting in complex geologic settings. Our methodology involves preprocessing UHRS and cone penetration test data, interpreting and integrating geologic unit (GU) information, and applying repeated k-fold cross validation to evaluate model performance. The field data results demonstrate the model’s efficacy in accurately predicting cone penetration test curve trends, albeit with some amplitude discrepancies. We highlight the significance of geologic interpretation in predicting soil strength and identify the dependency on valid data within each GU as a limitation.

P0368 Novel endoluminal parameters to predict primary loss of response in patients with Crohn’s disease: a national survey and multi-centre study

Abstract Background Approximately 30% of patients with Crohn’s disease (CD) experience primary loss of response (PLR) administrated with ustekinumab (UST) [1]. Unlike the assessment of traditional biologic treatments, there is still a lack of studies that thoroughly integrate imaging parameters to evaluate the response of UST for Crohn’s disease. This study aims to precisely quantify endoluminal and intestinal wall parameters from computed tomography enterography (CTE) and predict PLR. Methods This multicentre study included 466 segments from 161 CD patients between March 2020 and May 2024. A national survey and voting were conducted to identify indicators of disease activity and PLR prediction. Ten CTE parameters were selected to evaluate changes before and after UST treatment. Logistic regression models were developed, and area under the curve (AUC), integrated discrimination improvement (IDI), net reclassification improvement (NRI), and calibration curves were used to evaluate model performance in validation cohorts. Results Ten CTE parameters, representing length, luminal narrowing, and bowel wall thickness, were defined through a national survey to characterize lesion features in IBD, including newly introduced metrics: length, area, effective luminal diameter, mean bowel wall thickness, and stenosis. A total of 352 baseline segments and 114 follow-up segments from 161 patients across six centres were analyzed to evaluate changes in CTE parameters following UST treatment. Eight endoluminal parameters—minimum diameter, maximum diameter, effective luminal diameter, area, bowel wall thickness, relative thickness, and stenosis—demonstrated statistically significant changes after UST treatment (p < 0.001), with significantly greater improvements observed in the non-PLR group compared to the PLR group (six parameters achieve p values less than 0.001). The novel parameter “effective luminal diameter” emerged as an independent predictor of UST response, demonstrating strong discriminative performance with an AUC of 0.858 and an accuracy of 0.780 in the validation cohorts. The final prediction model was: p = -3.156 + 0.776 × Diameter. Accordingly, when the Diameter at the narrowest point of the diseased bowel segment exceeds 3.351 mm, or the Area surpasses 8.819 mm², the patient is predicted to exhibit a PLR to UST. Conclusion Novel endoluminal parameters, including effective luminal diameter, provide a comprehensive characterization of intestinal lesions in IBD and exhibit robust predictive value for PLR in UST-treated CD patients. These parameters offer a practical and accessible tool for quantifying intestinal lesions, with the potential to enhance treatment strategies and improve patient outcomes.

P0788 Exploring topics of Dutch Inflammatory Bowel Disease patient messages for improving transmural communication efficiency: A natural language processing approach

Abstract Background Patients with inflammatory bowel disease (IBD) in the Netherlands included in a remote monitoring care path can use the myIBDcoach app to communicate with their healthcare providers1. Currently, IBD physicians and nurses respond to the messages without triage, leading to limited communication efficiency and increased workload. We aim to identify common topics in Dutch IBD patient messages with natural language processing (NLP) topic models to enable automated triage. Methods We analyzed 13,199 text messages from 856 Dutch IBD patients sent via myIBDcoach between September 23, 2014, and October 19, 2022. After anonymization and cleaning, the messages were split into 42,537 sentences and processed using two Dutch text-embedding models: multilingual Sentence-Transformer and BERTje.2,3 Model-generated topics were manually merged and mapped to 16 predefined labels (medical or administrative) developed with expert input (Table 1). Messages were categorized as medical if they contained at least one sentence-level medical topic; otherwise, they were categorized as administrative. To evaluate model performance, we also manually annotated a random sample of 328 sentences from 100 messages. Model performance was assessed using label-matching accuracy, defined as the percentage of matches between computer-generated labels and manual annotations. Descriptive statistics were used to analyze topic distribution and message categories. Results The model with multilingual Sentence-Transformer embeddings (average label-matching accuracy = 50%) outperformed BERTje (average label-matching accuracy = 43%) and was chosen. Excluding irrelevant information (n = 8,722) such as greetings, the five most frequently identified sentence-level topics were Test Procedure & Results (n = 5,968), Appointment/Contact (n = 5,784), Symptoms (n = 4,690), Medication (n = 4,361), and General Health Update (n = 3,472). The two most common topics were administrative. The distribution of all identified sentence-level topics can be found in Figure 1. After aggregating the sentence-level topics back to the original 13,199 messages, we found that a significant proportion (41%, n = 5,412) contained exclusively administrative topics. In contrast, 59% of the messages (n = 7,787) concerned at least one medical topic requiring specialist responses. Conclusion Using NLP, we identified the common topics in text messages from Dutch IBD patients and evaluated the performance of general Dutch topic models in this specialized context. The findings of our research provide a strong basis for developing automated triage systems to increase response efficiency and reduce provider workload in the future for digital monitoring of IBD.

Developing a rapid screening tool for high-risk ICU patients of sepsis: integrating electronic medical records with machine learning methods for mortality prediction in hospitalized patients—model establishment, internal and external validation, and visualization

ObjectivesTo develop a machine learning-based prediction model using clinical data from the first 24 h of ICU admission to enable rapid screening and early intervention for sepsis patients.MethodsThis multicenter retrospective cohort study analyzed electronic medical records of sepsis patients using machine learning methods. We evaluated model performance in predicting sepsis outcomes within the first 24 h of ICU admission across US and Chinese healthcare settings.ResultsFrom 31 clinical features, machine learning models demonstrated significantly better predictive performance than traditional approaches for sepsis outcomes. While linear regression achieved low test scores (0.25), machine learning methods reached scores of 0.78 and AUCs above 0.8 in testing. Importantly, these models maintained robust performance (scores 0.63–0.77) in external validation.ConclusionsThe application of machine learning-based prediction models for sepsis could significantly improve patient outcomes through early detection and timely intervention in the critical first 24 h of ICU admission, supporting clinical decision-making.

Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation

ObjectivesWe aimed to evaluate the performance of multiple large language models (LLMs) in data extraction from unstructured and semi-structured electronic health records.Methods50 synthetic medical notes in English, containing a structured...

A Serial MRI-based Deep Learning Model to Predict Survival in Patients with Locoregionally Advanced Nasopharyngeal Carcinoma.

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To develop and evaluate a deep learning-based prognostic model for predicting survival in locoregionally- advanced nasopharyngeal carcinoma (LA-NPC) using serial MRI before and after induction chemotherapy (IC). Materials and Methods This multicenter retrospective study included 1039 LA-NPC patients (779 male, 260 female, mean age 44 [standard deviation: 11]) diagnosed between April 2009 and December 2015. A radiomics- clinical prognostic model (Model RC) was developed using pre-and post-IC MRI and other clinical factors using graph convolutional neural networks (GCN). The concordance index (C-index) was used to evaluate model performance in predicting disease-free survival (DFS). The survival benefits of concurrent chemoradiation therapy (CCRT) were analyzed in model-defined risk groups. Results The C-indexes of Model RC for predicting DFS were significantly higher than those of TNM staging in the internal (0.79 versus 0.53) and external (0.79 versus 0.62, both P < .001) testing cohorts. The 5-year DFS for the Model RC-defined low-risk group was significantly better than that of the high-risk group (90.6% versus 58.9%, P < .001). In high-risk patients, those who received CCRT had a higher 5-year DFS rate than those who did not (58.7% versus 28.6%, P = .03). There was no evidence of a difference in 5-year DFS rate in low-risk patients who did or did not receive CCRT (91.9% versus 81.3%, P = .19). Conclusion Serial MRI before and after IC can effectively predict survival in LA-NPC. The radiomics-clinical prognostic model developed using a GCN-based deep learning method showed good risk discrimination capabilities and may facilitate risk-adaptive therapy. ©RSNA, 2025.