L1-regularized Logistic Regression Research Articles

Predictive Models for Sustained, Uncontrolled Hypertension and Hypertensive Crisis Based on Electronic Health Record Data: Algorithm Development and Validation.

Assessing disease progression among patients with uncontrolled hypertension is important for identifying opportunities for intervention. We aim to develop and validate 2 models, one to predict sustained, uncontrolled hypertension (≥2 blood pressure [BP] readings ≥140/90 mm Hg or ≥1 BP reading ≥180/120 mm Hg) and one to predict hypertensive crisis (≥1 BP reading ≥180/120 mm Hg) within 1 year of an index visit (outpatient or ambulatory encounter in which an uncontrolled BP reading was recorded). Data from 142,897 patients with uncontrolled hypertension within Atrium Health Greater Charlotte in 2018 were used. Electronic health record-based predictors were based on the 1-year period before a patient's index visit. The dataset was randomly split (80:20) into a training set and a validation set. In total, 4 machine learning frameworks were considered: L2-regularized logistic regression, multilayer perceptron, gradient boosting machines, and random forest. Model selection was performed with 10-fold cross-validation. The final models were assessed on discrimination (C-statistic), calibration (eg, integrated calibration index), and net benefit (with decision curve analysis). Additionally, internal-external cross-validation was performed at the county level to assess performance with new populations and summarized using random-effect meta-analyses. In internal validation, the C-statistic and integrated calibration index were 0.72 (95% CI 0.71-0.72) and 0.015 (95% CI 0.012-0.020) for the sustained, uncontrolled hypertension model, and 0.81 (95% CI 0.79-0.82) and 0.009 (95% CI 0.007-0.011) for the hypertensive crisis model. The models had higher net benefit than the default policies (ie, treat-all and treat-none) across different decision thresholds. In internal-external cross-validation, the pooled performance was consistent with internal validation results; in particular, the pooled C-statistics were 0.70 (95% CI 0.69-0.71) and 0.79 (95% CI 0.78-0.81) for the sustained, uncontrolled hypertension model and hypertensive crisis model, respectively. An electronic health record-based model predicted hypertensive crisis reasonably well in internal and internal-external validations. The model can potentially be used to support population health surveillance and hypertension management. Further studies are needed to improve the ability to predict sustained, uncontrolled hypertension.

Investigation of bias in the automated assessment of school violence

ObjectivesNatural language processing and machine learning have the potential to lead to biased predictions. We designed a novel Automated RIsk Assessment (ARIA) machine learning algorithm that assesses risk of violence and aggression in adolescents using natural language processing of transcribed student interviews. This work evaluated the possible sources of bias in the study design and the algorithm, tested how much of a prediction was explained by demographic covariates, and investigated the misclassifications based on demographic variables. MethodsWe recruited students 10–18 years of age and enrolled in middle or high schools in Ohio, Kentucky, Indiana, and Tennessee. The reference standard outcome was determined by a forensic psychiatrist as either a “high” or “low” risk level. ARIA used L2-regularized logistic regression to predict a risk level for each student using contextual and semantic features. We conducted three analyses: a PROBAST analysis of risk in study design; analysis of demographic variables as covariates; and a prediction analysis. Covariates were included in the linear regression analyses and comprised of race, sex, ethnicity, household education, annual household income, age at the time of visit, and utilization of public assistance. ResultsWe recruited 412 students from 204 schools. ARIA performed with an AUC of 0.92, sensitivity of 71%, NPV of 77%, and specificity of 95%. Of these, 387 students with complete demographic information were included in the analysis. Individual linear regressions resulted in a coefficient of determination less than 0.08 across all demographic variables. When using all demographic variables to predict ARIA’s risk assessment score, the multiple linear regression model resulted in a coefficient of determination of 0.189. ARIA performed with a lower False Negative Rate (FNR) of 15.2% (CI [0 – 40]) for the Black subgroup and 12.7%, CI [0 – 41.4] for Other races, compared to an FNR of 26.1% (CI [14.1 – 41.8]) in the White subgroup. ConclusionsBias assessment is needed to address shortcomings within machine learning. In our work, student race, ethnicity, sex, use of public assistance, and annual household income did not explain ARIA’s risk assessment score of students. ARIA will continue to be evaluated regularly with increased subject recruitment.

Federated dual averaging learning algorithm with delayed gradients for composite optimization

Federated learning offers a promising paradigm for training machine learning models in distributed edge networks. This paper focuses on a critical aspect of federated learning, specifically examining the influence of communication delays between the server and edge devices. We study the problem of federated composite optimization (FCO) in the scenario where communication delays are introduced due to client-sides. To tackle this issue, we propose a federated dual averaging learning algorithm that incorporates delayed gradients. We conduct a thorough convergence analysis of our algorithm taking into account the impact of the delayed gradient information. Our analysis demonstrates that under the assumption of bounded expected random delays, the algorithm exhibits the asymptotic convergence towards the optimal value. Furthermore, we present explicit convergence rates of the proposed algorithm for fixed and random delays. Our results show that the impact of delays on FCO problems with smooth loss functions gradually becomes negligible. Finally, to demonstrate the efficacy of the proposed algorithm, we conduct numerical simulations on the FEMNIST dataset utilizing ℓ1-regularized logistic regression. These experimental results confirm our findings that the algorithm converges effectively even in the presence of delays.

Predictive Model for Extended-Spectrum β-Lactamase-Producing Bacterial Infections Using Natural Language Processing Technique and Open Data in Intensive Care Unit Environment: Retrospective Observational Study.

Machine learning has advanced medical event prediction, mostly using private data. The public MIMIC-3 (Medical Information Mart for Intensive Care III) data set, which contains detailed data on over 40,000 intensive care unit patients, stands out as it can help develop better models including structured and textual data. This study aimed to build and test a machine learning model using the MIMIC-3 data set to determine the effectiveness of information extracted from electronic medical record text using a named entity recognition, specifically QuickUMLS, for predicting important medical events. Using the prediction of extended-spectrum β-lactamase (ESBL)-producing bacterial infections as an example, this study shows how open data sources and simple technology can be useful for making clinically meaningful predictions. The MIMIC-3 data set, including demographics, vital signs, laboratory results, and textual data, such as discharge summaries, was used. This study specifically targeted patients diagnosed with Klebsiella pneumoniae or Escherichia coli infection. Predictions were based on ESBL-producing bacterial standards and the minimum inhibitory concentration criteria. Both the structured data and extracted patient histories were used as predictors. In total, 2 models, an L1-regularized logistic regression model and a LightGBM model, were evaluated using the receiver operating characteristic area under the curve (ROC-AUC) and the precision-recall curve area under the curve (PR-AUC). Of 46,520 MIMIC-3 patients, 4046 were identified with bacterial cultures, indicating the presence of K pneumoniae or E coli. After excluding patients who lacked discharge summary text, 3614 patients remained. The L1-penalized model, with variables from only the structured data, displayed a ROC-AUC of 0.646 and a PR-AUC of 0.307. The LightGBM model, combining structured and textual data, achieved a ROC-AUC of 0.707 and a PR-AUC of 0.369. Key contributors to the LightGBM model included patient age, duration since hospital admission, and specific medical history such as diabetes. The structured data-based model showed improved performance compared to the reference models. Performance was further improved when textual medical history was included. Compared to other models predicting drug-resistant bacteria, the results of this study ranked in the middle. Some misidentifications, potentially due to the limitations of QuickUMLS, may have affected the accuracy of the model. This study successfully developed a predictive model for ESBL-producing bacterial infections using the MIMIC-3 data set, yielding results consistent with existing literature. This model stands out for its transparency and reliance on open data and open-named entity recognition technology. The performance of the model was enhanced using textual information. With advancements in natural language processing tools such as BERT and GPT, the extraction of medical data from text holds substantial potential for future model optimization.

A Study on the Object-Based High-Resolution Remote Sensing Image Classification of Crop Planting Structures in the Loess Plateau of Eastern Gansu Province

The timely and accurate acquisition of information on the distribution of the crop planting structure in the Loess Plateau of eastern Gansu Province, one of the most important agricultural areas in Western China, is crucial for promoting fine management of agriculture and ensuring food security. This study uses multi-temporal high-resolution remote sensing images to determine optimal segmentation scales for various crops, employing the estimation of scale parameter 2 (ESP2) tool and the Ratio of Mean Absolute Deviation to Standard Deviation (RMAS) model. The Canny edge detection algorithm is then applied for multi-scale image segmentation. By incorporating crop phenological factors and using the L1-regularized logistic regression model, we optimized 39 spatial feature factors—including spectral, textural, geometric, and index features. Within a multi-level classification framework, the Random Forest (RF) classifier and Convolutional Neural Network (CNN) model are used to classify the cropping patterns in four test areas based on the multi-scale segmented images. The results indicate that integrating the Canny edge detection algorithm with the optimal segmentation scales calculated using the ESP2 tool and RMAS model produces crop parcels with more complete boundaries and better separability. Additionally, optimizing spatial features using the L1-regularized logistic regression model, combined with phenological information, enhances classification accuracy. Within the OBIC framework, the RF classifier achieves higher accuracy in classifying cropping patterns. The overall classification accuracies for the four test areas are 91.93%, 94.92%, 89.37%, and 90.68%, respectively. This paper introduced crop phenological factors, effectively improving the extraction precision of the shattered agricultural planting structure in the Loess Plateau of eastern Gansu Province. Its findings have important application value in crop monitoring, management, food security and other related fields.

Predicting Alzheimer’s disease from cognitive footprints in mid and late life: How much can register data and machine learning help?

BackgroundReal-world data with decades-long medical records are increasingly available alongside the growing adoption of machine learning in healthcare research. We evaluated the performance of machine learning models in predicting the risk of Alzheimer’s disease (AD) using data from the Finnish national registers. MethodsWe conducted a case-control study using data from the Finnish MEDALZ (Medication use and Alzheimer’s disease) study. Altogether 56,741 individuals with incident AD diagnosis (age ≥ 65 years at diagnosis and born after 1922) and their 1:1 age-, sex-, and region of residence-matched controls were included. The association of risk factors, evaluated at different age periods (45–54, 55–64, 65+), and AD were assessed with logistic regression. Predictive accuracies of logistic regressions were compared with seven machine learning models (L1-regularized logistic regression, Naive bayes, Decision tree, Random Forest, Multilayer perceptron, XGBoost, and LightGBM). Findings63.5 % of cases and controls were females and the mean age was 79.1 (SD = 5.1). The strongest associations with AD were observed for head injuries at age 55–64 (OR, 95 % CI 1.33, 1.19–1.48) and 65+ (1.31, 1.23–1.40), followed by antidepressant use (1.30, 1.22–1.38) at 55–64 and antipsychotic use (1.27, 1.19–1.35) at 65+. The predictive accuracies of all models were low, with the best performance (AUC 0.603) observed in Random Forest for predicting AD onset at age 65–69. InterpretationAlthough significant associations were identified between many risk factors and AD, the low predictive accuracies suggest that specialised healthcare diagnosis data is not sufficient for predicting AD and linkage with other data sources is needed.

Enhanced analysis of tabular data through Multi-representation DeepInsight

Tabular data analysis is a critical task in various domains, enabling us to uncover valuable insights from structured datasets. While traditional machine learning methods can be used for feature engineering and dimensionality reduction, they often struggle to capture the intricate relationships and dependencies within real-world datasets. In this paper, we present Multi-representation DeepInsight (MRep-DeepInsight), a novel extension of the DeepInsight method designed to enhance the analysis of tabular data. By generating multiple representations of samples using diverse feature extraction techniques, our approach is able to capture a broader range of features and reveal deeper insights. We demonstrate the effectiveness of MRep-DeepInsight on single-cell datasets, Alzheimer's data, and artificial data, showcasing an improved accuracy over the original DeepInsight approach and machine learning methods like random forest, XGBoost, LightGBM, FT-Transformer and L2-regularized logistic regression. Our results highlight the value of incorporating multiple representations for robust and accurate tabular data analysis. By leveraging the power of diverse representations, MRep-DeepInsight offers a promising new avenue for advancing decision-making and scientific discovery across a wide range of fields.

Machine Learning Methods Using Artificial Intelligence Deployed on Electronic Health Record Data for Identification and Referral of At-Risk Patients From Primary Care Physicians to Eye Care Specialists: Retrospective, Case-Controlled Study.

Identification and referral of at-risk patients from primary care practitioners (PCPs) to eye care professionals remain a challenge. Approximately 1.9 million Americans suffer from vision loss as a result of undiagnosed or untreated ophthalmic conditions. In ophthalmology, artificial intelligence (AI) is used to predict glaucoma progression, recognize diabetic retinopathy (DR), and classify ocular tumors; however, AI has not yet been used to triage primary care patients for ophthalmology referral. This study aimed to build and compare machine learning (ML) methods, applicable to electronic health records (EHRs) of PCPs, capable of triaging patients for referral to eye care specialists. Accessing the Optum deidentified EHR data set, 743,039 patients with 5 leading vision conditions (age-related macular degeneration [AMD], visually significant cataract, DR, glaucoma, or ocular surface disease [OSD]) were exact-matched on age and gender to 743,039 controls without eye conditions. Between 142 and 182 non-ophthalmic parameters per patient were input into 5 ML methods: generalized linear model, L1-regularized logistic regression, random forest, Extreme Gradient Boosting (XGBoost), and J48 decision tree. Model performance was compared for each pathology to select the most predictive algorithm. The area under the curve (AUC) was assessed for all algorithms for each outcome. XGBoost demonstrated the best performance, showing, respectively, a prediction accuracy and an AUC of 78.6% (95% CI 78.3%-78.9%) and 0.878 for visually significant cataract, 77.4% (95% CI 76.7%-78.1%) and 0.858 for exudative AMD, 79.2% (95% CI 78.8%-79.6%) and 0.879 for nonexudative AMD, 72.2% (95% CI 69.9%-74.5%) and 0.803 for OSD requiring medication, 70.8% (95% CI 70.5%-71.1%) and 0.785 for glaucoma, 85.0% (95% CI 84.2%-85.8%) and 0.924 for type 1 nonproliferative diabetic retinopathy (NPDR), 82.2% (95% CI 80.4%-84.0%) and 0.911 for type 1 proliferative diabetic retinopathy (PDR), 81.3% (95% CI 81.0%-81.6%) and 0.891 for type 2 NPDR, and 82.1% (95% CI 81.3%-82.9%) and 0.900 for type 2 PDR. The 5 ML methods deployed were able to successfully identify patients with elevated odds ratios (ORs), thus capable of patient triage, for ocular pathology ranging from 2.4 (95% CI 2.4-2.5) for glaucoma to 5.7 (95% CI 5.0-6.4) for type 1 NPDR, with an average OR of 3.9. The application of these models could enable PCPs to better identify and triage patients at risk for treatable ophthalmic pathology. Early identification of patients with unrecognized sight-threatening conditions may lead to earlier treatment and a reduced economic burden. More importantly, such triage may improve patients' lives.

Culture- and non-culture-based approaches reveal unique features of the ocular microbiome in dry eye patients

PurposeThe purpose of this study was to investigate the ocular microbiome in individuals with dry eye disease and to identify features of their ocular microbiome of possible health and diagnostic significance. MethodsConjunctival samples were collected from both eyes in duplicate from 91 individuals (61 dry eye, 30 healthy) and used for both culture-dependent and culture-independent analyses. Samples were either analysed using next generation sequencing (V3-V4 16S rDNA) or inoculated on a wide range of agar types and grown under a broad range of conditions to maximize recovery. Isolates were identified by partial sequencing of the 16S rDNA and rpoB genes and tested for antibiotic susceptibility. We applied a L2-regularized logistic regression model on the next generation sequencing data to investigate any potential association between severe dry eye disease and the ocular microbiome. ResultsCulture-dependent analysis showed the highest number of colony forming units in healthy individuals. The majority of isolates recovered from the samples were Corynebacterium, Micrococcus sp., Staphylococcus epidermidis, and Cutibacterium acnes. Culture independent analysis revealed 24 phyla, of which Actinobacteria, Firmicutes and Proteobacteria were the most abundant. Over 405 genera were detected of which Corynebacterium was the most dominant, followed by Staphylococcus and Cutibacterium. The L2-regularized logistic regression model indicated that Blautia and Corynebacterium sp. may be associated with severe DED. ConclusionsOur study indicates that the ocular microbiome has characteristic features in severe DED patients. Certain Corynebacterium species and Blautia are of particular interest for future studies.

Abstract 16844: Identifying Splenic Radiomics Features Associated With Risk of Coronary Artery Disease

Introduction: Inflammation is a validated mechanism for residual coronary artery disease (CAD) risk via clinical trials and analysis of germline and somatic genetic variation in blood cells. While the spleen plays an important role in regulating inflammation, the role it plays in CAD is unclear. Hypothesis: We aim to test the hypothesis that radiographic splenic biomarkers may offer novel insights for CAD. Methods: We used deep learning to segment spleens from the abdominal MRIs of 38,474 patients in the UK Biobank. CAD was defined as history of coronary artery bypass grafting, myocardial infarction (MI), coronary artery angioplasty, or billing codes (OPCS-4: K40, K41, K45, K49, K50.2, K75). We extracted image features using the pyradiomics package. To identify image features associated with CAD, we used L1-regularized logistic regression models with the splenic features and clinical features used in the Pooled Cohort Equations (PCE) for outcomes of CAD and cardiovascular disease (CVD) events (defined as a composite of CAD, MI, and stroke outcomes). We computed the area under the receiver operating characteristic (AUROC) on a held-out 30% of the patient population and determined 95% confidence intervals using 1,000 Monte-Carlo resampled bootstraps. Results: Study participants had mean (SD) age 55.1 (7.5) years, 19,921 (51.8%) were women, 1,412 (3.7%) had CAD, and 2,473 (6.4%) had CVD events. The model to predict CAD using the splenic and PCE features had an AUROC of 0.84 (95% CI, 0.84-0.85), and 3 out of 10 significant covariates were splenic features. 1-SD increases in sphericity, minor axis length, and entropy in neighboring voxel gray levels were associated with 5.4% (95% CI, 5.1-5.7) higher, 13.3% (95% CI, 13.0-13.6) lower, and 10.5% (95% CI, 10.2-10.8) higher odds of CAD after adjustment. Results were consistent in the model to predict CVD events, and decreased minor axis length and increased texture variation were associated with increased risk. Model performance was lower for older age groups, who had higher incidences of CAD. Conclusions: This study leverages deep-learning methods across a population based cohort to highlight splenic features correlated with CAD and offers new opportunities to understand the role of the hematopoietic system in CAD.

Diagnostic suspicion bias and machine learning: Breaking the awareness deadlock for sepsis detection.

Many early warning algorithms are downstream of clinical evaluation and diagnostic testing, which means that they may not be useful when clinicians fail to suspect illness and fail to order appropriate tests. Depending on how such algorithms handle missing data, they could even indicate "low risk" simply because the testing data were never ordered. We considered predictive methodologies to identify sepsis at triage, before diagnostic tests are ordered, in a busy Emergency Department (ED). One algorithm used "bland clinical data" (data available at triage for nearly every patient). The second algorithm added three yes/no questions to be answered after the triage interview. Retrospectively, we studied adult patients from a single ED between 2014-16, separated into training (70%) and testing (30%) cohorts, and a final validation cohort of patients from four EDs between 2016-2018. Sepsis was defined per the Rhee criteria. Investigational predictors were demographics and triage vital signs (downloaded from the hospital EMR); past medical history; and the auxiliary queries (answered by chart reviewers who were blinded to all data except the triage note and initial HPI). We developed L2-regularized logistic regression models using a greedy forward feature selection. There were 1164, 499, and 784 patients in the training, testing, and validation cohorts, respectively. The bland clinical data model yielded ROC AUC's 0.78 (0.76-0.81) and 0.77 (0.73-0.81), for training and testing, respectively, and ranged from 0.74-0.79 in four hospital validation. The second model which included auxiliary queries yielded 0.84 (0.82-0.87) and 0.83 (0.79-0.86), and ranged from 0.78-0.83 in four hospital validation. The first algorithm did not require clinician input but yielded middling performance. The second showed a trend towards superior performance, though required additional user effort. These methods are alternatives to predictive algorithms downstream of clinical evaluation and diagnostic testing. For hospital early warning algorithms, consideration should be given to bias and usability of various methods.

Voxel Based Analysis of Local Dose Patterns on Radiation Induced Xerostomia in Head and Neck Cancer with Machine Learning Methods

Despite the technological advancements, xerostomia caused by the irradiation of parotid and/or submandibular glands significantly impacts a patient's quality of life and remains a clinical challenge. The mean dose within the salivary glands that neglects the local dose effect is an oversimplified predictor thus its efficacy is severely limited. This study aims to uncover the local dose patterns associated with the xerostomia by performing a voxel-based analysis of the spatial parotid dose. A cohort of 143 patients who underwent intensity modulated radiation therapy treatment were retrospectively selected for this study. A binary classification was adopted for predictive modeling: group 1 (102 patients) if grade 0 or 1 and group 2 (41 patients) if grade 2 or 3. The anatomy of the parotid gland from the computed tomography (CT) of these patients were standardized to that of a reference patient with rigid and multistage B-spline deformable registration. The resultant deformation field vectors were subsequently utilized to warp the parotid doses and dose gradients to the reference ones. To facilitate pattern recognition, the bilateral parotid doses and dose gradients were further regrouped into contra- and ipsi-lateral depending on their proximity to the tumor targets. Four supervised machine learning (ML) models including ReliefF, L1 regularized logistic regression (L1LR), ridge regression (RR), and random forest (RF) were employed to determine the feature importance of each voxel and discover the dose patterns within the glands that are correlated with xerostomia. The predictive performance of each model was evaluated with the area under the curves (AUC) of the receiver operating characteristic. The image registration accuracy of an average of 95.6% for the parotid gland standardization on CT images was achieved. The four ML models were assessed with 10-fold cross validation showing comparable performance and the AUC scores were 0.74 ± 0.09, 0.75 ± 0.08, 0.73 ± 0.10, and 0.77 ± 0.09 for ReliefF, L1LR, RR, and RF, respectively. The importance patterns showed that the doses to the superior and medial region of the contralateral parotid gland were more correlated with xerostomia from the ReliefF model but not others. Similar importance patterns were observed from the dose gradient distributions along the left-right and anterior-posterior directions. The voxel-based model, which takes into account the spatial information of dose and dose gradient within the parotid gland, demonstrated an improvement in xerostomia predictive modeling as compared to the mean dose predictor, particularly in the local dose effect on xerostomia. If these findings are validated with more patient data and independent studies, they could be incorporated into treatment planning of head and neck patients to reduce the toxicity occurrence.

Stratification of Pediatric COVID-19 Cases Using Inflammatory Biomarker Profiling and Machine Learning.

While pediatric COVID-19 is rarely severe, a small fraction of children infected with SARS-CoV-2 go on to develop multisystem inflammatory syndrome (MIS-C), with substantial morbidity. An objective method with high specificity and high sensitivity to identify current or imminent MIS-C in children infected with SARS-CoV-2 is highly desirable. The aim was to learn about an interpretable novel cytokine/chemokine assay panel providing such an objective classification. This retrospective study was conducted on four groups of pediatric patients seen at multiple sites of Texas Children's Hospital, Houston, TX who consented to provide blood samples to our COVID-19 Biorepository. Standard laboratory markers of inflammation and a novel cytokine/chemokine array were measured in blood samples of all patients. Group 1 consisted of 72 COVID-19, 70 MIS-C and 63 uninfected control patients seen between May 2020 and January 2021 and predominantly infected with pre-alpha variants. Group 2 consisted of 29 COVID-19 and 43 MIS-C patients seen between January and May 2021 infected predominantly with the alpha variant. Group 3 consisted of 30 COVID-19 and 32 MIS-C patients seen between August and October 2021 infected with alpha and/or delta variants. Group 4 consisted of 20 COVID-19 and 46 MIS-C patients seen between October 2021 andJanuary 2022 infected with delta and/or omicron variants. Group 1 was used to train an L1-regularized logistic regression model which was tested using five-fold cross validation, and then separately validated against the remaining naïve groups. The area under receiver operating curve (AUROC) and F1-score were used to quantify the performance of the cytokine/chemokine assay-based classifier. Standard laboratory markers predict MIS-C with a five-fold cross-validated AUROC of 0.86 ± 0.05 and an F1 score of 0.78 ± 0.07, while the cytokine/chemokine panel predicted MIS-C with a five-fold cross-validated AUROC of 0.95 ± 0.02 and an F1 score of 0.91 ± 0.04, with only sixteen of the forty-five cytokines/chemokines sufficient to achieve this performance. Tested on Group 2 the cytokine/chemokine panel yielded AUROC = 0.98 and F1 = 0.93, on Group 3 it yielded AUROC = 0.89 and F1 = 0.89, and on Group 4 AUROC = 0.99 and F1 = 0.97. Adding standard laboratory markers to the cytokine/chemokine panel did not improve performance. A top-10 subset of these 16 cytokines achieves equivalent performance on the validation data sets. Our findings demonstrate that a sixteen-cytokine/chemokine panel as well as the top ten subset provides a highly sensitive, and specific method to identify MIS-C in patients infected with SARS-CoV-2 of all the major variants identified to date.

SDPS-39 AN UPDATED VALIDATION OF PREDICTIVE ALGORITHMS FOR BRAIN METASTASES IN NON-SMALL CELL LUNG CANCER: SYSTEMATIC REVIEW AND INDEPENDENT COHORTVALIDATION ANALYSIS

Abstract INTRODUCTION Many non-small cell lung cancer (NSCLC) patients eventually develop brain metastases (BM). Reliable risk stratification with predictive algorithms can lead to early intervention. Here, we evaluated the performance of published BM risk-stratification algorithms using an independent cohort of NSCLC patients. METHODS We evaluated statistical models predicting BM in NSCLC by systematically reviewing relevant studies and testing them on an independent cohort of NSCLC patients in electronic medical records (2011-2020) from Penn State Health. Patient data was randomly split into 70% training and 30% testing for modeling using L1-regularized logistic regression, and we assessed the models' performance using ROC analyses. RESULTS Out of 1,643 publications, 22 met our criteria, and 12 of those studies (527,258 patients) included variables consistently available in patient charts. Our validation cohort included 1,699 NSCLC patients, with a median age at diagnosis of 68 and 20.4% developing BM. Among feasible models, Zhang 2021 had the highest performance in our cohort (AUROC [95% CI]: 0.89 [0.85-0.93]) and was comprised of the following predictors: age at diagnosis; surgical, chemotherapy, and radiation status; T and N stage; histological grade; and number of organs with metastases. Within our independent cohort, logistic regression revealed that the most informative predictors were the number of organs with metastases (OR [95% CI]: 3.25 [2.70, 3.92]), age at NSCLC diagnosis (OR [95% CI]: 0.98 [0.96, 0.99]), N-stage 1 at diagnosis (OR [95% CI]: 1.83 [1.08, 3.11]), and non-carcinoid or -carcinoma histology (OR [95% CI]: 0.29 [0.10, 0.84]). CONCLUSION Our robust approach, including systematic review and one of the largest-to-date independent institutional datasets, proposes a clinically feasible, novel algorithm for stratifying NSCLC patients based on risk of developing BM. This work can inform pragmatic screening and surveillance guidelines to facilitate early detection of BM in NSCLC patients.

Interpretable machine learning to predict adverse perinatal outcomes: examining marginal predictive value of risk factors during pregnancy

The timely identification of nulliparas at high risk of adverse fetal and neonatal outcomes during pregnancy is crucial for initiating clinical interventions to prevent perinatal complications. Although machine learning methods have been applied to predict preterm birth and other pregnancy complications, many models do not provide explanations of their predictions, limiting the clinical use of the model. This study aimed to develop interpretable prediction models for a composite adverse perinatal outcome (stillbirth, neonatal death, estimated Combined Apgar score of <10, or preterm birth) at different points in time during the pregnancy and to evaluate the marginal predictive value of individual predictors in the context of a machine learning model. This was a secondary analysis of the Nulliparous Pregnancy Outcomes Study: Monitoring Mothers-to-be data, a prospective cohort study in which 10,038 nulliparous pregnant individuals with singleton pregnancies were enrolled. Here, interpretable prediction models were developed using L1-regularized logistic regression for adverse perinatal outcomes using data available at 3 study visits during the pregnancy (visit 1: 6 0/7 to 13 6/7 weeks of gestation; visit 2: 16 0/7 to 21 6/7 weeks of gestation; visit 3: 22 0/7 to 29 6/7 weeks of gestation). We identified the important predictors for each model using SHapley Additive exPlanations, a model-agnostic method of computing explanations of model predictions, and evaluated the marginal predictive value of each predictor using the DeLong test. Our interpretable machine learning model had an area under the receiver operating characteristic curves of 0.617 (95% confidence interval, 0.595-0.639; all predictor variables at visit 1), 0.652 (95% confidence interval, 0.631-0.673; all predictor variables at visit 2), and 0.673 (95% confidence interval, 0.651-0.694; all predictor variables at visit 3). For all visits, the placental biomarker inhibin A was a valuable predictor, as including inhibin A resulted in better performance in predicting adverse perinatal outcomes (P<.001, all visits). At visit 1, endoglin was also a valuable predictor (P<.001). At visit 2, free beta human chorionic gonadotropin (P=.001) and uterine artery pulsatility index (P=.023) were also valuable predictors. At visit 3, cervical length was also a valuable predictor (P<.001). Despite various advances in predictive modeling in obstetrics, the accurate prediction of adverse perinatal outcomes remains difficult. Interpretable machine learning can help clinicians understand how predictions are made, but barriers exist to the widespread clinical adoption of machine learning models for adverse perinatal outcomes. A better understanding of the evolution of risk factors for adverse perinatal outcomes throughout pregnancy is necessary for the development of effective interventions.

Real-Time Prediction of Sepsis in Critical Trauma Patients: Machine Learning–Based Modeling Study

Sepsis is a leading cause of death in patients with trauma, and the risk of mortality increases significantly for each hour of delay in treatment. A hypermetabolic baseline and explosive inflammatory immune response mask clinical signs and symptoms of sepsis in trauma patients, making early diagnosis of sepsis more challenging. Machine learning-based predictive modeling has shown great promise in evaluating and predicting sepsis risk in the general intensive care unit (ICU) setting, but there has been no sepsis prediction model specifically developed for trauma patients so far. To develop a machine learning model to predict the risk of sepsis at an hourly scale among ICU-admitted trauma patients. We extracted data from adult trauma patients admitted to the ICU at Beth Israel Deaconess Medical Center between 2008 and 2019. A total of 42 raw variables were collected, including demographics, vital signs, arterial blood gas, and laboratory tests. We further derived a total of 485 features, including measurement pattern features, scoring features, and time-series variables, from the raw variables by feature engineering. The data set was randomly split into 70% for model development with stratified 5-fold cross-validation, 15% for calibration, and 15% for testing. An Extreme Gradient Boosting (XGBoost) model was developed to predict the hourly risk of sepsis at prediction windows of 4, 6, 8, 12, and 24 hours. We evaluated model performance for discrimination and calibration both at time-step and outcome levels. Clinical applicability of the model was evaluated with varying levels of precision, and the potential clinical net benefit was assessed with decision curve analysis (DCA). A Shapley additive explanation algorithm was applied to show the effect of features on the prediction model. In addition, we trained an L2-regularized logistic regression model to compare its performance with XGBoost. We included 4603 trauma patients in the study, 1196 (26%) of whom developed sepsis. The XGBoost model achieved an area under the receiver operating characteristics curve (AUROC) ranging from 0.83 to 0.88 at the 4-to-24-hour prediction window in the test set. With a ratio of 9 false alerts for every true alert, it predicted 73% (386/529) of sepsis-positive timesteps and 91% (163/179) of sepsis events in the subsequent 6 hours. The DCA showed our model had a positive net benefit in the threshold probability range of 0 to 0.6. In comparison, the logistic regression model achieved lower performance, with AUROC ranging from 0.76 to 0.84 at the 4-to-24-hour prediction window. The machine learning-based model had good discrimination and calibration performance for sepsis prediction in critical trauma patients. Using the model in clinical practice might help to identify patients at risk of sepsis in a time window that enables personalized intervention and early treatment.

Evaluation of Feature Selection Methods for Preserving Machine Learning Performance in the Presence of Temporal Dataset Shift in Clinical Medicine.

Temporal dataset shift can cause degradation in model performance as discrepancies between training and deployment data grow over time. The primary objective was to determine whether parsimonious models produced by specific feature selection methods are more robust to temporal dataset shift as measured by out-of-distribution (OOD) performance, while maintaining in-distribution (ID) performance. Our dataset consisted of intensive care unit patients from MIMIC-IV categorized by year groups (2008-2010, 2011-2013, 2014-2016, and 2017-2019). We trained baseline models using L2-regularized logistic regression on 2008-2010 to predict in-hospital mortality, long length of stay (LOS), sepsis, and invasive ventilation in all year groups. We evaluated three feature selection methods: L1-regularized logistic regression (L1), Remove and Retrain (ROAR), and causal feature selection. We assessed whether a feature selection method could maintain ID performance (2008-2010) and improve OOD performance (2017-2019). We also assessed whether parsimonious models retrained on OOD data performed as well as oracle models trained on all features in the OOD year group. The baseline model showed significantly worse OOD performance with the long LOS and sepsis tasks when compared with the ID performance. L1 and ROAR retained 3.7 to 12.6% of all features, whereas causal feature selection generally retained fewer features. Models produced by L1 and ROAR exhibited similar ID and OOD performance as the baseline models. The retraining of these models on 2017-2019 data using features selected from training on 2008-2010 data generally reached parity with oracle models trained directly on 2017-2019 data using all available features. Causal feature selection led to heterogeneous results with the superset maintaining ID performance while improving OOD calibration only on the long LOS task. While model retraining can mitigate the impact of temporal dataset shift on parsimonious models produced by L1 and ROAR, new methods are required to proactively improve temporal robustness.

Variable importance evaluation with personalized odds ratio for machine learning model interpretability with applications to electronic health records-based mortality prediction.

The interpretability of machine learning models, even though with an excellent prediction performance, remains a challenge in practical applications. The model interpretability and variable importance for well-performed supervised machine learning models are investigated in this study. With the commonly accepted concept of odds ratio (OR), we propose a novel and computationally efficient Variable Importance evaluation framework based on the Personalized Odds Ratio (VIPOR). It is a model-agnostic interpretation method that can be used to evaluate variable importance both locally and globally. Locally, the variable importance is quantified by the personalized odds ratio (POR), which can account for subject heterogeneity in machine learning. Globally, we utilize a hierarchical tree to group the predictors into five groups: completely positive, completely negative, positive dominated, negative dominated, and neutral groups. The relative importance of predictors within each group is ranked based on different statistics of PORs across subjects for different application purposes. For illustration, we apply the proposed VIPOR method to interpreting a multilayer perceptron (MLP) model, which aims to predict the mortality of subarachnoid hemorrhage (SAH) patients using real-world electronic health records (EHR) data. We compare the important variables derived from MLP with other machine learning models, including tree-based models and the L1-regularized logistic regression model. The top importance variables are consistently identified by VIPOR across different prediction models. Comparisons with existing interpretation methods are also conducted and discussed based on publicly available data sets.

Identification of Predictive Biomarkers for Response of R/R DLBCL Patients Treated with Loncastuximab Tesirine Using Low Pass Whole-Genome Sequencing (WGS)

Plasma profiling can non-invasively identify biomarkers associated with treatment outcomes and reveal mechanisms underlying drug resistance. Loncastuximab tesirine (loncastuximab tesirine-lpyl; Lonca), an antibody-drug conjugate comprising a humanized anti-CD19 antibody and a pyrrolobenzodiazepine dimer cytotoxin, is indicated for relapsed/refractory (R/R) diffuse large B-cell lymphoma (DLBCL) after ≥2 systemic treatments. Here, we explored biological signatures associated with response to lonca using plasma-derived cell-free DNA (cfDNA) samples from 145 R/R DLBCL patients in a phase 2 trial (NCT3589469, LOTIS-2). cfDNA was extracted from plasma from 100 patients at baseline and 80 patients on treatment (cycle 2 day 1; C2D1). Low pass WGS was performed to characterize cfDNA fragments, reflecting nucleosome protection and chromatin state. Transcriptional activation for protein-coding genes was inferred by modeling fragment distribution around each transcription start site (TSS-GAP scores1). Binding activity of 504 cancer-associated transcription factors (TFs) was inferred by measuring chromatin accessibility from ~1000 binding sites per factor across the genome (TFBA scores2). Class comparison statistics were performed using the Wilcoxon's rank sum test, with a paired test for longitudinal comparisons. Gene Set Enrichment Analysis (GSEA) was performed using Molecular Signatures Database (MSigDB) sets. A multivariate classifier for RECIST response was constructed by applying L1-regularized logistic regression on both TSS-GAP and TFBA features. We evaluated molecular changes occurring during treatment with lonca by comparing TSS-GAP scores from C2D1 against baseline. Estimated levels of CD19, a proxy for B-cell abundance, significantly decreased after treatment (p = 6.08E-3), which supports the hypothesis that lonca is actively reducing B-cells. Markers of two other immune populations showed no significant change (CD3E p = 0.55; CD14 p = 0.88). GSEA identified 157 significantly enriched sets (152 reduced after treatment; False Discovery Rate (FDR) < 5%), including B-cell gene set signatures decreasing from baseline. To construct a multivariate classifier for response using baseline data, we focused on TSS-GAP scores of genes encoding for all 2751 TFs across the genome. Our final model integrated both TSS-GAP TF scores and clinical lab data to distinguish patients based on predicted response groups (AUC = 0.76). Using GSEA we characterized molecular differences between responders and non-responders and identified 3145 significant gene sets (FDR < 5%), with the majority being enriched with resistance to lonca (3070). Identified biological signatures were enriched for deregulated DNA damage repair response, cell-cycle checkpoints, elevated MYC target levels, and higher metabolic activity. Overall, our results showcase the potential of our approach to identify markers associated with response to lonca and suggest mechanisms of resistance, potentially informing rational drug combination research. Our modeling efforts integrating TSS-GAP and clinical data indicate the potential of this platform to predict lonca efficacy, pending additional validation.

A comparison of logistic regression methods for Ising model estimation.

The Ising model has received significant attention in network psychometrics during the past decade. A popular estimation procedure is IsingFit, which uses nodewise l1-regularized logistic regression along with the extended Bayesian information criterion to establish the edge weights for the network. In this paper, we report the results of a simulation study comparing IsingFit to two alternative approaches: (1) a nonregularized nodewise stepwise logistic regression method, and (2) a recently proposed global l1-regularized logistic regression method that estimates all edge weights in a single stage, thus circumventing the need for nodewise estimation. MATLAB scripts for the methods are provided as supplemental material. The global l1-regularized logistic regression method generally provided greater accuracy and sensitivity than IsingFit, at the expense of lower specificity and much greater computation time. The stepwise approach showed considerable promise. Relative to the l1-regularized approaches, the stepwise method provided better average specificity for all experimental conditions, as well as comparable accuracy and sensitivity at the largest sample size.