Internal 5-fold Cross-validation Research Articles

Radiomics-based prediction of local control in patients with brain metastases following postoperative stereotactic radiotherapy.

Surgical resection is the standard of care for patients with large or symptomatic brain metastases (BMs). Despite improved local control after adjuvant stereotactic radiotherapy, the risk of local failure (LF) persists. Therefore, we aimed to develop and externally validate a pre-therapeutic radiomics-based prediction tool to identify patients at high LF risk. Data were collected from A Multicenter Analysis of Stereotactic Radiotherapy to the Resection Cavity of Brain Metastases (AURORA) retrospective study (training cohort: 253 patients from two centers; external test cohort: 99 patients from five centers). Radiomic features were extracted from the contrast-enhancing BM (T1-CE MRI sequence) and the surrounding edema (FLAIR sequence). Different combinations of radiomic and clinical features were compared. The final models were trained on the entire training cohort with the best parameter set previously determined by internal 5-fold cross-validation and tested on the external test set. The best performance in the external test was achieved by an elastic net regression model trained with a combination of radiomic and clinical features with a concordance index (CI) of 0.77, outperforming any clinical model (best CI: 0.70). The model effectively stratified patients by LF risk in a Kaplan-Meier analysis (p < 0.001) and demonstrated an incremental net clinical benefit. At 24 months, we found LF in 9% and 74% of the low and high-risk groups, respectively. A combination of clinical and radiomic features predicted freedom from LF better than any clinical feature set alone. Patients at high risk for LF may benefit from stricter follow-up routines or intensified therapy.

Establishment and validation of a 5-factor diagnostic model for obstructive and non-obstructive azoospermia based on routine clinical parameters.

Azoospermia is a serious leading male-factor cause of infertility in couples of childbearing age. The two main azoospermia types, obstructive (OA) and non-obstructive (NOA) azoospermia, differ in their treatment approaches. Therefore, their clinical diagnosis is extremely important, requiring an accurate, efficient, and easy-to-use diagnostic model. This retrospective observational study included 707 patients with azoospermia treated between 2017 and 2021, 498 with OA, and 209 with NOA. Hematological and seminal plasma parameters, hormone levels, and testicular volume were used in logistic regression analysis to evaluate and compare their diagnostic performance, results showed that the optimal diagnostic model is constructed by five variables including semen volume, semen pH, seminal plasma neutral α-glucosidase activity, follicle-stimulating hormone in the serum, and testicular volume, compared with follicle-stimulating hormone-based and testicular volume-based models. The 5-factor diagnostic model had an accuracy of 90.4%, sensitivity of 96.4%, positive predictive value of 90.6%, negative predictive value of 89.8%, and area under the curve of 0.931, all higher than in the other two models. However, its specificity (76.1%) was slightly lower than in the other models. Meantime, the internal 5-fold cross-validation results indicated that the 5-factor diagnostic model had a good clinical application value. This study established an accurate, efficient, and relatively accessible 5-factor diagnostic model for OA and NOA, providing a reference for clinical decision-making when selecting an appropriate treatment.

Abstract 3537: Assessing machine learning models of drug response predictions using orthotopic patient-derived xenograft

Abstract Accurate prediction of drug response using machine learning (ML) remains a significant challenge in drug development as well as in personalized cancer therapy. Furthermore, there is a scarcity of rigorous external validation studies for evaluating drug response prediction models. Even when available, such validations are often confined to in-vitro studies. Orthotopic patient-derived xenograft (O-PDX) mice serve as essential preclinical models that closely replicate the human tumor microenvironment and assess the therapeutic response to cancer treatments. We present CertisAITM, a novel ensemble of machine learning models that have been trained on a vast array of experimental high-throughput screenings of both monotherapies and combination therapies, incorporating over 4500 investigational and FDA-approved drugs across 10 major cancer indications. The results of our ML drug response prediction models show an average R2 of 0.75 and RMSE of 0.62 (across internal 5-fold cross-validation) for all 10 cancer prediction models. Our evaluation of CertisAI's drug response predictions within O-PDX pharmacology studies has shown a notable overall correlation between actual observed tumor growth inhibition (TGI) and predicted TGI, with an r value of 0.45 across seven indications. The most accurate prediction model achieved an r value of 0.7 for colorectal cancer, encompassing 37 treatments in 6 O-PDX studies. Altogether, these results highlight CertisAI’s potential in enhancing pre-clinical model selection as well as personalized treatment strategies in oncology. Citation Format: Yuan-Hung Chien, Raffaella Pippa, Warren Andrews, Javier Rodriguez, Kristen Buck, Derrick Gorospe, Elizabeth Valencia, Bianca Carapia, Emily Eastwood, Jantzen Sperry, Jonathan Nakashima, Long Do. Assessing machine learning models of drug response predictions using orthotopic patient-derived xenograft [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3537.

An Explainable Transformer-Based Deep Learning Model for the Prediction of Incident Heart Failure

Abstract: Predicting the incidence of complex chronic conditions such as heart failure is challenging. Deep learning models applied to rich electronic health records may improve prediction but remain unexplainable hampering their wider use in medical practice. We aimed to develop a deep-learning framework for accurate and yet explainable prediction of 6-month incident heart failure (HF). Using 100,071 patients from longitudinal linked electronic health records. we applied a novel Transformer based risk model using all community and hospital diagnoses and medications contextualized within the age and calendar year for each patient’s clinical encounter. Feature importance was investigated with an ablation analysis to compare model performance when alternatively removing features and by comparing the variability of temporal representations. A post-hoc perturbation technique was conducted to propagate the changes in the input to the outcome for feature contribution analyses. Our model achieved 0.93 area under the receiver operator curve and 0.69 area under the precision-recall curve on internal 5-fold cross validation and outperformed existing deep learning models. Ablation analysis indicated medication is important for predicting HF risk, calendar year is more important than chronological age, which was further reinforced by temporal variability analysis. Contribution analyses identified risk factors that are closely related to HF. Many of them were consistent with existing knowledge from clinical and epidemiological research but several new associations were revealed which had not been considered in expert driven risk prediction models. In conclusion, the results highlight that our deep learning model, in addition high predictive performance, can inform data-driven risk factor identification.

Clinically Oriented CBCT Periapical Lesion Evaluation via 3D CNN Algorithm

Apical periodontitis (AP) is one of the most prevalent disorders in dentistry. However, it can be underdiagnosed in asymptomatic patients. In addition, the perioperative evaluation of 3-dimensional (3D) lesion volume is of great clinical relevance, but the required slice-by-slice manual delineation method is time- and labor-intensive. Here, for quickly and accurately detecting and segmenting periapical lesions (PALs) associated with AP on cone beam computed tomography (CBCT) images, we proposed and geographically validated a novel 3D deep convolutional neural network algorithm, named PAL-Net. On the internal 5-fold cross-validation set, our PAL-Net achieved an area under the receiver operating characteristic curve (AUC) of 0.98. The algorithm also improved the diagnostic performance of dentists with varying levels of experience, as evidenced by their enhanced average AUC values (junior dentists: 0.89–0.94; senior dentists: 0.91–0.93), and significantly reduced the diagnostic time (junior dentists: 69.3 min faster; senior dentists: 32.4 min faster). Moreover, our PAL-Net achieved an average Dice similarity coefficient over 0.87 (0.85–0.88), which is superior or comparable to that of other existing state-of-the-art PAL segmentation algorithms. Furthermore, we validated the generalizability of the PAL-Net system using multiple external data sets from Central, East, and North China, showing that our PAL-Net has strong robustness. Our PAL-Net can help improve the diagnostic performance and speed of dentists working from CBCT images, provide clinically relevant volume information to dentists, and can potentially be applied in dental clinics, especially without expert-level dentists or radiologists.

Brain health scores to predict neurological outcomes from electronic health records

BackgroundPreserving brain health is a critical priority in primary care, yet screening for these risk factors in face-to-face primary care visits is challenging to scale to large populations. We aimed to develop automated brain health risk scores calculated from data in the electronic health record (EHR) enabling population-wide brain health screening in advance of patient care visits. MethodsThis retrospective cohort study included patients with visits to an outpatient neurology clinic at Massachusetts General Hospital, between January 2010 and March 2021. Survival analysis with an 11-year follow-up period was performed to predict the risk of intracranial hemorrhage, ischemic stroke, depression, death and composite outcome of dementia, Alzheimer’s disease, and mild cognitive impairment. Variables included age, sex, vital signs, laboratory values, employment status and social covariates pertaining to marital, tobacco and alcohol status. Random sampling was performed to create a training (70%) set for hyperparameter tuning in internal 5-fold cross validation and an external hold-out testing (30%) set of patients, both stratified by age. Risk ratios for high and low risk groups were evaluated in the hold-out test set, using 1000 bootstrapping iterations to calculate 95% confidence intervals (CI). ResultsThe cohort comprised 17,040 patients with an average age of 49 ± 15.6 years; majority were males (57 %), White (78 %) and non-Hispanic (80 %). The low and high groups average risk ratios [95 % CI] were: intracranial hemorrhage 0.46 [0.45–0.48] and 2.07 [1.95–2.20], ischemic stroke 0.57 [0.57–0.59] and 1.64 [1.52–1.69], depression 0.68 [0.39–0.74] and 1.29 [0.78–1.38], composite of dementia 0.27 [0.26–0.28] and 3.52 [3.18–3.81] and death 0.24 [0.24–0.24] and 3.96 [3.91–4.00]. ConclusionsSimple risk scores derived from routinely collected EHR accurately quantify the risk of developing common neurologic and psychiatric diseases. These scores can be computed automatically, prior to medical care visits, and may thus be useful for large-scale brain health screening.

CT Based Tumor Radiomics with Machine Learning Classifiers for Molecular Subtyping of Diffuse Large B Cell Lymphoma.

Diffuse large B-cell lymphoma (DLBL) has been classified into two distinct molecular subtypes based on their cell of origin- namely germinal center B-cell-like (GCB) and activated B-cell-subtype (ABC) based on Han's algorithm. These two entities are also prognostically distinct. We assessed CT based tumor radiomic features using machine learning classifiers to distinguish the two molecular subtypes. One hundred one patients were accrued in the study after institutional ethics committee approval. GCB subtype was diagnosed in 59 while 42 patients were ABC subtype. Lesion with maximum SUV was delineated using a semi-automated segmentation tool (3D slicer) and radiomic features were extracted using Pyradiomics. A total of 1030 features were extracted including Shape features, first, second, third, higher order and wavelet-based features. Recursive feature elimination was used for feature selection and optimal 10 features were selected. Five machine learning (ML) tools were used to build a model to sub- classify the two molecular subtypes. (Table -1) Model performance was assessed using Accuracy, Precision, Recall, F1 score and Area under curve of receiver operating characteristic curve. The models were constructed by two methods.1) A 5-fold internal cross validation strategy & 2) splitting the data into training (70%) and validation set (30%). Recursive feature elimination using a random forest algorithm method was used to select 7 radiomic features. These 7 features were then used to build the model using the 5 ML classifiers. Of the five classifiers, Random Forest Classifier (RFC) using a 70:30 training: test strategy was the best performer among all the ML models, with the highest accuracy of 80%, AUC of 0.87, recall(sensitivity), f1score(specificity) & precision (Positive Predictive value; PPV) or precision of 82% respectively. When an internal 5-fold cross validation strategy was used, RFC again performed better than other ML classifiers with an accuracy of 75%, AUC of 0.80, recall of 79%, PPV of 77% and specificity of 79%. The machine-learning based Radiomics features extracted from pre-treatment CT images can provide a simple and non-invasive method for the prediction of DLBL molecular subtypes with favorable predictive accuracy. The Random Forest Classifier was the most accurate in distinguishing GCB subtype from the ABC subtype.

A Machine Learning Framework Integrating Electronic Health Records for Survival Prognostication in Cervix Cancer

Recent advances in Machine learning (ML) have led to promising clinical applications in oncology, such as improved detection on imaging, predicting emergency department visits, and predicting survival. The hypothesis of this study is a machine learning model using features extracted from electronic health records can be created to improve individual prognostication in patients with cervix cancer managed with radiotherapy in a binary higher and lower risk group model. This study used a single-institution retrospective dataset. Electronic records of patients treated for cervix cancer with definitive radiotherapy from 2003-2014 were queried. Features were selected for model building were based on relevant clinical variables and ease of abstraction from records: age, race, smoking status, parity, self-reported unintentional weight loss, histology, revised 2018 Fédération Internationale de Gynécologie et d'Obstétrique (FIGO) staging, tumor size, number of enlarged lymph nodes, involved node location, presence of metastases, radiotherapy duration, and concurrent chemotherapy. The dataset was split into training and testing cohorts, with multivariable Cox regression with Lasso regularization performed to predict hazard ratios (HR), and internal 5-fold cross-validation (training set) to determine the regularization parameter. For each patient, a partial HR using all variables was predicted. A cut-off value maximally splitting the training dataset into higher and lower risk cohorts was calculated in the training dataset using Kaplan-Meier estimates and log-rank tests, and this predicted cut-off value was evaluated in the test dataset. A total of 226 patients were included in study, with a median follow-up of 55.1 months; 226 patients in training cohort, and 57 in testing cohort. Feature concordance indices of 0.74 and 0.75 were obtained in the testing and training datasets, respectively, with minimal over-fitting. The three variables that contributed most to the model (with Log HR, 95% CI, p-value) were FIGO stage (0.38, 0.1-0.66, p = 0.01), presence of metastases (0.26, -0.02-0.53, p = 0.07), and unintentional weight loss (-0.34, -0.54-0.14, p<0.01), and FIGO IIB maximally divided the cohort into higher and lower risk stage groupings. A partial HR of 1.15 threshold using all features predicted the largest survival separation between higher and lower risk patients, more so than stage alone (p = 1.18 × 10-3 separated by stage alone vs p = 8.97 × 10-7 separated by partial hazard). A machine learning approach was able to improve prognostication of survival in cervix cancer using additional features from electronic health records. Future work should explore prognostication using large scale datasets and pre-treatment variables for potential incorporation into patient discussions and shared decision making.

DeepBreath—automated detection of respiratory pathology from lung auscultation in 572 pediatric outpatients across 5 countries

The interpretation of lung auscultation is highly subjective and relies on non-specific nomenclature. Computer-aided analysis has the potential to better standardize and automate evaluation. We used 35.9 hours of auscultation audio from 572 pediatric outpatients to develop DeepBreath : a deep learning model identifying the audible signatures of acute respiratory illness in children. It comprises a convolutional neural network followed by a logistic regression classifier, aggregating estimates on recordings from eight thoracic sites into a single prediction at the patient-level. Patients were either healthy controls (29%) or had one of three acute respiratory illnesses (71%) including pneumonia, wheezing disorders (bronchitis/asthma), and bronchiolitis). To ensure objective estimates on model generalisability, DeepBreath is trained on patients from two countries (Switzerland, Brazil), and results are reported on an internal 5-fold cross-validation as well as externally validated (extval) on three other countries (Senegal, Cameroon, Morocco). DeepBreath differentiated healthy and pathological breathing with an Area Under the Receiver-Operator Characteristic (AUROC) of 0.93 (standard deviation [SD] ± 0.01 on internal validation). Similarly promising results were obtained for pneumonia (AUROC 0.75 ± 0.10), wheezing disorders (AUROC 0.91 ± 0.03), and bronchiolitis (AUROC 0.94 ± 0.02). Extval AUROCs were 0.89, 0.74, 0.74 and 0.87 respectively. All either matched or were significant improvements on a clinical baseline model using age and respiratory rate. Temporal attention showed clear alignment between model prediction and independently annotated respiratory cycles, providing evidence that DeepBreath extracts physiologically meaningful representations. DeepBreath provides a framework for interpretable deep learning to identify the objective audio signatures of respiratory pathology.

Radiomic model to predict the expression of PD-1 and overall survival of patients with ovarian cancer.

Programmed cell death 1 (PD-1), encoded by programmed cell death protein 1 (PDCD1), is widely investigated in clinical trials. We aimed to develop a radiomic model to discriminate its expression levels patients with ovarian cancer (OC) and explore its prognostic value. Computed tomography (CT) images with the corresponding sequencing data and clinicopathological features were used. The volumes of interest were manually delineated. After extraction and normalization, the radiomic features were screened using repeat least absolute shrinkage and selection operator. A radiomic model for PD-1 prediction, radiomic score (rad_score), was developed using logistic regression and validated via internal 5-fold cross-validation. The Kaplan-Meier curves, COX proportional hazards model, and landmark analysis were used for survival analysis. The mRNA level of PDCD1 significantly affects the overall survival (OS) of OC patients. The rad_score for PDCD1 prediction was based on four features and was significantly correlated with other genes involved in T-cell exhaustion and immune checkpoint molecules. The areas under the receiver operating characteristic curves reached 0.810 and 0.772 in the training and validation datasets, respectively. The calibration curves and decision curve analysis proved the model's fitness and clinical benefits. Patients with higher rad_score had poorer OS (P<0.001, 0.031, 0.014, 0.01, and<0.001, after landmark of 12months, before and after landmark of 36months, and before and after landmark of 60months, respectively). The radiomic signature from CT images can discriminate the PD-1 expression status and OC prognosis, which is correlated with T-cell exhaustion.

MACHINE LEARNING PREDICTION OF IN-HOSPITAL DISEASE PROGRESSION IN COMMUNITY-ACQUIRED PNEUMONIA: DERIVATION AND VALIDATION OF CLINICAL DETERIORATION IN PNEUMONIA (CDIP)

MACHINE LEARNING PREDICTION OF IN-HOSPITAL DISEASE PROGRESSION IN COMMUNITY-ACQUIRED PNEUMONIA: DERIVATION AND VALIDATION OF CLINICAL DETERIORATION IN PNEUMONIA (CDIP)

An Explainable Transformer-Based Deep Learning Model for the Prediction of Incident Heart Failure.

Predicting the incidence of complex chronic conditions such as heart failure is challenging. Deep learning models applied to rich electronic health records may improve prediction but remain unexplainable hampering their wider use in medical practice. We aimed to develop a deep-learning framework for accurate and yet explainable prediction of 6-month incident heart failure (HF). Using 100,071 patients from longitudinal linked electronic health records across the U.K., we applied a novel Transformer-based risk model using all community and hospital diagnoses and medications contextualized within the age and calendar year for each patient's clinical encounter. Feature importance was investigated with an ablation analysis to compare model performance when alternatively removing features and by comparing the variability of temporal representations. A post-hoc perturbation technique was conducted to propagate the changes in the input to the outcome for feature contribution analyses. Our model achieved 0.93 area under the receiver operator curve and 0.69 area under the precision-recall curve on internal 5-fold cross validation and outperformed existing deep learning models. Ablation analysis indicated medication is important for predicting HF risk, calendar year is more important than chronological age, which was further reinforced by temporal variability analysis. Contribution analyses identified risk factors that are closely related to HF. Many of them were consistent with existing knowledge from clinical and epidemiological research but several new associations were revealed which had not been considered in expert-driven risk prediction models. In conclusion, the results highlight that our deep learning model, in addition high predictive performance, can inform data-driven risk factor identification.

Detection of maxillary sinus fungal ball via 3-D CNN-based artificial intelligence: Fully automated system and clinical validation.

BackgroundThis study aims to develop artificial intelligence (AI) system to automatically classify patients with maxillary sinus fungal ball (MFB), chronic rhinosinusitis (CRS), and healthy controls (HCs).MethodsWe collected 512 coronal image sets from ostiomeatal unit computed tomography (OMU CT) performed on subjects who visited a single tertiary hospital. These data included 254 MFB, 128 CRS, and 130 HC subjects and were used for training the proposed AI system. The AI system takes these 1024 sets of half CT images as input and classifies these as MFB, CRS, or HC. To optimize the classification performance, we adopted a 3-D convolutional neural network of ResNet 18. We also collected 64 coronal OMU CT image sets for external validation, including 26 MFB, 18 CRS, and 20 HCs from subjects from another referral hospital. Finally, the performance of the developed AI system was compared with that of the otolaryngology resident physicians.ResultsClassification performance was evaluated using internal 5-fold cross-validation (818 training and 206 internal validation data) and external validation (128 data). The area under the receiver operating characteristic over the internal 5-fold cross-validation and the external validation was 0.96 ±0.006 and 0.97 ±0.006, respectively. The accuracy of the internal 5-fold cross-validation and the external validation was 87.5 ±2.3% and 88.4 ±3.1%, respectively. As a result of performing a classification test on external validation data from six otolaryngology resident physicians, the accuracy was obtained as 84.6 ±11.3%.ConclusionsThis AI system is the first study to classify MFB, CRS, and HC using deep neural networks to the best of our knowledge. The proposed system is fully automatic but performs similarly to or better than otolaryngology resident physicians. Therefore, we believe that in regions where otolaryngology specialists are scarce, the proposed AI will perform sufficiently effective diagnosis on behalf of doctors.

Association of CYP1A1 and CYP1B1 inhibition in in vitro assays with drug-induced liver injury.

Drug-induced liver injury (DILI) is one of the major causes for the discontinuation of drug development and withdrawal of drugs from the market. Since it is known that reactive metabolite formation and being substrates or inhibitors of cytochrome P450s (P450s) are associated with DILI, we systematically investigated the association between human P450 inhibition and DILI. The inhibitory activity of 266 DILI-positive drugs (DILI drugs) and 92 DILI-negative drugs (no-DILI drugs), which were selected from Liver Toxicity Knowledge Base (US Food and Drug Administration), against 8 human P450 forms was assessed using recombinant enzymes and luminescent substrates, and the threshold values showing the highest balanced accuracy for DILI discrimination were determined for each P450 enzyme using receiver operating characteristic analyses. The results showed that among the P450s tested, CYP1A1 and CYP1B1 were inhibited by DILI drugs more than no-DILI drugs with a statistical significance. We found that 91% of drugs that showed inhibitory activity greater than the threshold values against CYP1A1 or CYP1B1 were DILI drugs. The results of internal 5-fold cross-validation confirmed the usefulness of CYP1A1 and CYP1B1 inhibition data for the threshold-based discrimination of DILI drugs. Although the contribution of these P450s to drug metabolism in the liver is considered minimal, our present findings suggest that the assessment of CYP1A1 and CYP1B1 inhibition is useful for screening DILI risk of drug candidates at the early stage of drug development.

Comparison of Machine Learning Models for the Androgen Receptor.

The androgen receptor (AR) is a target of interest for endocrine disruption research, as altered signaling can affect normal reproductive and neurological development for generations. In an effort to prioritize compounds with alternative methodologies, the U.S. Environmental Protection Agency (EPA) used in vitro data from 11 assays to construct models of AR agonist and antagonist signaling pathways. While these EPA ToxCast AR models require in vitro data to assign a bioactivity score, Bayesian machine learning methods can be used for prospective prediction from molecule structure alone. This approach was applied to multiple types of data corresponding to the EPA's AR signaling pathway with proprietary software, Assay Central. The training performance of all machine learning models, including six other algorithms, was evaluated by internal 5-fold cross-validation statistics. Bayesian machine learning models were also evaluated with external predictions of reference chemicals to compare prediction accuracies to published results from the EPA. The machine learning model group selected for further studies of endocrine disruption consisted of continuous AC50 data from the February 2019 release of ToxCast/Tox21. These efforts demonstrate how machine learning can be used to predict AR-mediated bioactivity and can also be applied to other targets of endocrine disruption.

Comparing Machine Learning Algorithms for Predicting Drug-Induced Liver Injury (DILI).

Drug-induced liver injury (DILI) is one the most unpredictable adverse reactions to xenobiotics in humans and the leading cause of postmarketing withdrawals of approved drugs. To date, these drugs have been collated by the FDA to form the DILIRank database, which classifies DILI severity and potential. These classifications have been used by various research groups in generating computational predictions for this type of liver injury. Recently, groups from Pfizer and AstraZeneca have collated DILI in vitro data and physicochemical properties for compounds that can be used along with data from the FDA to build machine learning models for DILI. In this study, we have used these data sets, as well as the Biopharmaceutics Drug Disposition Classification System data set, to generate Bayesian machine learning models with our in-house software, Assay Central. The performance of all machine learning models was assessed through both the internal 5-fold cross-validation metrics and prediction accuracy of an external test set of compounds with known hepatotoxicity. The best-performing Bayesian model was based on the DILI-concern category from the DILIRank database with an ROC of 0.814, a sensitivity of 0.741, a specificity of 0.755, and an accuracy of 0.746. A comparison of alternative machine learning algorithms, such as k-nearest neighbors, support vector classification, AdaBoosted decision trees, and deep learning methods, produced similar statistics to those generated with the Bayesian algorithm in Assay Central. This study demonstrates machine learning models grouped in a tool called MegaTox that can be used to predict early-stage clinical compounds, as well as recent FDA-approved drugs, to identify potential DILI.

The CLASSE GATOR (CLinical Acronym SenSE disambiGuATOR): A Method for predicting acronym sense from neonatal clinical notes

ObjectiveTo develop an algorithm for identifying acronym ‘sense’ from clinical notes without requiring a clinically annotated training set. Materials and MethodsOur algorithm is called CLASSE GATOR: Clinical Acronym SenSE disambiGuATOR. CLASSE GATOR extracts acronyms and definitions from PubMed Central (PMC). A logistic regression model is trained using words associated with specific acronym-definition pairs from PMC. CLASSE GATOR uses this library of acronym-definitions and their corresponding word feature vectors to predict the acronym ‘sense’ from Beth Israel Deaconess (MIMIC-III) neonatal notes. ResultsWe identified 1,257 acronyms and 8,287 definitions including a random definition from 31,764 PMC articles on prenatal exposures and 2,227,674 PMC open access articles. The average number of senses (definitions) per acronym was 6.6 (min = 2, max = 50). The average internal 5-fold cross validation was 87.9 % (on PMC). We found 727 unique acronyms (57.29 %) from PMC were present in 105,044 neonatal notes (MIMIC-III). We evaluated the performance of acronym prediction using 245 manually annotated clinical notes with 9 distinct acronyms. CLASSE GATOR achieved an overall accuracy of 63.04 % and outperformed random for 8/9 acronyms (88.89 %) when applied to clinical notes. We also compared our algorithm with UMN's acronym set, and found that CLASSE GATOR outperformed random for 63.46 % of 52 acronyms when using logistic regression, 75.00 % when using Bert and 76.92 % when using BioBert as the prediction algorithm within CLASSE GATOR. ConclusionsCLASSE GATOR is the first automated acronym sense disambiguation method for clinical notes. Importantly, CLASSE GATOR does not require an expensive manually annotated acronym-definition corpus for training.

Development and evaluation of in silico prediction model for drug-induced respiratory toxicity by using naïve Bayes classifier method

Respiratory toxicity is considered as main cause of drug withdrawal, which could seriously injure human health or even lead to death. The objective of this investigation was to develop an in silico prediction model of drug-induced respiratory toxicity by using naïve Bayes classifier. The genetic algorithm was used to select important molecular descriptors related to respiratory toxicity, and the ECFP_6 fingerprint descriptors were applied to the respiratory toxic/non-toxic fragments production. The established prediction model was validated by the internal 5-fold cross validation and external test set. The naïve Bayes classifier generated overall prediction accuracy of 91.8% for the training set and 84.3% for the external test set. Furthermore, six molecular descriptors (e.g., number of O atoms, number of N atoms, molecular weight, Apol, number of H acceptors and molecular polar surface area) considered as important for the drug-induced respiratory toxicity were identified, and some critical fragments related to the respiratory toxicity were achieved. We hope the established naïve Bayes prediction model could be used as a toxicological screening of chemicals for respiratory sensitization potential in drug development, and these obtained important information of respiratory toxic chemical structures could offer theoretical guidance for hit and lead optimization.

Development of novel prediction model for drug-induced mitochondrial toxicity by using naïve Bayes classifier method

Mitochondrial dysfunction has been considered as an important contributing factor in the etiology of drug-induced organ toxicity, and even plays an important role in the pathogenesis of some diseases. The objective of this investigation was to develop a novel prediction model of drug-induced mitochondrial toxicity by using a naïve Bayes classifier. For comparison, the recursive partitioning classifier prediction model was also constructed. Among these methods, the prediction performance of naïve Bayes classifier established here showed best, which yielded average overall prediction accuracies for the internal 5-fold cross validation of the training set and external test set were 95 ± 0.6% and 81 ± 1.1%, respectively. In addition, four important molecular descriptors and some representative substructures of toxicants produced by ECFP_6 fingerprints were identified. We hope the established naïve Bayes prediction model can be employed for the mitochondrial toxicity assessment, and these obtained important information of mitochondrial toxicants can provide guidance for medicinal chemists working in drug discovery and lead optimization.

AlzhCPI: A knowledge base for predicting chemical-protein interactions towards Alzheimer's disease.

Alzheimer's disease (AD) is a complicated progressive neurodegeneration disorder. To confront AD, scientists are searching for multi-target-directed ligands (MTDLs) to delay disease progression. The in silico prediction of chemical-protein interactions (CPI) can accelerate target identification and drug discovery. Previously, we developed 100 binary classifiers to predict the CPI for 25 key targets against AD using the multi-target quantitative structure-activity relationship (mt-QSAR) method. In this investigation, we aimed to apply the mt-QSAR method to enlarge the model library to predict CPI towards AD. Another 104 binary classifiers were further constructed to predict the CPI for 26 preclinical AD targets based on the naive Bayesian (NB) and recursive partitioning (RP) algorithms. The internal 5-fold cross-validation and external test set validation were applied to evaluate the performance of the training sets and test set, respectively. The area under the receiver operating characteristic curve (ROC) for the test sets ranged from 0.629 to 1.0, with an average of 0.903. In addition, we developed a web server named AlzhCPI to integrate the comprehensive information of approximately 204 binary classifiers, which has potential applications in network pharmacology and drug repositioning. AlzhCPI is available online at http://rcidm.org/AlzhCPI/index.html. To illustrate the applicability of AlzhCPI, the developed system was employed for the systems pharmacology-based investigation of shichangpu against AD to enhance the understanding of the mechanisms of action of shichangpu from a holistic perspective.