Permutation Importance Research Articles

Development and internal validation of machine learning algorithms to predict patient satisfaction after total hip arthroplasty

BackgroundPatient satisfaction is a unique and important measure of success after total hip arthroplasty (THA). Our study aimed to evaluate the use of machine learning (ML) algorithms to predict patient satisfaction after THA.MethodsProspectively collected data of 1508 primary THAs performed between 2006 and 2018 were extracted from our joint replacement registry and split into training (80%) and test (20%) sets. Supervised ML algorithms (Random Forest, Extreme Gradient Boosting, Support Vector Machines, Logistic LASSO) were developed with the training set, using patient demographics, comorbidities and preoperative patient reported outcome measures (PROMs) (Short Form-36 [SF-36], physical component summary [PCS] and mental component summary [MCS], Western Ontario and McMaster’s Universities Osteoarthritis Index [WOMAC] and Oxford Hip Score [OHS]) to predict patient satisfaction at 2 years postoperatively. Predictive performance was evaluated using the independent test set.ResultsPreoperative models demonstrated fair discriminative ability in predicting patient satisfaction, with the LASSO model achieving a maximum AUC of 0.76. Permutation importance revealed that the most important predictors of dissatisfaction were (1) patient’s age, (2) preoperative WOMAC, (3) number of comorbidities, (4) preoperative MCS, (5) previous lumbar spine surgery, and (6) low BMI (< 18.5).ConclusionMachine learning algorithms demonstrated fair discriminative ability in predicting patient satisfaction after THA. We have identified modifiable and non-modifiable predictors of postoperative satisfaction which could enhance preoperative counselling and improve health optimization prior to THA.

Prediction Models of Early Childhood Caries Based on Machine Learning Algorithms.

In this study, we developed machine learning-based prediction models for early childhood caries and compared their performances with the traditional regression model. We analyzed the data of 4195 children aged 1–5 years from the Korea National Health and Nutrition Examination Survey data (2007–2018). Moreover, we developed prediction models using the XGBoost (version 1.3.1), random forest, and LightGBM (version 3.1.1) algorithms in addition to logistic regression. Two different methods were applied for variable selection, including a regression-based backward elimination and a random forest-based permutation importance classifier. We compared the area under the receiver operating characteristic (AUROC) values and misclassification rates of the different models and observed that all four prediction models had AUROC values ranging between 0.774 and 0.785. Furthermore, no significant difference was observed between the AUROC values of the four models. Based on the results, we can confirm that both traditional logistic regression and ML-based models can show favorable performance and can be used to predict early childhood caries, identify ECC high-risk groups, and implement active preventive treatments. However, further research is essential to improving the performance of the prediction model using recent methods, such as deep learning.

Characterizing the Aging of Alphasense NO2 Sensors in Long-Term Field Deployments

Low-cost NO2 sensors have been widely deployed for atmospheric sampling. While their initial performance has been characterized, few studies have examined their long-term degradation. This study focused on the performance of Alphasense low-cost NO2 sensors (NO2-B42F and NO2-B43F) over 4 years (2016-2020). A total of 29 NO2 sensors from 10 batches were collocated 78 times at two sites with reference instruments. Raw signals from "functional" NO2 sensors correlated linearly with reference NO2 concentrations. After long-term deployment, sensor raw signals started to deviate from reference NO2 concentrations due to sensor aging, an accumulated effect after sensor unpacking. Several sensors eventually became "non-functional" as sensor raw signals showed no correlation with reference NO2 concentrations. Sensor aging and non-functionality may be primarily caused by expiration of the ozone (O3) scrubber built into these sensors so that sensors responded to both ambient NO2 and O3. The influence of O3 on sensor response is quantified through the permutation importance method. Most of the sensors are non-functional after approximately 200-400 days of deployment, and no sensor was functional after 400 days of deployment. This result agrees well with the estimated lifetime of the built-in ozone scrubbers considering the ambient ozone concentration in the Pittsburgh area where these sensors were deployed. To ensure reliable data quality in long-term field deployments, we recommend collocating NO2 sensors with reference instruments regularly after 200-400 days of deployment to identify and replace non-functional sensors in a timely manner.

Study becomes insight: Ecological learning from machine learning.

The ecological and environmental science communities have embraced machine learning (ML) for empirical modelling and prediction. However, going beyond prediction to draw insights into underlying functional relationships between response variables and environmental ‘drivers’ is less straightforward. Deriving ecological insights from fitted ML models requires techniques to extract the ‘learning’ hidden in the ML models.We revisit the theoretical background and effectiveness of four approaches for deriving insights from ML: ranking independent variable importance (Gini importance, GI; permutation importance, PI; split importance, SI; and conditional permutation importance, CPI), and two approaches for inference of bivariate functional relationships (partial dependence plots, PDP; and accumulated local effect plots, ALE). We also explore the use of a surrogate model for visualization and interpretation of complex multi‐variate relationships between response variables and environmental drivers. We examine the challenges and opportunities for extracting ecological insights with these interpretation approaches. Specifically, we aim to improve interpretation of ML models by investigating how effectiveness relates to (a) interpretation algorithm, (b) sample size and (c) the presence of spurious explanatory variables.We base the analysis on simulations with known underlying functional relationships between response and predictor variables, with added white noise and the presence of correlated but non‐influential variables. The results indicate that deriving ecological insight is strongly affected by interpretation algorithm and spurious variables, and moderately impacted by sample size. Removing spurious variables improves interpretation of ML models. Meanwhile, increasing sample size has limited value in the presence of spurious variables, but increasing sample size does improves performance once spurious variables are omitted. Among the four ranking methods, SI is slightly more effective than the other methods in the presence of spurious variables, while GI and SI yield higher accuracy when spurious variables are removed. PDP is more effective in retrieving underlying functional relationships than ALE, but its reliability declines sharply in the presence of spurious variables. Visualization and interpretation of the interactive effects of predictors and the response variable can be enhanced using surrogate models, including three‐dimensional visualizations and use of loess planes to represent independent variable effects and interactions.Machine learning analysts should be aware that including correlated independent variables in ML models with no clear causal relationship to response variables can interfere with ecological inference. When ecological inference is important, ML models should be constructed with independent variables that have clear causal effects on response variables. While interpreting ML models for ecological inference remains challenging, we show that careful choice of interpretation methods, exclusion of spurious variables and adequate sample size can provide more and better opportunities to ‘learn from machine learning’.

Aggressively optimizing validation statistics can degrade interpretability of data-driven materials models.

One of the key factors in enabling trust in artificial intelligence within the materials science community is the interpretability (or explainability) of the underlying models used. By understanding what features were used to generate predictions, scientists are then able to critically evaluate the credibility of the predictions and gain new insights. Here, we demonstrate that ignoring hyperparameters viewed as less impactful to the overall model performance can deprecate model explainability. Specifically, we demonstrate that random forest models trained using unconstrained maximum depths, in accordance with accepted best practices, often can report a randomly generated feature as being one of the most important features in generated predictions for classifying an alloy as being a high entropy alloy. We demonstrate that this is the case for impurity, permutation, and Shapley importance rankings, and the latter two showed no strong structure in terms of optimal hyperparameters. Furthermore, we demonstrate that, for the case of impurity importance rankings, only optimizing the validation accuracy, as is also considered standard in the random forest community, yields models that prefer the random feature in generating their predictions. We show that by adopting a Pareto optimization strategy to model performance that balances validation statistics with the differences between the training and validation statistics, one obtains models that reject random features and thus balance model predictive power and explainability.

Predicting 1-Year Mortality after Hip Fracture Surgery: An Evaluation of Multiple Machine Learning Approaches.

Postoperative death within 1 year following hip fracture surgery is reported to be up to 27%. In the current study, we benchmarked the predictive precision and accuracy of the algorithms support vector machine (SVM), naïve Bayes classifier (NB), and random forest classifier (RF) against logistic regression (LR) in predicting 1-year postoperative mortality in hip fracture patients as well as assessed the relative importance of the variables included in the LR model. All adult patients who underwent primary emergency hip fracture surgery in Sweden, between 1 January 2008 and 31 December 2017 were included in the study. Patients with pathological fractures and non-operatively managed hip fractures, as well as those who died within 30 days after surgery, were excluded from the analysis. A LR model with an elastic net regularization were fitted and compared to NB, SVM, and RF. The relative importance of the variables in the LR model was then evaluated using the permutation importance. The LR model including all the variables demonstrated an acceptable predictive ability on both the training and test datasets for predicting one-year postoperative mortality (Area under the curve (AUC) = 0.74 and 0.74 respectively). NB, SVM, and RF tended to over-predict the mortality, particularly NB and SVM algorithms. In contrast, LR only over-predicted mortality when the predicted probability of mortality was larger than 0.7. The LR algorithm outperformed the other three algorithms in predicting 1-year postoperative mortality in hip fracture patients. The most important predictors of 1-year mortality were the presence of a metastatic carcinoma, American Society of Anesthesiologists(ASA) classification, sex, Charlson Comorbidity Index (CCI) ≤ 4, age, dementia, congestive heart failure, hypertension, surgery using pins/screws, and chronic kidney disease.

Prediction of well-being and insight into work-life integration among physicians using machine learning approach.

There has been increasing interest in examining physician well-being and its predictive factors. However, few studies have revealed the characteristics associated with physician well-being and work-life integration using a machine learning approach. To investigate predictive factors of well-being and obtain insights into work-life integration, the survey was conducted by letter mail in a sample of Japanese physicians. A total of 422 responses were collected from 846 physicians. The mean age was 47.9 years, males constituted 83.3% of the physicians, and 88.6% were considered to be well. The most accurate machine learning model showed a mean area under the curve of 0.72. The mean permutation importance of career satisfaction, work hours per week, existence of family support, gender, and existence of power harassment were 0.057, 0.022, 0.009, 0.01, and 0.006, respectively. Using a machine learning model, physician well-being could be predicted. It seems to be influenced by multiple factors, such as career satisfaction, work hours per week, family support, gender, and power harassment. Career satisfaction has the highest impact, while long work hours have a negative effect on well-being. These findings support the need for organizational interventions to promote physician well-being and improve the quality of medical care.

Predicting Depression From Smartphone Behavioral Markers Using Machine Learning Methods, Hyperparameter Optimization, and Feature Importance Analysis: Exploratory Study.

BackgroundDepression is a prevalent mental health challenge. Current depression assessment methods using self-reported and clinician-administered questionnaires have limitations. Instrumenting smartphones to passively and continuously collect moment-by-moment data sets to quantify human behaviors has the potential to augment current depression assessment methods for early diagnosis, scalable, and longitudinal monitoring of depression.ObjectiveThe objective of this study was to investigate the feasibility of predicting depression with human behaviors quantified from smartphone data sets, and to identify behaviors that can influence depression.MethodsSmartphone data sets and self-reported 8-item Patient Health Questionnaire (PHQ-8) depression assessments were collected from 629 participants in an exploratory longitudinal study over an average of 22.1 days (SD 17.90; range 8-86). We quantified 22 regularity, entropy, and SD behavioral markers from the smartphone data. We explored the relationship between the behavioral features and depression using correlation and bivariate linear mixed models (LMMs). We leveraged 5 supervised machine learning (ML) algorithms with hyperparameter optimization, nested cross-validation, and imbalanced data handling to predict depression. Finally, with the permutation importance method, we identified influential behavioral markers in predicting depression.ResultsOf the 629 participants from at least 56 countries, 69 (10.97%) were females, 546 (86.8%) were males, and 14 (2.2%) were nonbinary. Participants’ age distribution is as follows: 73/629 (11.6%) were aged between 18 and 24, 204/629 (32.4%) were aged between 25 and 34, 156/629 (24.8%) were aged between 35 and 44, 166/629 (26.4%) were aged between 45 and 64, and 30/629 (4.8%) were aged 65 years and over. Of the 1374 PHQ-8 assessments, 1143 (83.19%) responses were nondepressed scores (PHQ-8 score <10), while 231 (16.81%) were depressed scores (PHQ-8 score ≥10), as identified based on PHQ-8 cut-off. A significant positive Pearson correlation was found between screen status–normalized entropy and depression (r=0.14, P<.001). LMM demonstrates an intraclass correlation of 0.7584 and a significant positive association between screen status–normalized entropy and depression (β=.48, P=.03). The best ML algorithms achieved the following metrics: precision, 85.55%-92.51%; recall, 92.19%-95.56%; F1, 88.73%-94.00%; area under the curve receiver operating characteristic, 94.69%-99.06%; Cohen κ, 86.61%-92.90%; and accuracy, 96.44%-98.14%. Including age group and gender as predictors improved the ML performances. Screen and internet connectivity features were the most influential in predicting depression.ConclusionsOur findings demonstrate that behavioral markers indicative of depression can be unobtrusively identified from smartphone sensors’ data. Traditional assessment of depression can be augmented with behavioral markers from smartphones for depression diagnosis and monitoring.

Ecological niche modeling of toxic dinoflagellate Prorocentrum cordatum in the Black Sea

The article deals with risk assessment of mass development of a toxic dinoflagellate species Prorocentrum cordatum (Prorocentrales, Dinophyceae) in various regions of the Black Sea by applying ecological niche modeling. Niche modeling was developed using the MaxEnt algorithm. The MaxEnt model training was based on the data of P. cordatum general distribution according to the GBIF source, along with the data on 8 environmental variables retrieved from the Bio-Oracle database. Quality evaluation of the obtained model was executed with the values of AUC statistics, and relative contribution of selected environmental variables to the model was also estimated. In order to additionally assess the applicability of the model, analyses of literature data (from 1886 through 2020) together with original data on the distribution of P. cordatum in the Black Sea (from 1986 to 2020) were carried out. Response curves of environmental variables with the highest permutation importance are given. Map projections of MaxEnt habitats suitability scores for P. cordatum in the Black Sea area are presented. According to our estimates, about 16 percents of the whole Black Sea area are in the high or very high risk zone of P. cordatum mass outbreaks. Maximum values of predicted suitability were obtained for the northwestern part of the Black Sea, while in other regions of the sea they coincided with location of large sources of anthropogenic eutrophication and further distribution of polluted waters by currents. The proposed mathematical model can be used to predict the appearance and mass development of other toxic species.

A Methodological Framework to Discover Pharmacogenomic Interactions Based on Random Forests.

The identification of genomic alterations in tumor tissues, including somatic mutations, deletions, and gene amplifications, produces large amounts of data, which can be correlated with a diversity of therapeutic responses. We aimed to provide a methodological framework to discover pharmacogenomic interactions based on Random Forests. We matched two databases from the Cancer Cell Line Encyclopaedia (CCLE) project, and the Genomics of Drug Sensitivity in Cancer (GDSC) project. For a total of 648 shared cell lines, we considered 48,270 gene alterations from CCLE as input features and the area under the dose-response curve (AUC) for 265 drugs from GDSC as the outcomes. A three-step reduction to 501 alterations was performed, selecting known driver genes and excluding very frequent/infrequent alterations and redundant ones. For each model, we used the concordance correlation coefficient (CCC) for assessing the predictive performance, and permutation importance for assessing the contribution of each alteration. In a reasonable computational time (56 min), we identified 12 compounds whose response was at least fairly sensitive (CCC > 20) to the alteration profiles. Some diversities were found in the sets of influential alterations, providing clues to discover significant drug-gene interactions. The proposed methodological framework can be helpful for mining pharmacogenomic interactions.

Machine Learning Methods for the Prediction of the Inclusion Content of Clean Steel Fabricated by Electric Arc Furnace and Rolling

Machine Learning classification models have been trained and validated from a dataset (73 features and 13,616 instances) including experimental information of a clean cold forming steel fabricated by electric arc furnace and hot rolling. A classification model was developed to identify inclusion contents above the median. The following algorithms were implemented: Logistic Regression, K-Nearest Neighbors, Decision Tree, Random Forests, AdaBoost, Gradient Boosting, Support Vector Classifier and Artificial Neural Networks. Random Forest displayed the best results overall and was selected for the subsequent analyses. The Permutation Importance method was used to identify the variables that influence the inclusion cleanliness and the impact of these variables was determined by means of Partial Dependence Plots. The influence of the final diameter of the coil has been interpreted considering the changes induced by the process of hot rolling in the distribution of inclusions. Several variables related to the secondary metallurgy and tundish operations have been identified and interpreted in metallurgical terms. In addition, the inspection area during the microscopic examination of the samples also appears to influence the inclusion content. Recommendations have been established for the sampling process and for the manufacturing conditions to optimize the inclusionary cleanliness of the steel.

PCN244 Supervised Machine Learning for Predicting Mortality in Acute Myeloid Leukemia Patients Using Electronic Health Record Data

This study implements supervised machine learning (ML) to predict mortality in acute myeloid leukemia (AML) patients and determine the important features in this prediction. Patients were selected from a large US electronic health records database (Cerner Real-World Data) that contains over 87 million patients. We investigated the first visit for patients with an AML ICD-10 diagnosis code, inpatient stay, length of stay of at least 48 hours, and non-missing gender and age. Patient characteristics, hospital characteristics, Charlson Index, quick sequential organ failure assessment (qSOFA), interventions (e.g., mechanical ventilation) and lab values (e.g., minimum white blood cell count) were included in this analysis. Several ML algorithms were compared through 10-fold cross validation; the best performing algorithm was tuned and evaluated with a test dataset. Feature importance was extracted from the final model through permutation importance. There were 8,968 patients included in this study. The ML algorithms that were compared included (mean cross-validation accuracy ± cross-validation standard deviation): logistic regression (72.9% ± 1.6%); random forests (77.5% ± 1.5%); extreme gradient boosting (XGBoost) (78.0% ± 1.2%); k-nearest neighbors (70.8% ± 1.1%); support vector machines (75.8% ± 1.3%). XGBoost was selected for the final model and after hyperparameter tuning had a prediction accuracy of 80.0%. The final model had an F1 score of 0.52, an area under the receiver operator characteristic curve (AUC ROC) of 0.79, a precision of 0.68, and a recall of 0.42. The top five most important features in this prediction were mechanical ventilation, qSOFA, age, intensive care unit admission and minimum white blood cell count. Supervised ML performed well in predicting mortality in AML patients, while identifying the most important features. Similar ML algorithms may identify higher risk AML patients earlier in the hospital to support earlier efforts to modify routine management.

Pathogenic potential assessment of the Shiga toxin–producing Escherichia coli by a source attribution–considered machine learning model

Instead of conventional serotyping and virulence gene combination methods, methods have been developed to evaluate the pathogenic potential of newly emerging pathogens. Among them, the machine learning (ML)-based method using whole-genome sequencing (WGS) data are getting attention because of the recent advances in ML algorithms and sequencing technologies. Here, we developed various ML models to predict the pathogenicity of Shiga toxin-producing Escherichia coli (STEC) isolates using their WGS data. The input dataset for the ML models was generated using distinct gene repertoires from positive (pathogenic) and negative (nonpathogenic) control groups in which each STEC isolate was designated based on the source attribution, the relative risk potential of the isolation sources. Among the various ML models examined, a model using the support vector machine (SVM) algorithm, the SVM model, discriminated between the two control groups most accurately. The SVM model successfully predicted the pathogenicity of the isolates from the major sources of STEC outbreaks, the isolates with the history of outbreaks, and the isolates that cannot be assessed by conventional methods. Furthermore, the SVM model effectively differentiated the pathogenic potentials of the isolates at a finer resolution. Permutation importance analyses of the input dataset further revealed the genes important for the estimation, proposing the genes potentially essential for the pathogenicity of STEC. Altogether, these results suggest that the SVM model is a more reliable and broadly applicable method to evaluate the pathogenic potential of STEC isolates compared with conventional methods.

Machine learning in prediction of intrinsic aqueous solubility of drug‐like compounds: Generalization, complexity, or predictive ability?

AbstractWe present a collection of publicly available intrinsic aqueous solubility data of 829 drug‐like compounds. Four different machine learning algorithms (random forests [RF], LightGBM, partial least squares, and least absolute shrinkage and selection operator [LASSO]) coupled with multistage permutation importance for feature selection and Bayesian hyperparameter optimization were used for the prediction of solubility based on chemical structural information. Our results show that LASSO yielded the best predictive ability on an external test set with a root mean square error (RMSE) (test) of 0.70 log points, an R2(test) of 0.80, and 105 features. Taking into account the number of descriptors as well, an RF model achieves the best balance between complexity and predictive ability with an RMSE(test) of 0.72 log points, an R2(test) of 0.78, and with only 17 features. On a more aggressive test set (principal component analysis [PCA]‐based split), better generalization was observed for the RF model. We propose a ranking score for choosing the best model, as test set performance is only one of the factors in creating an applicable model. The ranking score is a weighted combination of generalization, number of features, and test performance. Out of the two best learners, a consensus model was built exhibiting the best predictive ability and generalization with RMSE(test) of 0.67 log points and a R2(test) of 0.81.

Deep neural networks for global wildfire susceptibility modelling

Wildfire susceptibility is of great importance to the prevention and management of global wildfires. Artificial neural networks (ANNs), particularly multilayer perceptrons (MLPs), have been widely used in wildfire susceptibility. Recently, deep neural networks (DNNs) have become state-of-the-art algorithms, especially convolutional neural networks (CNNs). However, the applicability of different ANNs in wildfire susceptibility has not been thoroughly discussed, and the interpretability of CNNs remains problematic. This paper consists of two parts: one part deeply compares and analyses the application of two feedforward neural network models (a CNNs and an MLPs) in global wildfire susceptibility prediction, and the other part explores the interpretability of the CNNs model. By constructing response variables from the Global Fire Atlas (GFA) and monthly wildfire predictors, four MLPs and CNNs architectures (namely, the pixel-based CNN-1D and MLP-1D models and the grid-based CNN-2D and MLP-2D models) are constructed for four seasons from 2003 to 2016. After model training, validation and testing, seasonal global susceptibility maps are constructed, and the seasonal peaks in fire activity and highly fire-prone areas can be adequately reflected. Finally, five statistical measures (i.e., the overall accuracy, recall, precision, F1 score, and kappa coefficient), the receiver operating characteristic (ROC) curve, area under the ROC curve (AUC), and Wilcoxon signed-rank tests (WSRTs), were used to evaluate the prediction performance of the models. The results show that the contextual-based CNN-2D model can make full use of the neighbourhood information and has the highest accuracy, the MLPs model is more suitable for pixel-based classification, and the performance ranking of the four models is CNN-2D > MLP-1D > MLP-2D > CNN-1D. In addition, the interpretability of the CNNs was explored using an improved permutation importance (PI) algorithm and partial dependence plots (PDPs). The PI algorithm shows that explanatory variables such as maximum temperature (Tasmax), soil temperature (SoilTemp), normalized difference vegetation index (NDVI) and accumulated precipitation (AccuPre) have a large impact on the model in the four seasons, and the nine one-way and two two-way PDPs perfectly represent how the predictor variables influence the prediction on average.

Predictive Values of Preoperative Characteristics for 30-Day Mortality in Traumatic Hip Fracture Patients.

Hip fracture patients have a high risk of mortality after surgery, with 30-day postoperative rates as high as 10%. This study aimed to explore the predictive ability of preoperative characteristics in traumatic hip fracture patients as they relate to 30-day postoperative mortality using readily available variables in clinical practice. All adult patients who underwent primary emergency hip fracture surgery in Sweden between 2008 and 2017 were included in the analysis. Associations between the possible predictors and 30-day mortality was performed using a multivariate logistic regression (LR) model; the bidirectional stepwise method was used for variable selection. An LR model and convolutional neural network (CNN) were then fitted for prediction. The relative importance of individual predictors was evaluated using the permutation importance and Gini importance. A total of 134,915 traumatic hip fracture patients were included in the study. The CNN and LR models displayed an acceptable predictive ability for predicting 30-day postoperative mortality using a test dataset, displaying an area under the ROC curve (AUC) of as high as 0.76. The variables with the highest importance in prediction were age, sex, hypertension, dementia, American Society of Anesthesiologists (ASA) classification, and the Revised Cardiac Risk Index (RCRI). Both the CNN and LR models achieved an acceptable performance in identifying patients at risk of mortality 30 days after hip fracture surgery. The most important variables for prediction, based on the variables used in the current study are age, hypertension, dementia, sex, ASA classification, and RCRI.

Predicting Discharge Disposition Following Meningioma Resection Using a Multi-Institutional Natural Language Processing Model.

Machine learning (ML)-based predictive models are increasingly common in neurosurgery, but typically require large databases of discrete variables for training. Natural language processing (NLP) can extract meaningful data from unstructured text. To present an NLP model that predicts nonhome discharge and a point-of-care implementation. We retrospectively collected age, preoperative notes, and radiology reports from 595 adults who underwent meningioma resection in an academic center from 1995 to 2015. A total of 32 algorithms were trained with the data; the 3 best performing algorithms were combined to form an ensemble. Predictive ability, assessed by area under the receiver operating characteristic curve (AUC) and calibration, was compared to a previously published model utilizing 52 neurosurgeon-selected variables. We then built a multi-institutional model by incorporating notes from 693 patients at another center into algorithm training. Permutation importance was used to analyze the relative importance of each input to model performance. Word clouds and non-negative matrix factorization were used to analyze predictive features of text. The single-institution NLP model predicted nonhome discharge with AUC of 0.80 (95% CI=0.74-0.86) on internal and 0.76 on holdout validation compared to AUC of 0.77 (95% CI=0.73-0.81) and 0.74 for the 52-variable ensemble. The multi-institutional model performed similarly well with AUC=0.78 (95% CI=0.74-0.81) on internal and 0.76 on holdout validation. Preoperative notes most influenced predictions. The model is available at http://nlp-home.insds.org. ML and NLP are underutilized in neurosurgery. Here, we construct a multi-institutional NLP model that predicts nonhome discharge.

MaxEnt distribution modeling for predicting Oreochromis niloticus invasion into the Ganga river system, India and conservation concern of native fish biodiversity

Abstract In order to assess the distribution pattern and understand the prevailing factors for predicting further expansion of an exotic fish Oreochromis niloticus, this study was undertaken in the Ganga river flowing through the state of Uttar Pradesh using MaxEnt model. The authors report the distribution pattern of O. niloticus and prevailing causative factors mounting the expansion of O. niloticus in the Ganges based on MaxEnt modeling technique. The presence only occurrence data-set for this invasive species was prepared from the field data and also from data collated from the authenticated publications of different fisheries researchers. The data-set was analyzed with environmental and topographical variables typically incorporating seasonal and temporal variability using MaxEnt, a maximum entropy algorithm which showed that the area under curve was much closer to 1 ( 0.999). The model predicted elevation as the most influential predictor variable with permutation importance of 69.2% followed by slope_steepness (10.1%), Tmax_1 (7.3%) and Srad_5 (6.8%). The findings from the results suggest that invasive O. niloticus tend to spread in rivers where elevation is lower as well as slope_steepness of the river is higher and thus indicated that invasion might be higher in the downstream of the river. The model suggests that topography and its derived variable are the most significant predictors for distribution of invasive O. niloticus. The results of this study also confirm that the water qualities of the Ganga river are suitable for O. niloticus and if the model is supplemented with water quality variables data, the influential predictor variable in water quality can be well investigated with permutation importance.

Short-Term Memory Binding Distinguishing Amnestic Mild Cognitive Impairment from Healthy Aging: A Machine Learning Study.

Amnestic mild cognitive impairment (aMCI) is the most common preclinical stage of Alzheimer's disease (AD). A strategy to reduce the impact of AD is the early aMCI diagnosis and clinical intervention. Neuroimaging, neurobiological, and genetic markers have proved to be sensitive and specific for the early diagnosis of AD. However, the high cost of these procedures is prohibitive in low-income and middle-income countries (LIMCs). The neuropsychological assessments currently aim to identify cognitive markers that could contribute to the early diagnosis of dementia. Compare machine learning (ML) architectures classifying and predicting aMCI and asset the contribution of cognitive measures including binding function in distinction and prediction of aMCI. We conducted a two-year follow-up assessment of a sample of 154 subjects with a comprehensive multidomain neuropsychological battery. Statistical analysis was proposed using complete ML architectures to compare subjects' performance to classify and predict aMCI. Additionally, permutation importance and Shapley additive explanations (SHAP) routines were implemented for feature importance selection. AdaBoost, gradient boosting, and XGBoost had the highest performance with over 80%success classifying aMCI, and decision tree and random forest had the highest performance with over 70%success predictive routines. Feature importance points, the auditory verbal learning test, short-term memory binding tasks, and verbal and category fluency tasks were used as variables with the first grade of importance to distinguish healthy cognition and aMCI. Although neuropsychological measures do not replace biomarkers' utility, it is a relatively sensitive and specific diagnostic tool for aMCI. Further studies with ML must identify cognitive performance that differentiates conversion from average MCI to the pathological MCI observed in AD.

Development of a machine learning model for the prediction of nodal metastasis in early T classification oral squamous cell carcinoma: SEER‐based population study

AbstractBackgroundThis study aimed to develop and compare machine learning (ML) based predictive models for lymph node metastasis (LNM) in early T classification oral squamous cell carcinoma (OSCC).MethodsWe used data from the Surveillance Epidemiology and End Results Database to develop and validate the predictive models for LNM in patients with T1, T2 OSCC. Using simple clinical and histopathological data, we developed six ML algorithms to predict LNM. The predictive performance of models was compared.ResultsThe areas under the receiver operating characteristic curves (AUCs) of the six models ranged from 0.768 to 0.956. The best prediction performance was achieved with a XGBoost (AUC = 0.956). Permutation importance analysis showed that tumor size is the most important feature in predicting metastasis.ConclusionsWe developed a simplified and reproducible ML‐based predictive model for metastasis in early T classification OSCC that could be helpful for the decision of a treatment strategy.