Commentary: Development of Machine Learning Algorithms for Prediction of 30-Day Mortality After Surgery for Spinal Metastasis.

Updated external validation of the SORG machine learning algorithms for prediction of ninety-day and one-year mortality after surgery for spinal metastasis

Artificial intelligence and spine: rise of the machines

Preoperative prognostication of short-term postoperative mortality in patients with spinal metastatic disease can improve shared decision making around end-of-life care. To (1) develop machine learning algorithms for prediction of short-term mortality and (2) deploy these models in an open access web application. The American College of Surgeons, National Surgical Quality Improvement Program was used to identify patients that underwent operative intervention for metastatic disease. Four machine learning algorithms were developed, and the algorithm with the best performance across discrimination, calibration, and overall performance was integrated into an open access web application. The 30-d mortality for the 1790 patients undergoing surgery for spinal metastatic disease was 8.49%. Preoperative factors used for prognostication were albumin, functional status, white blood cell count, hematocrit, alkaline phosphatase, spinal location (cervical, thoracic, lumbosacral), and severity of comorbid systemic disease (American Society of Anesthesiologist Class). In this population, machine learning algorithms developed to predict 30-d mortality performed well on discrimination (c-statistic), calibration (assessed by calibration slope and intercept), Brier score, and decision analysis. An open access web application was developed for the best performing model and this web application can be found here: https://sorg-apps.shinyapps.io/spinemets/. Machine learning algorithms are promising for prediction of postoperative outcomes in spinal oncology and these algorithms can be integrated into clinically useful decision tools. As the volume of data in oncology continues to grow, creation of learning systems and deployment of these systems as accessible tools may significantly enhance prognostication and management.

The aging and multimorbid population and health personnel shortages pose a substantial burden on primary health care. While predictive machine learning (ML) algorithms have the potential to address these challenges, concerns include transparency and insufficient reporting of model validation and effectiveness of the implementation in the clinical workflow. To systematically identify predictive ML algorithms implemented in primary care from peer-reviewed literature and US Food and Drug Administration (FDA) and Conformité Européene (CE) registration databases and to ascertain the public availability of evidence, including peer-reviewed literature, gray literature, and technical reports across the artificial intelligence (AI) life cycle. PubMed, Embase, Web of Science, Cochrane Library, Emcare, Academic Search Premier, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI.org (Association for the Advancement of Artificial Intelligence), arXiv, Epistemonikos, PsycINFO, and Google Scholar were searched for studies published between January 2000 and July 2023, with search terms that were related to AI, primary care, and implementation. The search extended to CE-marked or FDA-approved predictive ML algorithms obtained from relevant registration databases. Three reviewers gathered subsequent evidence involving strategies such as product searches, exploration of references, manufacturer website visits, and direct inquiries to authors and product owners. The extent to which the evidence for each predictive ML algorithm aligned with the Dutch AI predictive algorithm (AIPA) guideline requirements was assessed per AI life cycle phase, producing evidence availability scores. The systematic search identified 43 predictive ML algorithms, of which 25 were commercially available and CE-marked or FDA-approved. The predictive ML algorithms spanned multiple clinical domains, but most (27 [63%]) focused on cardiovascular diseases and diabetes. Most (35 [81%]) were published within the past 5 years. The availability of evidence varied across different phases of the predictive ML algorithm life cycle, with evidence being reported the least for phase 1 (preparation) and phase 5 (impact assessment) (19% and 30%, respectively). Twelve (28%) predictive ML algorithms achieved approximately half of their maximum individual evidence availability score. Overall, predictive ML algorithms from peer-reviewed literature showed higher evidence availability compared with those from FDA-approved or CE-marked databases (45% vs 29%). The findings indicate an urgent need to improve the availability of evidence regarding the predictive ML algorithms' quality criteria. Adopting the Dutch AIPA guideline could facilitate transparent and consistent reporting of the quality criteria that could foster trust among end users and facilitating large-scale implementation.

Patients presenting with neurological deficits and/or pain due to spinal metastasis usually require immediate or subacute surgical treatment. Nevertheless, it is unclear whether or not side effects of primary cancer location might influence postoperative complication rate. We therefore analyzed our spinal database to identify factors influencing early postoperative complications after surgery for symptomatic spinal metastases. From 2013 to 2017, 163 consecutive patients suffering from symptomatic spinal metastases were treated at our department. Early postoperative complications were defined as any postoperative event requiring additional medical or surgical treatment within 30days of spinal surgery. A multivariate regression analysis was performed to identify independent predictors for postoperative complications after surgery for spinal metastasis. Overall, 39 of 163 patients who underwent spinal surgery for spinal metastasis developed early postoperative complications throughout the treatment course (24%). Preoperative ASA score ≥ 3 (p = 0.003), preoperative C-reactive protein level > 10mg/l (p = 0.008), preoperative Karnofsky Performance Score < 60% (p = 0.03), radiation treatment within 2months of surgery (p = 0.01), presence of diabetes mellitus (p = 0.008), and preoperative complete neurological impairment (p = 0.04) were significant and independent predictors for early postoperative complications in patients with surgery for spinal metastasis. The ability to preoperatively predict postoperative complication risk is valuable to select critically ill patients at higher risk requiring special attention. Therefore, the present study identified several significant and independent risk factors for the development of early postoperative complication in patients who underwent surgery for spinal metastasis.

Development of machine learning algorithms for prediction of mortality in spinal epidural abscess

BackgroundMedical researchers and clinicians have shown much interest in developing machine learning (ML) algorithms to detect/predict surgical site infections (SSIs). However, little is known about the overall performance of ML algorithms in predicting SSIs and how to improve the algorithm's robustness. We conducted a systematic review and meta-analysis to summarize the performance of ML algorithms in SSIs case detection and prediction and to describe the impact of using unstructured and textual data in the development of ML algorithms.MethodsMEDLINE, EMBASE, CINAHL, CENTRAL and Web of Science were searched from inception to March 25, 2021. Study characteristics and algorithm development information were extracted. Performance statistics (e.g., sensitivity, area under the receiver operating characteristic curve [AUC]) were pooled using a random effect model. Stratified analysis was applied to different study characteristic levels. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) was followed.ResultsOf 945 articles identified, 108 algorithms from 32 articles were included in this review. The overall pooled estimate of the SSI incidence rate was 3.67%, 95% CI: 3.58–3.76. Mixed-use of structured and textual data-based algorithms (pooled estimates of sensitivity 0.83, 95% CI: 0.78–0.87, specificity 0.92, 95% CI: 0.86–0.95, AUC 0.92, 95% CI: 0.89–0.94) outperformed algorithms solely based on structured data (sensitivity 0.56, 95% CI:0.43–0.69, specificity 0.95, 95% CI:0.91–0.97, AUC = 0.90, 95% CI: 0.87–0.92).ConclusionsML algorithms developed with structured and textual data provided optimal performance. External validation of ML algorithms is needed to translate current knowledge into clinical practice.

40. Surgical Site Infection in Spinal Metastasis - Risk Factor and Countermeasure

Category:Ankle Arthritis; AnkleIntroduction/Purpose:Ankle arthrodesis and total ankle replacement are the most commonly performed procedures for surgical management of ankle arthritis. Arthrodesis provides effective pain relief but the rate of complications after arthrodesis is higher as it is more commonly performed in patients with comorbidities that preclude ankle replacement. Accurately risk- stratifying patients who undergo ankle arthrodesis would be of great utility, given the significant cost and morbidity associated with developing major perioperative complications. There is a paucity of accurate prediction models that can be used to pre- operatively risk-stratify patients for ankle arthrodesis. We aim to develop a machine learning (ML) algorithm for prediction of major perioperative complication after ankle arthrodesis as well as compare its performance against traditional predictive models based on logistic regression.Methods:This is a retrospective cohort study of adult patients who underwent ankle arthrodesis at any non-federal California hospital between 2015 and 2017. The primary outcome was readmission within 30 days or major perioperative complication - venous thromboembolism within 30 days, myocardial infarction within 7 days, pneumonia within 7 days, systemic infection within 7 days, surgical site bleeding within 90 days, and wound complications within 90 days. We build ML and logistic regression models that span different classes of modeling approaches: XGBoost, AdaBoost, Gradient Boosting, and Random Forest. Discrimination and calibration were assessed using area under the receiver operating characteristic curve (AUROC) and Brier score, respectively. We utilize a partial dependence function to measure the importance of an individual feature by assessing the average effect in predicted risks when its value is altered. We rank the contribution of the included variables to the prediction of adverse outcomes.Results:A total of 1,084 patients met inclusion criteria for this study. There were 131 major complications or readmission (12.1%). The optimized XGBoost algorithm demonstrates higher discrimination (AUROC: 0.707 + 0.052) compared to LR (0.691 + 0.055). The receiver operating characteristic curves for the XGBoost and logistic regression models are visualized in Figure 1. XGBoost also outperforms the three other ML models. This model was well calibrated (Brier score: 0.103 + 0.001). The variables most important for the XGBoost model include diabetes, chronic kidney disease, implant complication, and major fracture. Five of the ten most important features for XGBoost were markedly less important for the traditional logistic regression model: male sex, prior hip fracture, cardiorespiratory failure, acute renal failure, and dialysis status.Conclusion:We report a ML algorithm for prediction of major perioperative complications after ankle arthrodesis. The optimized XGBoost model is well-calibrated and demonstrates superior risk prediction to logistic regression. This tool may identify and address potentially modifiable risk factors, helping to accurately risk-stratify patients and decrease likelihood of major complications. Notably, the predictors most important for XGBoost are different from those for logistic regression. This suggests that the superior discriminative capability of ML methods stems from their ability to capture complex non-linear relationships between variables that logistic regression is unable to detect.

Predictive factors of the 30-day mortality after surgery for spinal metastasis: Analysis of a nationwide database

INTRODUCTION: This study compares and contrasts various machine learning algorithms for predicting diabetes. The study of current research work is to analyse the effectiveness of various machine learning algorithms for diabetes prediction.&#x0D; OBJECTIVES: To compare the efficacy of various machine learning algorithms for diabetic prediction.&#x0D; METHODS: For the same, a diabetic dataset was subjected to the application of various well-known machine learning algorithms. Unbalanced data was handled by pre-processing the dataset. The models were subsequently trained and assessed using different performance metrics namely F1-score, accuracy, sensitivity, and specificity.&#x0D; RESULTS: The experimental results show that the Decision Tree and ensemble model outperforms all other comparative models in terms of accuracy and other evaluation metrics.&#x0D; CONCLUSION: This study can help healthcare practitioners and researchers to choose the best machine learning model for diabetes prediction based on their specific needs and available data.

Scoring systems for metastatic spine disease focus on predicting long- to medium-term mortality or a combination of perioperative morbidity and mortality. However, accurate prediction of perioperative mortality alone may be the most important factor when considering surgical intervention. We aimed to develop and evaluate a new tool, the H2-FAILS score, to predict 30-day mortality after surgery for metastatic spine disease. Using the National Surgical Quality Improvement Program database, we identified 1195 adults who underwent surgery for metastatic spine disease from 2010 to 2018. Incidence of 30-day mortality was 8.7% (n = 104). Independent predictors of 30-day mortality were used to derive the H2-FAILS score. H2-FAILS is an acronym for: Heart failure (2 points), Functional dependence, Albumin deficiency, International normalized ratio elevation, Leukocytosis, and Smoking (1 point each). Discrimination was assessed using area under the receiver operating characteristic curve (AUC). The H2-FAILS score was compared with the American Society of Anesthesiologists Physical Status Classification (ASA Class), the 5-item modified Frailty Index (mFI-5), and the New England Spinal Metastasis Score (NESMS). Internal validation was performed using bootstrapping. Alpha = 0.05. Predicted 30-day mortality was 1.8% for an H2-FAILS score of 0 and 78% for a score of 6. AUC of the H2-FAILS was 0.77 (95% confidence interval: 0.72-0.81), which was higher than the mFI-5 (AUC 0.58, p < 0.001), ASA Class (AUC 0.63, p < 0.001), and NESMS (AUC 0.70, p = 0.004). Internal validation showed an optimism-corrected AUC of 0.76. The H2-FAILS score accurately predicts 30-day mortality after surgery for spinal metastasis. Prognostic level III.

P38. Predicting 30-day mortality after surgery for metastatic disease of the spine: the H2-FAILS score

Development of Machine Learning Algorithms for Prediction of Sustained Postoperative Opioid Prescriptions After Total Hip Arthroplasty

Surgery for spinal metastasis is a palliative treatment aimed at improving patient quality of life by alleviating pain and reversing or delaying neurologic dysfunction, but with a mean survival time of less than 1 year and significant complication rates, appropriate patient selection is crucial. To identify the most significant prognostic variables of survival after surgery for spinal metastasis. Chart review was performed on 200 surgically treated spinal metastasis patients at Stanford Hospital between 1999 and 2009. Survival analysis was performed and variables entered into a Cox proportional hazards model to determine their significance. Median overall survival was 8.0 months, with a 30-day mortality rate of 3.0% and a 30-day complication rate of 34.0%. A Cox proportional hazards model showed radiosensitivity of the tumor (hazard ratio: 2.557, P<.001), preoperative ambulatory status (hazard ratio: 2.355, P=.0001), and Charlson Comorbidity Index (hazard ratio: 2.955, P<.01) to be significant predictors of survival. Breast cancer had the best prognosis (median survival, 27.1 months), whereas gastrointestinal tumors had the worst (median survival, 2.66 months). We identified the Charlson Comorbidity Index score as one of the strongest predictors of survival after surgery for spinal metastasis. We confirmed previous findings that radiosensitivity of the tumor and ambulatory status are significant predictors of survival.