External Validation Population Research Articles

Prediction of coronary artery lesions in children with Kawasaki syndrome based on machine learning

ObjectiveKawasaki syndrome (KS) is an acute vasculitis that affects children < 5 years of age and leads to coronary artery lesions (CAL) in about 20-25% of untreated cases. Machine learning (ML) is a branch of artificial intelligence (AI) that integrates complex data sets on a large scale and uses huge data to predict future events. The purpose of the present study was to use ML to present the model for early risk assessment of CAL in children with KS by different algorithms.MethodsA total of 158 children were enrolled from Women and Children’s Hospital, Qingdao University, and divided into 70–30% as the training sets and the test sets for modeling and validation studies. There are several classifiers are constructed for models including the random forest (RF), the logistic regression (LR), and the eXtreme Gradient Boosting (XGBoost). Data preprocessing is analyzed before applying the classifiers to modeling. To avoid the problem of overfitting, the 5-fold cross validation method was used throughout all the data.ResultsThe area under the curve (AUC) of the RF model was 0.925 according to the validation of the test set. The average accuracy was 0.930 (95% CI, 0.905 to 0.956). The AUC of the LG model was 0.888 and the average accuracy was 0.893 (95% CI, 0,837 to 0.950). The AUC of the XGBoost model was 0.879 and the average accuracy was 0.935 (95% CI, 0.891 to 0.980).ConclusionThe RF algorithm was used in the present study to construct a prediction model for CAL effectively, with an accuracy of 0.930 and AUC of 0.925. The novel model established by ML may help guide clinicians in the initial decision to make a more aggressive initial anti-inflammatory therapy. Due to the limitations of external validation and regional population characteristics, additional research is required to initiate a further application in the clinic.

Radiomics and artificial intelligence for risk stratification of pulmonary nodules: Ready for primetime?

Pulmonary nodules are ubiquitously found on computed tomography (CT) imaging either incidentally or via lung cancer screening and require careful diagnostic evaluation and management to both diagnose malignancy when present and avoid unnecessary biopsy of benign lesions. To engage in this complex decision-making, clinicians must first risk stratify pulmonary nodules to determine what the best course of action should be. Recent developments in imaging technology, computer processing power, and artificial intelligence algorithms have yielded radiomics-based computer-aided diagnosis tools that use CT imaging data including features invisible to the naked human eye to predict pulmonary nodule malignancy risk and are designed to be used as a supplement to routine clinical risk assessment. These tools vary widely in their algorithm construction, internal and external validation populations, intended-use populations, and commercial availability. While several clinical validation studies have been published, robust clinical utility and clinical effectiveness data are not yet currently available. However, there is reason for optimism as ongoing and future studies aim to target this knowledge gap, in the hopes of improving the diagnostic process for patients with pulmonary nodules.

Merging machine learning and patient preference: a novel tool for risk prediction of percutaneous coronary interventions.

Predicting personalized risk for adverse events following percutaneous coronary intervention (PCI) remains critical in weighing treatment options, employing risk mitigation strategies, and enhancing shared decision-making. This study aimed to employ machine learning models using pre-procedural variables to accurately predict common post-PCI complications. A group of 66 adults underwent a semiquantitative survey assessing a preferred list of outcomes and model display. The machine learning cohort included 107 793 patients undergoing PCI procedures performed at 48 hospitals in Michigan between 1 April 2018 and 31 December 2021 in the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) registry separated into training and validation cohorts. External validation was conducted in the Cardiac Care Outcomes Assessment Program database of 56 583 procedures in 33 hospitals in Washington. Overall rate of in-hospital mortality was 1.85% (n = 1999), acute kidney injury 2.51% (n = 2519), new-onset dialysis 0.44% (n = 462), stroke 0.41% (n = 447), major bleeding 0.89% (n = 942), and transfusion 2.41% (n = 2592). The model demonstrated robust discrimination and calibration for mortality {area under the receiver-operating characteristic curve [AUC]: 0.930 [95% confidence interval (CI) 0.920-0.940]}, acute kidney injury [AUC: 0.893 (95% CI 0.883-0.903)], dialysis [AUC: 0.951 (95% CI 0.939-0.964)], stroke [AUC: 0.751 (95%CI 0.714-0.787)], transfusion [AUC: 0.917 (95% CI 0.907-0.925)], and major bleeding [AUC: 0.887 (95% CI 0.870-0.905)]. Similar discrimination was noted in the external validation population. Survey subjects preferred a comprehensive list of individually reported post-procedure outcomes. Using common pre-procedural risk factors, the BMC2 machine learning models accurately predict post-PCI outcomes. Utilizing patient feedback, the BMC2 models employ a patient-centred tool to clearly display risks to patients and providers (https://shiny.bmc2.org/pci-prediction/). Enhanced risk prediction prior to PCI could help inform treatment selection and shared decision-making discussions.

Establishment and validation of systematic prognostic nomograms in patients over 60 years of age with osteosarcoma: A multicenter external verification study.

The aim of this study was to develop and validate systematic nomograms to predict cancer specific survival (CSS) and overall survival (OS) in osteosarcoma patients aged over 60 years. We used data from the Surveillance, Epidemiology, and End Results (SEER) database and identified 982 patients with osteosarcoma over 60 years of age diagnosed between 2004 and 2015. Overall, 306 patients met the requirements for the training group. Next, we enrolled 56 patients who met the study requirements from multiple medical centers as the external validation group to validate and analyze our model. We collected all available variables and finally selected eight that were statistically associated with CSS and OS through Cox regression analysis. Integrating the identified variables, we constructed 3- and 5-year OS and CSS nomograms, respectively, which were further evaluated by calculating the C-index. A calibration curve was used to evaluate the accuracy of the model. Receiver operating characteristic (ROC) curves measured the predictive capacity of the nomograms. The Kaplan-Meier analysis was used for all patient-based variables to explore the influence of various factors on patient survival. Finally, a decision curve analysis (DCA) curve was used to analyze whether our model would be suitable for application in clinical practice. Cox regression analysis of clinical variables identified age, sex, marital status, tumor grade, tumor laterality, tumor size, M-stage, and surgical treatment as prognostic factors. Nomograms showed good predictive capacity for OS and CSS. We calculated that the C-index of the OS nomogram of the training population was 0.827 (95% CI 0.778-0.876), while that of the CSS nomogram was 0.722 (95% CI 0.665-0.779). The C-index of the OS nomogram evaluated on the external validation population was 0.716 (95% CI 0.575-0.857), while that of the CSS nomogram was 0.642 (95% CI 0.50-0.788). Furthermore, the calibration curve of our prediction models indicated the nomograms could accurately predict patient outcome. The constructed nomogram is a useful tool for accurately predicting OS and CSS at 3 and 5 years for patients over 60 years of age with osteosarcoma and can assist clinicians in making appropriate decisions in practice.

Validation of prediction models for radiation-induced late rectal bleeding: Evidence from a large pooled population of prostate cancer patients

PurposeTo validate published models for the risk estimate of grade ≥ 1 (G1+), grade ≥ 2 (G2+) and grade = 3 (G3) late rectal bleeding (LRB) after radical radiotherapy for prostate cancer in a large pooled population from three prospective trials. Materials and methodsThe external validation population included patients from Europe, and Oceanian centres enrolled between 2003 and 2014. Patients received 3DCRT or IMRT at doses between 66–80 Gy. IMRT was administered with conventional or hypofractionated schemes (2.35–2.65 Gy/fr). LRB was prospectively scored using patient-reported questionnaires (LENT/SOMA scale) with a 3-year follow-up.All Normal Tissue Complication Probability (NTCP) models published until 2021 based on the Equivalent Uniform Dose (EUD) from the rectal Dose Volume Histogram (DVH) were considered for validation.Model performance in validation was evaluated through calibration and discrimination. ResultsSixteen NTCP models were tested on data from 1633 patients. G1+ LRB was scored in 465 patients (28.5%), G2+ in 255 patients (15.6%) and G3 in 112 patients (6.8%). The best performances for G2+ and G3 LRB highlighted the importance of the medium–high doses to the rectum (volume parameters n = 0.24 and n = 0.18, respectively). Good performance was seen for models of severe LRB. Moreover, a multivariate model with two clinical factors found the best calibration slope. ConclusionFive published NTCP models developed on non-contemporary cohorts were able to predict a relative increase in the toxicity response in a more recent validation population. Compared to QUANTEC findings, dosimetric results pointed toward mid-high doses of rectal DVH. The external validation cohort confirmed abdominal surgery and cardiovascular diseases as risk factors.

A clinical indicator-based prognostic model predicting treatment outcomes of pulmonary tuberculosis: a prospective cohort study

ObjectivesIdentifying prognostic factors helps optimize the treatment regimen and promote favorable outcomes. We conducted a prospective cohort study on patients with pulmonary tuberculosis to construct a clinical indicator-based model and estimate its performance.MethodsWe performed a two-stage study by recruiting 346 pulmonary tuberculosis patients diagnosed between 2016 and 2018 in Dafeng city as the training cohort and 132 patients diagnosed between 2018 and 2019 in Nanjing city as the external validation population. We generated a risk score based on blood and biochemistry examination indicators by the least absolute shrinkage and selection operator (LASSO) Cox regression. Univariate and multivariate Cox regression models were used to assess the risk score, and the strength of association was expressed as the hazard ratio (HR) and 95% confidence interval (CI). We plotted the receiver operating characteristic (ROC) curve and calculated the area under the curve (AUC). Internal validation was conducted by 10-fold cross-validation.ResultsTen significant indicators (PLT, PCV, LYMPH, MONO%, NEUT, NEUT%, TBTL, ALT, UA, and Cys-C) were selected to generate the risk score. Clinical indicator-based score (HR: 10.018, 95% CI: 4.904–20.468, P < 0.001), symptom-based score (HR: 1.356, 95% CI: 1.079–1.704, P = 0.009), pulmonary cavity (HR: 0.242, 95% CI: 0.087–0.674, P = 0.007), treatment history (HR: 2.810, 95% CI: 1.137–6.948, P = 0.025), and tobacco smoking (HR: 2.499, 95% CI: 1.097–5.691, P = 0.029) were significantly related to the treatment outcomes. The AUC was 0.766 (95% CI: 0.649–0.863) in the training cohort and 0.796 (95% CI: 0.630–0.928) in the validation dataset.ConclusionIn addition to the traditional predictive factors, the clinical indicator-based risk score determined in this study has a good prediction effect on the prognosis of tuberculosis.

Risk Score for Prediction of Acute Kidney Injury in Patients with Acute ST-Segment Elevation Myocardial Infarction.

Acute kidney injury (AKI) is an important comorbidity of ST-Segment Elevation Myocardial Infarction (STEMI) and worsens the prognosis. The purpose of this study was to investigate the relationship between clinical data, test results, surgical findings, and AKI in STEMI patients and to develop a simple, practical model for predicting the risk of AKI. Prognostic prediction research with clinical risk score development was conducted. The data used for model development was derived from the database of the Henan Province Cardiovascular Disease Clinical Data and Sample Resource Bank Engineering Research Center. The data used for external validation was derived from the China Chest Pain Center database. The study endpoint was defined as the occurrence of AKI. Logistic regression analysis was used to identify independent predictors of AKI. Logistic coefficients of each predictor were used for score weighting and transformation. The predictive performance of the newly derived risk scores was validated, respectively, by receiver operating characteristic (ROC) regression in the development population and the external validation population. A total of 364 patients, 57 (15.6%) with AKI and 307 (84.4%) without AKI, were included for score derivation. The validation crowd includes 88 STEMI patients in different institutions. A total of 11 potential predictors were explored under the multivariable logistic regression model. The new risk score was based on five final predictors which were age > 72 years, ejection fraction of no more than 40%, baseline serum creatinine > 102.7 mmol/L, red blood cell distribution width > 13.15, and culprit lesion located in the middistal segment. With only five predictor variables, the score predicted the risk of AKI with the effective discriminative ability (area under the receiver operating characteristic curve (AuROC): 0.721, 95% confidence interval (CI): 0.652-0.790). In the external validation, the newly developed score confirmed a similar discrimination as the crowd used for derivation (AuROC: 0.731, 95% CI: 0.624-0.838). The newly developed score was proved to have good predictive performance and could be practically applied for risk stratification of AKI in STEMI patients.

Development and validation of a simple tool composed of items on dyspnea, respiration rates, and C-reactive protein for pneumonia prediction among acute febrile respiratory illness patients in primary care settings

BackgroundAcute febrile respiratory illness (AFRI) patients are susceptible to pneumonia and suffer from significant morbidity and mortality throughout the world. In primary care settings, the situation is worse. Limited by computerized tomography resources and physician experiences, AFRI patients in primary care settings may not be diagnosed appropriately, which would affect following treatment. In this study, we aimed to develop and validate a simple prediction model to help physicians quickly identify AFRI patients of pneumonia risk in primary care settings.MethodsA total of 1977 AFRI patients were enrolled at two fever clinics in Shanghai, China, and among them, 727 patients who underwent CT scans were included in the analysis. Acute alveolar or interstitial infiltrates found on CT images were diagnosed with pneumonia. Characteristics and blood parameters were compared between pneumonia and non-pneumonia patients. Then a multivariable model for pneumonia prediction was developed through logistic regression analysis. Its value for pneumonia prediction was prospectively assessed in an external multi-center population, which included 1299 AFRI patients in primary settings from 5 different provinces throughout China.ResultsIn the model development population, pneumonia patients (n = 227) had a longer duration of fever; higher frequencies of purulent sputum, dyspnea, and thoracic pain; and higher levels of respiration rates and C-reactive protein (CRP) than non-pneumonia patients (n = 500). Logistic regression analysis worked out a model composed of items on dyspnea, respiration rates > 20/min, and CRP > 20 mg/l (DRC) for pneumonia prediction with an area under curve (AUC) of 0.8506. In the external validation population, the predictive accuracy of the DRC model was the highest when choosing at least one positive item (1 score) as a cut-off point with a sensitivity of 87.0% and specificity of 80.5%. DRC scores increased with pneumonia severity and lung lobe involvement and showed good performance for both bacterial and viral pneumonia. For viral pneumonia, dyspnea plus respiration rates > 20/min had good predictive capacity regardless of CRP concentration.ConclusionsDRC model is a simple tool that predicts pneumonia among AFRI patients, which would help physicians utilize medical resources rationally in primary care settings.

Concurrent external validation of bloodstream infection probability models

ObjectiveAccurately estimating the likelihood of bloodstream infection (BSI) can help clinicians make diagnostic and therapeutic decisions. Many multivariate models predicting BSI probability have been published. This study measured the performance of BSI probability models within the same patient sample. MethodsWe retrieved validated BSI probability models included in a recently published systematic review that returned a patient-level BSI probability for adults. Model applicability, discrimination, and accuracy was measured in a simple random sample of 4485 admitted adults having blood cultures ordered in the emergency department or the initial 48 hours of hospitalization. ResultsTen models were included (publication years 1991–2015). Common methodological threats to model performance included overfitting and continuous variable categorization. Restrictive inclusion criteria caused seven models to apply to <15% of validation patients. Model discrimination was less than originally reported in derivation groups (median c-statistic 60%, range 48–69). The observed BSI risk frequently deviated from expected (median integrated calibration index 4.0%, range 0.8–12.4). Notable disagreement in expected BSI probabilities was seen between models (median (25th–75th percentile) relative difference between expected risks 68.0% (28.6–113.6%)). DiscussionIn a large randomly selected external validation population, many published BSI probability models had restricted applicability, limited discrimination and calibration, and extensive inter-model disagreement. Direct comparison of model performance is hampered by dissimilarities between model-specific validation groups.

Deep learning to detect anterior cruciate ligament tear on knee MRI: multi-continental external validation.

To develop a deep-learning algorithm for anterior cruciate ligament (ACL) tear detection and to compare its accuracy using two external datasets. A database of 19,765 knee MRI scans (17,738 patients) issued from different manufacturers and magnetic fields was used to build a deep learning-based ACL tear detector. Fifteen percent showed partial or complete ACL rupture. Coronal and sagittal fat-suppressed proton density or T2-weighted sequences were used. A Natural Language Processing algorithm was used to automatically label reports associated with each MRI exam. We compared the accuracy of our model on two publicly available external datasets: MRNet, Bien et al, USA (PLoS Med 15:e1002699, 2018); and KneeMRI, Stajduhar et al, Croatia (Comput Methods Prog Biomed 140:151-164, 2017). Receptor operating characteristics (ROC) curves, area under the curve (AUC), sensitivity, specificity, and accuracy were used to evaluate our model. Our neural networks achieved an AUC value of 0.939 for detection of ACL tears, with a sensitivity of 87% (0.875) and a specificity of 91% (0.908). After retraining our model on Bien dataset and Stajduhar dataset, our algorithm achieved AUC of 0.962 (95% CI 0.930-0.988) and 0.922 (95% CI 0.875, 0.962) respectively. Sensitivity, specificity, and accuracy were respectively 85% (95% CI 75-94%, 0.852), 89% (95% CI 82-97%, 0.894), 0.875 (95% CI 0.817-0.933) for Bien dataset, and 68% (95% CI 54-81%, 0.681), 93% (95% CI 89-97%, 0.934), and 0.870 (95% CI 0.821-0.913) for Stajduhar dataset. Our algorithm showed high performance in the detection of ACL tears with AUC on two external datasets, demonstrating its generalizability on different manufacturers and populations. This study shows the performance of an algorithm for detecting anterior cruciate ligament tears with an external validation on populations from countries and continents different from the study population. • An algorithm for detecting anterior cruciate ligament ruptures was built from a large dataset of nearly 20,000 MRI with AUC values of 0.939, sensitivity of 87%, and specificity of 91%. • This algorithm was tested on two external populations from different other countries: a dataset from an American population and a dataset from a Croatian population. Performance remains high on these two external validation populations (AUC of 0.962 and 0.922 respectively).

Massive external validation of a machine learning algorithm to predict pulmonary embolism in hospitalized patients

BackgroundPulmonary embolism (PE) is a life-threatening condition associated with ~10% of deaths of hospitalized patients. Machine learning algorithms (MLAs) which predict the onset of pulmonary embolism (PE) could enable earlier treatment and improve patient outcomes. However, the extent to which they generalize to broader patient populations impacts their clinical utility. ObjectiveTo conduct the first large-scale external validation of a machine learning–based PE prediction model which uses EHR data from the first three hours of a patient's hospital stay to predict the occurrence of PE within the next 10 days of the inpatient stay. MethodsThis retrospective study included approximately two million adult hospital admissions across 44 medical institutions in the US from 2011 to 2017. Demographics, vital signs, and lab tests from adult inpatients at 12 institutions (n = 331,268; 3.3% PE positive) were used for training an XGBoost model. External validation of the model was conducted on patient populations from each of 32 medical institutions (total n = 1,660,715; 3.7% PE positive) without retraining. Model performance was assessed using the area under the receiver operating characteristic curve (AUROC). Backward elimination regression was used to identify correlations between characteristics of the external validation sets and AUROC. ResultsThe model performed well (AUROC = 0.87) on the 20% hold-out subset of the training set. Despite demographic differences between the 32 external validation populations (percent PE positive: min = 1.54%, max = 6.47%), without retraining, the model had excellent discrimination, with a mean AUROC of 0.88 (min = 0.79, max = 0.93). Fixing sensitivity at 0.80, the model had a mean specificity of 0.85 (min = 0.64, max = 0.93). Backward elimination regression identified a negative association (β = −0.015, p < 0.001) between the percentage of PE positive encounters and AUROC. ConclusionsA PE prediction model performed remarkably well across 32 different external patient populations without retraining and despite significant differences in demographic characteristics, demonstrating its generalizability and potential as a clinical decision support tool to aid PE detection and improve patient outcomes in a clinical setting.

A Deep Learning-Based Automatic Collateral Assessment in Patients with Acute Ischemic Stroke.

This study aimed to develop a supervised deep learning (DL) model for grading collateral status from dynamic susceptibility contrast magnetic resonance perfusion (DSC-MRP) images from patients with large vessel occlusion (LVO) acute ischemic stroke (AIS) and compare its performance against experts' manual grading. Among consecutive LVO-AIS at three medical center sites, DSC-MRP data were processed to generate collateral flow maps consisting of arterial, capillary, and venous phases. With the use of expert readings as a reference, a DL model was developed to analyze collateral status with output classified into good and poor grades. The resulting model was externally validated in a later-collected population from one medical center site. The model was trained on 255 patients and externally validated on 72 patients. In the all-site internal validation population, DL grading of good collateral probability yielded a c statistic of 0.91; in the external validation population, the c statistic was 0.85. In the external validation population, there was moderate agreement between the experts' grades and DL grades (kappa = 0.53, 95% CI = 0.32-0.73, p < 0.0001). Day 7 infarct growth volume was higher in DL-graded poor collateral group than good collateral group patients (median volume [26mL vs. 6mL], p = 0.01) in patients with successful reperfusion (modified treatment in cerebral infarction (mTICI) = 2b-3). In all patients with a 90-day modified Rankin Scale (mRS) score, there was a shift to more favorable outcomes in the good collateral group, with a common odds ratio of 2.99 (95% CI = 1.89-4.76, p < 0.0001). The DL-based collateral grading was in good agreement with expert manual grading in both development and validation populations. After exclusion of patients with large infarct volume, early reperfusion is more likely to benefit patients with the poor collateral flow, and the DL method has the potential to aid the assessment of collateral status.

Establishment and Validation of a Nomogram for the Risk of New Vertebral Compression Fractures After Percutaneous Vertebroplasty in Patients With Osteoporotic Vertebral Compression Fractures: A Retrospective Study.

PurposeNew vertebral compression fractures(NVCFs) after minimally invasive surgery in patients with osteoporotic vertebral compression fracture (OVCF) is a challenging issue worldwide. Predicting the occurrence of NVCFs is key to addressing such questions. Therefore, we aimed to investigate the risk factors for patients who developed NVCFs after undergoing surgical treatment and establish a nomogram model to reduce the occurrence of NVCFs.MethodsThis study is a retrospective analysis that collected the general characteristics and surgical features of patients who underwent surgical treatment at 2 central institutions between January 2017 and December 2020. Patients were divided into training and testing sets based on the presence or absence of NVCFs. Independent risk factors for NVCFs were obtained in the training set of patients, and then a nomogram model was constructed. Internal and external validation of the nomogram model was performed using the consistency index (C index), receiver operating characteristic curve(ROC), calibration curves, and decision curve analysis (DCA).ResultsA total of 562 patients were included in this study. Patients from the first center were used for nomogram construction and internal validation, and patients from the second center were used as an external validation population. Multivariate regression analysis showed that age, Hounsfield unit (Hu) value, cement leakage, and thoracolumbar (TL) junction fracture were independent risk factors for NVCFs after minimally invasive surgery. The C index was .85, and the validation of internal and external validation shows that the predicted values of the established model is in good agreement with the actual values.ConclusionsIn this study, 4 independent risk factors were obtained by regression analysis, and a nomogram model was constructed to guide clinical work. The application of this model can help surgeons to make more accurate judgments to prevent the occurrence of NVCFs.

External Validation of the Bone Metastases Ensemble Trees for Survival (BMETS) Machine Learning Model to Predict Survival in Patients With Symptomatic Bone Metastases.

The Bone Metastases Ensemble Trees for Survival (BMETS) model uses a machine learning algorithm to estimate survival time following consultation for palliative radiation therapy for symptomatic bone metastases (SBM). BMETS was developed at a tertiary-care, academic medical center, but its validity and stability when applied to external data sets are unknown. Patients treated with palliative radiation therapy for SBM from May 2013 to May 2016 at two hospital-based community radiation oncology clinics were included, and medical records were retrospectively reviewed to collect model covariates and survival time. The Kaplan-Meier method was used to estimate overall survival from consultation to death or last follow-up. Model discrimination was estimated using time-dependent area under the curve (tAUC), which was calculated using survival predictions from BMETS based on the initial training data set. A total of 216 sites of SBM were treated in 182 patients. Most common histologies were breast (27%), lung (23%), and prostate (23%). Compared with the BMETS training set, the external validation population was older (mean age, 67 v 62 years; P < .001), had more primary breast (27% v 19%; P = .03) and prostate cancer (20% v 12%; P = .01), and survived longer (median, 10.7 v 6.4 months). When the BMETS model was applied to the external data set, tAUC values at 3, 6, and 12 months were 0.82, 0.77, and 0.77, respectively. When refit with data from the combined training and external validation sets, tAUC remained > 0.79. BMETS maintained high discriminative ability when applied to an external validation set and when refit with new data, supporting its generalizability, stability, and the feasibility of dynamic modeling.

Abstract 9078: The Bad Bleeding Score: A Simple Score to Assess Bleeding Risk in Atrial Fibrillation Patients With Direct Oral Anticoagulants

Backgrounds: Therapeutic decisions in atrial fibrillation (AF) are often influenced by assessment of bleeding risk. However, existing bleeding risk scores have limitations. Methods and Results: We retrospectively studied 924 AF patients who were prescribed with direct oral anticoagulants (DOAC) in a single center cohort between May 2011 and July 2015. Using Cox proportional hazards regression, 3 factors were identified to be associated with bleeding among those patients over 1.9 year-follow up. The predictive performance of 3-item score (BAD score: the sum of the point values of the following parameters; bleeding history, age (≥75yrs) and dual antiplatelet therapy) and 3 existing risk scores (HAS-BLED, ORBIT, and ATRIA) were then assessed in the sequel cohort between Aug 2015 and Jun 2016. BAD score had good ability to identify those who bled in an external validation population (C-index: BAD score, 0.63; HAS-BLED, 0.61; ORBIT, 0.64; ATRIA, 0.59). The risk of bleeding increased with higher BAD score. Conclusions: BAD score might be a useful and simple bleeding risk score in AF patients with DOAC.

Corneal Tomography Multivariate Index (CTMVI) effectively distinguishes healthy corneas from those susceptible to ectasia

Background:Keratoconus (KC) is a multifactorial progressive corneal disease, pathogenically characterized by genetic, biochemical, and environmental key factors. Nevertheless, current tools used for its diagnosis, such as Pentacam, examine the posterior and anterior surfaces of the cornea based on a rotating Scheimpflug camera and provide, as output, a set of features for which the clinical interpretation has been error-prone. Objective:Consequently, the objective of this study is to describe a novel Artificial Intelligence (AI) strategy that receives subsets of the 52 original Pentacam tomographic descriptors as input and produces, as output, the scalar value called Corneal Tomography Multivariate Index (CTMVI). Specifically based on Paraconsistent Feature Engineering (PFE) and Support Vector Machine (SVM), it intends to help clinicians in the pre-diagnosis of cornea ectasia risk assessment. Methods:We collected data from patients’ eyes in three different classes: 411 with healthy corneas to compose the control group (CG), 302 with KC to represent the KC group (KCG) and, additionally, 64 in a group with very asymmetric ectasia but with normal corneal topography (VAE-NTG) in one eye, totaling 777 samples. A set of features from the patients’ corneal tomographic assessments was extracted to serve as input to the paraconsistent feature selector algorithm. Once it provides the best subsets of features representing the classes, a one-against-all SVM-based algorithm was successfully used to classify those groups. Notably, normal topography criteria (NTPC) were considered based on the steepest front curvature (Kmax) <47.2 diopters, a paracentral inferior–superior (I–S value) <1.45, and a keratoconus percentage index (KISA%) score <60. Normal corneal tomography (NTMC) consists of an anterior chamber depth <3.8μm, a front corneal apical elevation <4μm, a front corneal elevation (FCE) at the thinnest point <3.8μm, and an FCE <12μm in the corneal 4.0 mm center. Respective back corneal elevation values were 7μm, 13μm, and 25μm. To assess the clinical validity of the models and their abilities to correctly classify new data, two procedures were used: the first was a holdout cross validation, and the second was the external validation consisting of an independent test with samples that were not part of the training data. Results:Notably, PFE-based analysis effectively found dominant representative features which allowed for relevant classification accuracies based on SVM. Among numerous results, a parallel test combining CTMVI and existing Pentacam Random Forest Index (PRFI) produced a value of sensitivity of 0.99 and a value of specificity of 0.84, using 50% of the dataset for system modeling and the remaining 50% for assessment, improving current reports found in the literature. CTMVI, BAD-D, and PRFI showed an accuracy (acc) of 0.89, 0.86, and 0.89, respectively, in the external validation population to distinguish the healthy eyes from subclinical cases. The three indexes resulted in 0.99 acc when used to distinguish healthy eyes from corneal ectasia ones. CTMVI showed 100% sensitivity and 100% specificity between CG and KCG (cutoff ⩽−1.36). Regarding VAE-NTG patients, CTMVI had an area under the curve (AUC) of 0.935 (87.5% sensitivity; 84.95% specificity; cutoff ⩽0.639), which was higher than those of Belin/Ambrósio Enhanced Ectasia Total Deviation Value (BAD D) with AUC 0.765; 78.12% sensitivity; 65.05% specificity; p = .0001, and PRFI with AUC 0.805; 59.38% sensitivity; 90.78% specificity; p = .0004. During the external validation, it was shown to be as effective as BAD and PRFI. Conclusions:Based on the innovative joint application of PFE and SVM to distinguish healthy corneas from those pathologically-affected, we provided an original contribution to the state-of-the-art in terms of AI-aided diagnosis. Therefore, CTMVI improves the detection of corneal ectasia susceptibility using tomographic-derived descriptors. This reassures the efficacy of the proposed approach.

Untying the Gordian knot of sex and heart failure therapy

Untying the Gordian knot of sex and heart failure therapy

Effect of competing mortality risks on predictive performance of the QRISK3 cardiovascular risk prediction tool in older people and those with comorbidity: external validation population cohort study

SummaryBackgroundPrimary prevention of cardiovascular disease (CVD) is guided by risk-prediction tools, but these rarely account for the risk of dying from other conditions (ie, competing mortality risk). In England and Wales, the recommended risk-prediction tool is QRISK2, and a new version (QRISK3) has been derived and internally validated. We aimed to externally validate QRISK3 and to assess the effects of competing mortality risk on its predictive performance.MethodsFor this retrospective population cohort study, we used data from the Clinical Practice Research Datalink. We included patients aged 25–84 years with no previous history of CVD or statin treatment who were permanently registered with a primary care practice, had up-to-standard data for at least 1 year, and had linkage to Hospital Episode Statistics discharge and Office of National Statistics mortality data. We compared the QRISK3-predicted 10-year CVD risk with the observed 10-year risk in the whole population and in important subgroups of age and multimorbidity. QRISK3 discrimination and calibration were examined with and without accounting for competing risks.FindingsOur study population included 1 484 597 women with 42 451 incident CVD events (4·9 cases per 1000 person-years of follow-up, 95% CI 4·89–4·99), and 1 420 176 men with 53 066 incident CVD events (6·7 cases per 1000 person-years, 6·66–6·78), with median follow-up of 5·0 years (IQR 1·9–9·2). Non-CVD death rose markedly with age (0·4% of women and 0·5% of men aged 25–44 years had a non-CVD death vs 20·1% of women and 19·6% of men aged 75–84 years). QRISK3 discrimination in the whole population was excellent (Harrell's C-statistic 0·865 in women and 0·834 in men) but was poor in older age groups (<0·65 in all subgroups aged 65 years or older). Ignoring competing risks, QRISK3 calibration in the whole population and in younger people was excellent, but there was significant over-prediction in older people. Accounting for competing risks, QRISK3 systematically over-predicted CVD risk, particularly in older people and in those with high multimorbidity.InterpretationQRISK3 performed well at the whole population level when ignoring competing mortality risk. The tool performed considerably less well in important subgroups, including older people and people with multimorbidity, and less well again after accounting for competing mortality risk.FundingNational Institute for Health Research.

Early risk stratification of acute type A aortic dissection: development and validation of a predictive score.

The performance of published preoperative risk scores for acute type A aortic dissection (aTAAD) is suboptimal. So, the predictive power of these scores were externally validated in order to develop and validate a more reliable preoperative score for identification of patients at high risk of mortality. Potential preoperative risk variables of consecutively admitted patients with aTAAD were prospectively collected. Seven published risk scores were validated with our dataset. For derivation and internal validation, the original population was divided at a ratio of 7:3. Logistic regression was used to identify variables for the new score. A 50-patient retrospective dataset was used for external validation. The predictive accuracy for post-operative mortality was evaluated using the area under the receiver operating characteristic (AUROC) curve. During the study period, 225 patients with aTAAD were admitted preoperatively. Of these, 209 underwent surgical repair and 29 died postoperatively. The AUROCs of the seven published pre-operative risk scores for post-operative mortality ranged from 0.57 to 0.77. Four variables were derived for the new score system, i.e., Acute myocardial ischemia, Lactate, Iliac arteries involved, and CreatininE (the ALICE score). The AUROCs for post-operative mortality in the derivation, internal and external validation populations were 0.85, 0.88 and 0.83, respectively. At a cutoff value of 3, the ALICE score for post-operative mortality had a sensitivity of 71% to 88% and specificity of 78% to 86%. The ALICE score comprising four components might help bedside clinicians in early detection of the most severe aTAAD patients.

External validation of prognostic models: what,why, how, when and where?

Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model’s reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.