Poor Calibration Research Articles

Abstract 4143270: PREDICTIVE MODELS AID PHYSICIAN PROGNOSTICATION: A SECONDARY ANALYSIS EVALUATING INTEGRATED MODEL AND PHYSICIAN PROGNOSTIC ESTIMATES IN PATIENTS WITH HEART FAILURE WITH REDUCED EJECTION FRACTION

Background: In recent studies from a multicenter Canadian cohort of outpatients with heart failure (HF), we found that model predictions were significantly more accurate than HF cardiologists. In this study, trying to mimic practice, we evaluated the additional predictive value and clinical impact of model predictions to refine physician estimated risk of 1-year mortality by combining model and physician estimates. Methods: We included consented consecutive HF outpatients (LVEF <40%) followed at 11 HF clinics in Canada. HF cardiologists estimated patient 1-year mortality using their clinical judgment. We calculated model predicted mortality using the Seattle HF Model (SHFM). We followed patients for at least a year to record mortality (or urgent heart transplant or ventricular assist device implant as mortality-equivalent events). Using random forest survival model and cross-validation, we compared the performance SHFM and the HF cardiologist alone, and the integrated HF cardiologist and the SHFM predictions by evaluating model discrimination (c-statistic), calibration (observed vs predicted event rate), risk reclassification and clinical net benefit analyses. Results: Among 1,643 HF patients, 1-year event rate was 9% (95%CI 8%-11%). The SHFM had the adequate discrimination (c-statistic 0.76) and excellent calibration while cardiologists showed adequate discrimination (c-statistic 0.75) and poor calibration with significant risk overestimation ( Figure 1 ). When the SHFM estimates were added physician predictions, discrimination significantly improved (0.82, 95%CI 0.78-0.86) with excellent calibration. By risk reclassification analysis, among patients with events, HF cardiologist better reclassified 44% than the SHFM or the integrated model. Among patients without event, however, HF cardiologists worse risk-classified 52% in comparison to SHFM and 71% to the integrated model. By net clinical benefit analysis ( Figure 2 ), when the decision to treat involves patients with 1-year mortality of >5%, SHFM predictions would lead to higher benefit than guiding care by physician judgement. Integrating model and HF cardiologist predictions led to minimally increased benefit in comparison to SHFM alone. Conclusions: Integrating prediction from the SHFM to physician judgment or using the SHFM alone showed superior accuracy than HF cardiologist predictions, proving that model-informed care may provide more accurate prognostic information to tailor clinical decision making.

Development of a prediction model for clinically-relevant fatigue: a multi-cancer approach.

Fatigue is the most prevalent symptom across cancer types. To support clinicians in providing fatigue-related supportive care, this study aims to develop and compare models predicting clinically relevant fatigue (CRF) occurring between two and three years after diagnosis, and to assess the validity of the best-performing model across diverse cancer populations. Patients with non-metastatic bladder, colorectal, endometrial, ovarian, or prostate cancer who completed a questionnaire within three months after diagnosis and a subsequent questionnaire between two and three years thereafter, were included. Predictor variables included clinical, socio-demographic, and patient-reported variables. The outcome was CRF (EORTC QLQC30 fatigue ≥ 39). Logistic regression using LASSO selection was compared to more advanced Machine Learning (ML) based models, including Extreme gradient boosting (XGBoost), support vector machines (SVM), and artificial neural networks (ANN). Internal-external cross-validation was conducted on the best-performing model. 3160 patients were included. The logistic regression model had the highest C-statistic (0.77) and balanced accuracy (0.65), both indicating good discrimination between patients with and without CRF. However, sensitivity was low across all models (0.22-0.37). Following internal-external validation, performance across cancer types was consistent (C-statistics 0.73-0.82). Although the models' discrimination was good, the low balanced accuracy and poor calibration in the presence of CRF indicates a relatively high likelihood of underdiagnosis of future CRF. Yet, the clinical applicability of the model remains uncertain. The logistic regression performed better than the ML-based models and was robust across cohorts, suggesting an advantage of simpler models to predict CRF.

Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation without the selected completely at random assumption

Positive and unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the selected completely at random (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, α, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, α, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate α, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates α for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.

IMPACT OF DNA METHYLATION PROfiLING IN REfiNING MENINGIOMA RECURRENCE RISK STRATIfiCATION AND CLINICAL FOLLOW UP

Abstract AIMS To assess clinical impact of the additive information gained from Integrated Meningioma Risk Score on the risk of meningioma recurrence. METHOD DNA methylation (Illumina EPIC Array) analysis was performed on all recurrent meningioma tumours, primary tumours CNS5 WHO grade 2, 3 and grade 1 with unusual morphological features from Dec 2021 to Jan 2024. DNA methylation analysis was performed, and data files were processed on the DKFZ Heidelberg Classifier v12.5/6 (molecularneuropathology.org). The output from the classifier including methylation class combined with WHO Grade and chromosomal loss (1p, 6q,14q) was used to produce an integrated meningioma risk score for recurrence. RESULTS 70 meningioma cases were profiled. This included 68 cranial meningiomas, ((WHO grade 1, n=19; recurrences=7), (grade 2, n=47; recurrences =9), (grade 3 n=2; recurrences=1)) and 2 spinal meningiomas (grade 1 n=1, grade2 n=1). 3 tumours were excluded due to poor calibration scores. Of the 67 cases the diagnosis was revised in 1 case following DNA methylation analysis due to presence of CDKN2A/B loss. Molecular-morphologic integrated risk score was calculated for all 67 cases. In 10% of overall patients the predicted risk of meningioma recurrence was increased to either intermediate or high, whilst 32% had the risk lowered mainly from intermediate to low. Interestingly, 1 patient who underwent a gross total resection of a WHO Grade 2 meningioma, experienced recurrence with the integrated meningioma risk score indicating low risk of recurrence, which is contrary to the clinical aggressiveness of the tumour. CONCLUSION Meningioma integrated risk stratification by DNA methylation profiling has shown concordance of integrated risk score with WHO Grade in 58% of cases. In 42% of cases the predicted risk was altered when compared to the WHO Grade alone. These results need to be further validated with long term data on patient outcomes.

Large language model uncertainty proxies: discrimination and calibration for medical diagnosis and treatment.

The inability of large language models (LLMs) to communicate uncertainty is a significant barrier to their use in medicine. Before LLMs can be integrated into patient care, the field must assess methods to estimate uncertainty in ways that are useful to physician-users. Evaluate the ability for uncertainty proxies to quantify LLM confidence when performing diagnosis and treatment selection tasks by assessing the properties of discrimination and calibration. We examined confidence elicitation (CE), token-level probability (TLP), and sample consistency (SC) proxies across GPT3.5, GPT4, Llama2, and Llama3. Uncertainty proxies were evaluated against 3 datasets of open-ended patient scenarios. SC discrimination outperformed TLP and CE methods. SC by sentence embedding achieved the highest discriminative performance (ROC AUC 0.68-0.79), yet with poor calibration. SC by GPT annotation achieved the second-best discrimination (ROC AUC 0.66-0.74) with accurate calibration. Verbalized confidence (CE) was found to consistently overestimate model confidence. SC is the most effective method for estimating LLM uncertainty of the proxies evaluated. SC by sentence embedding can effectively estimate uncertainty if the user has a set of reference cases with which to re-calibrate their results, while SC by GPT annotation is the more effective method if the user does not have reference cases and requires accurate raw calibration. Our results confirm LLMs are consistently over-confident when verbalizing their confidence (CE).

An evaluation of source-blending impact on the calibration of SKA EoR experiments

ABSTRACT Twenty-one-centimetre signals from the Epoch of Reionization (EoR) are expected to be detected in the low-frequency radio window by the next-generation interferometers, particularly the Square Kilometre Array (SKA). However, precision data analysis pipelines are required to minimize the systematics within an infinitesimal error budget. Consequently, there is a growing need to characterize the sources of errors in EoR analysis. In this study, we identify one such error origin, namely source blending, which is introduced by the overlap of objects in the densely populated observing sky under SKA1-Low’s unprecedented sensitivity and resolution, and evaluate its two-fold impact in both the spatial and frequency domains using a novel hybrid evaluation (HEVAL) pipeline combining end-to-end simulation with an analytic method to mimic EoR analysis pipelines. Sky models corrupted by source blending induce small but severe frequency-dependent calibration errors when coupled with astronomical foregrounds, impeding EoR parameter inference with strong additive residuals in the two-dimensional power spectrum space. We report that additive residuals from poor calibration against sky models with blending ratios of 5 and 0.5 per cent significantly contaminate the EoR window. In contrast, the sky model with a 0.05 per cent blending ratio leaves little residual imprint within the EoR window, therefore identifying a blending tolerance at approximately 0.05 per cent. Given that the SKA observing sky is estimated to suffer from an extended level of blending, strategies involving de-blending, frequency-dependent error mitigation, or a combination of both, are required to effectively attenuate the calibration impact of source-blending defects.

Framework to process vehicles uncertain locations for intelligent transportation

Many factors affect the precision and accuracy of location data. These factors include, but not limited to, environmental obstructions (e.g., high buildings and forests), hardware issues (e.g., malfunctioning and poor calibration), and privacy concerns (e.g., users denying consent to fine-grained location tracking). These factors lead to uncertainty about users’ location which in turn affects the quality of location-aware services. This paper proposes a novel framework called UMove, which stands for uncertain movements, to manage the trajectory of moving objects under location uncertainty. The UMove framework employs the connectivity (i.e., links between edges) and constraints (i.e., travel time and distance) on road network graphs to reduce the uncertainty of the object’s past, present, and projected locations. To accomplish this, UMove incorporates (i) a set-based pruning algorithm to reduce or eliminate uncertainty from imprecise trajectories; and (ii) a wrapper that can extend user-defined probability models designed to predict future locations of moving objects under uncertainty. Intensive experimental evaluations based on real data sets of GPS trajectories collected by Didi Chuxing in China prove the efficiency of the proposed UMove framework. In terms of accuracy, for past exact-location inference, UMove achieves rates from 88% to 97% for uncertain regions with sizes of 75 meters and 25 meters respectively; for future exact-location inference, accuracy rates reach up to 72% and 82% for 75 meters and 25 meters of uncertain regions.

Assessing a prediction model for depression risk using an early adolescent sample with self‐reported depression

AbstractBackgroundMajor depressive disorder (MDD) in adolescence is a risk factor for poor physical and psychiatric outcomes in adulthood, with earlier age of onset associated with poorer outcomes. Identifying Depression Early in Adolescence Risk Score (IDEA‐RS) is a model for predicting MDD in youth aged >15 years, but replication in younger samples (<15 years) is lacking. Here, we tested IDEA‐RS in a younger sample (9–11 years) to assess whether IDEA‐RS could be applied to earlier onset depression.MethodsWe applied IDEA‐RS predictor weights to 9854 adolescents (9–11 years) from the Adolescent Brain Cognitive Development (ABCD) Study, United States. We derived incident depression outcomes from self‐reported data at 2‐year follow‐up (11–13 years): incident MDD and increase in depression symptoms (DS). Sensitivity analyses were conducted using parent‐reported data. We assessed accuracy and calibration in predicting self‐reported incident depression and compared this to a refitted model with predictor weights derived in ABCD. Lastly, we tested associations between IDEA‐RS predictors and self‐reported incident depression.ResultsExternal replication yielded better‐than‐chance discriminative capacity for self‐reported incident depression (MDD: AUC = 61.4%, 95% CI = 53.5%–69.4%; DS: AUC = 57.9%, 95% CI = 54.6%–61.3%) but showed poor calibration with overly extreme risk estimates. Re‐estimating predictor weights improved discriminative capacity (MDD: AUC = 75.9%, 95% CI = 70.3%–81.4%; DS: AUC = 64.8%, 95% CI = 61.9%–67.7%) and calibration. IDEA‐RS predictors ‘poorest level of relationship with the primary caregiver’ (OR = 4.25, 95% CI = 1.73–10.41) and ‘high/highest levels of family conflict’ (OR = 3.36 [95% CI = 1.34–8.43] and OR = 3.76 [95% CI = 1.50–9.38], respectively) showed greatest associations with self‐reported incident MDD.ConclusionsWhile IDEA‐RS yields better‐than‐chance predictions on external replication, accuracy is improved when differences between samples, such as case‐control mix, are adjusted for. IDEA‐RS may be more suited to research settings with sufficient data for refitting. Altogether, we find that IDEA‐RS can be generalisable to early adolescents after refitting and that family dysfunction may be especially impactful for this period of development.

External validation of the Hong Kong Chinese non-laboratory risk models and scoring algorithm for case finding of prediabetes and diabetes mellitus in primary care.

Two Hong Kong Chinese non-laboratory-based prediabetes/diabetes mellitus (pre-DM/DM) risk models were developed using logistic regression (LR) and machine learning, respectively. We aimed to evaluate the models' validity in case finding of pre-DM/DM in a Chinese primary care (PC) population. We also evaluated the validity of a risk-scoring algorithm derived from the LR model. This was a cross-sectional external validation study on Chinese adults, without a prior DM diagnosis, who were recruited from public/private PC clinics in Hong Kong. A total of 1,237 participants completed a questionnaire on the models' predictors. Of that, 919 underwent blood glucose testing. The primary outcome was the models' and the algorithm's sensitivity in finding pre-DM/DM cases. The secondary outcomes were the models' and the algorithm's specificity, positive/negative predictive values, discrimination and calibration. The models' sensitivity were 0.70 (machine learning) and 0.72 (LR). Both showed good external discrimination (area under the receiver operating characteristic curve: machine learning 0.744, LR 0.739). The risks estimated by the models were lower than the observed incidence, indicating poor calibration. Both models were more effective among participants with lower pretest probabilities; that is, age 18-44 years. The algorithm's sensitivity was 0.77 at the cut-off score of ≥16 out of 41. This study showed the validity of the models and the algorithm for finding pre-DM/DM cases in a Chinese PC population in Hong Kong. They can facilitate more cost-effective identification of high-risk individuals for blood testing to diagnose pre-DM/DM in PC. Further studies should recalibrate the models for more precise risk estimation in PC populations.

Evaluating the performance of the SORG machine learning algorithm for predicting discharge disposition in lumbar surgery patients

PurposeProtracted admissions following lumbar surgeries are rising, often stemming from inefficient identification of patients requiring nonhome discharge for rehabilitation. The SORG Orthopaedic Research Group at Harvard Medical School have developed a machine learning algorithm for predicting discharge following lumbar surgery. This study assessed its predictive performance on an independent tertiary centre patient cohort. MethodsMedical records were retrospectively reviewed for all elective adult lumbar disc degeneration or herniation surgeries performed between July 2017–2021 at a tertiary neurosurgical centre in the United Kingdom. Preoperative variables were collated and discharge destinations noted. Algorithm predictions were analysed using the concordance (c) statistic, Brier score and calibration plot. Positive and negative predictive values (PPV, NPV) were calculated, and a decision curve analysis (DCA) plotted. Results251 subjects were included (48.2 % female, mean age 55.3 years). 2.8 % underwent nonhome discharge. Most had surgery at 1/2 spinal levels (98.4 %) and were functionally independent (84.5 %). Algorithm predictions yielded a 0.88 c-statistic and 0.029 Brier score. The algorithm was miscalibrated to the data (calibration plot slope 1.31 and intercept -1.12). At a 0.25 threshold for nonroutine discharge risk, the PPV was 0.19 and NPV 0.98. DCA revealed limited clinical utility. ConclusionsAlgorithm predictive performance was mixed for this cohort, displaying strong discrimination but poor calibration and overestimation of nonroutine discharges. Differences in patient management practices and the low nonhome discharge rate may explain this. Larger validation studies across different healthcare systems, alongside geographically specific algorithm development, will improve predictive accuracy prior to clinical application.

An unmanned ground vehicle phenotyping-based method to generate three-dimensional multispectral point clouds for deciphering spatial heterogeneity in plant traits

Fusing three-dimensional (3D) and multispectral (MS) imaging data holds promise for high-throughput and comprehensive plant phenotyping to decipher genome-to-phenome knowledge. Acquiring high-quality 3D MS point clouds (3DMPCs) of plants remains challenging because of poor 3D data quality and limited radiometric calibration methods for plants with a complex canopy structure. Here, we present a novel 3D spatial–spectral data fusion approach to collect high-quality 3DMPCs of plants by integrating the next-best-view planning for adaptive data acquisition and neural reference field (NeREF) for radiometric calibration. This approach was used to acquire 3DMPCs of perilla, tomato, and rapeseed plants with diverse plant architecture and leaf morphological features evaluated by the accuracy of chlorophyll content and equivalent water thickness (EWT) estimation. The results showed that the completeness of plant point clouds collected by this approach was improved by an average of 23.6% compared with the fixed viewpoints alone. The NeREF-based radiometric calibration with the hemispherical reference outperformed the conventional calibration method by reducing the root mean square error (RMSE) of 58.93% for extracted reflectance spectra. The RMSE for chlorophyll content and EWT predictions decreased by 21.25% and 14.13% using partial least squares regression with the generated 3DMPCs. Collectively, our study provides an effective and efficient way to collect high-quality 3DMPCs of plants under natural light conditions, which improves the accuracy and comprehensiveness of phenotyping plant morphological and physiological traits, and thus will facilitate plant biology and genetic studies as well as crop breeding.

Systematic Evaluation of Uncertainty Calibration in Pretrained Object Detectors

AbstractIn the field of deep learning based computer vision, the development of deep object detection has led to unique paradigms (e.g., two-stage or set-based) and architectures (e.g., Faster-RCNN or DETR) which enable outstanding performance on challenging benchmark datasets. Despite this, the trained object detectors typically do not reliably assess uncertainty regarding their own knowledge, and the quality of their probabilistic predictions is usually poor. As these are often used to make subsequent decisions, such inaccurate probabilistic predictions must be avoided. In this work, we investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation and conduct detailed analyses assessing the calibration under distributional changes (e.g., distributional shift and application to out-of-distribution data). Furthermore, by investigating the influence of different detector paradigms, post-processing steps, and suitable choices of metrics, we deliver novel insights into why poor detector calibration emerges. Based on these insights, we are able to improve the calibration of a detector by simply finetuning its last layer.

External Validation of the Charlson Comorbidity Index-based Model for Survival Prediction in Thai Patients Diagnosed with Dementia

BackgroundThe Charlson Comorbidity Index (CCI) is commonly employed for predicting mortality. Nonetheless, its performance has rarely been evaluated in patients with dementia. This study aimed to examine the predictive capability of the CCI-based model for survival prediction in Thai patients diagnosed with dementia.MethodsAn external validation study was conducted using retrospective data from adults with dementia who had visited the outpatient departments at Maharaj Nakorn Chiang Mai Hospital between 2006 and 2012. The data obtained from electronic medical records included age, gender, date of dementia diagnosis and death, types of dementia, and comorbidities at the time of dementia diagnosis. The discriminative ability and calibration of the CCI-based model were estimated using Harrell’s C Discrimination Index and visualized with calibration plot. As the initial performance did not meet satisfaction, model updating and recalibration were performed.ResultsOf 702 patients, 56.9% were female. The mean age at dementia diagnosis was 75.22 (SD 9.75) year-old. During external validation, Harrell's C-statistic of the CCI-based model was 0.58 (95% CI, 0.54–0.61). The model showed poor external calibration. Model updating was subsequently performed. All updated models demonstrated a modest increase in Harrell's C-statistic. Temporal recalibration did not significantly improve the calibration of any of the updated models.ConclusionThe CCI-based model exhibited fair discriminative ability and poor calibration for predicting survival in Thai patients diagnosed with dementia. Despite attempts at model updating, significant improvements were not achieved. Therefore, it is important to consider the incorporation of other influential prognostic factors.

Assessment of Natural Reference Materials for U‐Pb Geochronology of Grossular‐Andradite Garnet

Garnet U‐Pb dating by laser ablation‐inductively coupled plasma‐mass spectrometry requires the development of matrix‐matched reference materials of variable chemistry and U mass fraction for accurate analysis. Additional calibration of existing primary reference materials is also justified based on the relatively poor calibration of some of the widely available primary reference materials that are currently utilised by the geoscience community. We present a micro sampling workflow combined with a refined ID‐TIMS methodology for the generation of high precision (~ 0.1%) U‐Pb dates from domains within garnet single crystals. Using this workflow, we calibrated two new natural andradite reference materials, the Jumbo andradite (And99; 110.34 ± 0.03 (0.04) [0.13] Ma, n = 7, MSWD = 1.21) and the Tiptop andradite (And87; 209.57 ± 0.11 (0.13) [0.26] Ma, n = 6, MSWD = 1.39). We also present additional calibration of the widely utilised Willsboro‐Lewis andradite primary reference material (And90; 1024.7 ± 9.5 (9.6) [9.6] Ma (2s; overdispersed), n = 6). Wafers of the Jumbo and Tiptop andradite reference materials are available from the authors upon request.

The IMPACT Score: Does Sex Matter?

ObjectiveThe Index for Mortality Prediction After Cardiac Transplantation (IMPACT) score is a quantitative risk index that predicts 1-yr mortality risk, derived from United Network for Organ Sharing (UNOS) data in which women are underrepresented. The validity of the IMPACT score in 1-yr mortality risk after OHT in women is unknown. The objective of this study is to assess differences in score performance by sex. We hypothesize that the IMPACT score is a poor predictor of 1-yr mortality risk after OHT in women. DesignExternal validation study including patients 18 years and older first-time single organ OHT recipients from The International Society of Heart and Lung Transplantation (ISHLT) data from 2009-2018. Demographic and clinical characteristics were compared by sex. IMPACT score was calculated and regression models constructed for the entire sample, and stratified by sex. Model discrimination was assessed with the area under the receiver operating characteristic curve (AUROC) and calibration was assessed graphically. SettingN/A InterventionsN/A ParticipantsPatients 18 years and older first time single OHT recipients from ISHLT registry from 2009-2018. Measurements and Main ResultsFor 1-yr mortality, the AUROC (95% CI) for the full sample was 0.59 (0.57-0.60); 0.58 (0.55-0.61) for women; 0.59 (0.58-0.61) for men. 1-yr mortality was 9.4% in the overall cohort, with no difference in mortality by sex (9.0% vs 9.6% women vs men, p=0.22). ConclusionsThe IMPACT score exhibited poor discrimination and calibration in the ISHLT 2009-2019 cohort, overall and by sex. There was no difference in 1-yr mortality between women and men.

Prediction of 30‐day mortality after surgery for infective endocarditis using risk scores: Insights from a European multicenter comparative validation study

BackgroundIt remains unclear today whether risk scores created specifically to predict early mortality after cardiac operations for infective endocarditis (IE) outperform or not the European System for Cardiac Operative Risk Evaluation II (EuroSCORE II). MethodsPerioperative data and outcomes from a European multicenter series of patients undergoing surgery for definite IE were retrospectively reviewed. Only the cases with known pathogen and without missing values for all considered variables were retained for analyses. A comparative validation of EuroSCORE II and five specific risk scores for early mortality after surgery for IE – (1) STS-IE (Society of Thoracic Surgeons for IE); (2) PALSUSE (Prosthetic valve, Age ≥70, Large intra-cardiac destruction, Staphylococcus spp, Urgent surgery, Sex (female), EuroSCORE ≥10); (3) ANCLA (Anemia, New York Heart Association class IV, Critical state, Large intra-cardiac destruction, surgery on thoracic Aorta); (4) AEPEI II (Association pour l’Étude et la Prévention de l'Endocardite Infectieuse II); (5) APORTEI (Análisis de los factores PROnósticos en el Tratamiento quirúrgico de la Endocarditis Infecciosa) – was carried out using calibration plot and receiver-operating characteristic curve analysis. Areas under the curve (AUCs) were compared 1:1 according to the Hanley-McNeil's method. The agreement between APORTEI score and EuroSCORE II of the 30-day mortality prediction after surgery was also appraised. ResultsA total of 1,012 patients from five European university-affiliated centers underwent 1,036 cardiac operations, with a 30-day mortality after surgery of 9.7%. All IE-specific risk scores considered achieved better results than EuroSCORE II in terms of calibration; AEPEI II and APORTEI score showed the best performances. Despite poor calibration, EuroSCORE II overcame in discrimination every specific risk score (AUC, 0.751 vs. 0.693 or less, p=0.01 or less). For a higher/lesser than 20% expected mortality, the agreement of prediction between APORTEI score and EuroSCORE II was 86%. ConclusionEuroSCORE II discrimination for 30-day mortality after surgery for IE was higher than five established IE-specific risk scores. AEPEI II and APORTEI score showed the best results in terms of calibration.

Risk prediction of advanced colorectal neoplasia varies by race and neighbourhood socioeconomic status

ObjectiveNeighbourhood deprivation increases the risk of colorectal neoplasia and contributes to racial disparities observed in this disease. Developing race-specific advanced colorectal neoplasia (ACN) prediction models that include neighbourhood socioeconomic status...

Clinical complexity of an Italian cardiovascular intensive care unit: the role of mortality and severity risk scores.

The identification of patients at greater mortality risk of death at admission into an intensive cardiovascular care unit (ICCU) has relevant consequences for clinical decision-making. We described patient characteristics at admission into an ICCU by predicted mortality risk assessed with noncardiac intensive care unit (ICU) and evaluated their performance in predicting patient outcomes. A total of 202 consecutive patients (130 men, 75 ± 12 years) were admitted into our tertiary-care ICCU in a 20-week period. We evaluated, on the first 24 h data, in-hospital mortality risk according to Acute Physiology and Chronic Health Evaluation II (APACHE II) and Simplified Acute Physiology Score 3 (SAPS 3); Sepsis related Organ Failure Assessment (SOFA) Score and the Mayo Cardiac intensive care unit Admission Risk Score (M-CARS) were also calculated. Predicted mortality was significantly lower than observed (5% during ICCU and 7% at discharge) for APACHE II and SAPS 3 (17% for both scores). Mortality risk was associated with older age, more frequent comorbidities, severe clinical presentation and complications. The APACHE II, SAPS 3, SOFA and M-CARS had good discriminative ability in distinguishing deaths and survivors with poor calibration of risk scores predicting mortality. In a recent contemporary cohort of patients admitted into the ICCU for a variety of acute and critical cardiovascular conditions, scoring systems used in general ICU had good discrimination for patients' clinical severity and mortality. Available scores preserve powerful discrimination but the overestimation of mortality suggests the importance of specific tailored scores to improve risk assessment of patients admitted into ICCUs.

Calibration method for binocular vision system with large field of view based on small target image splicing

In vision measurement, camera calibration has a significant impact on measurement precision. The classical target-based calibration methods require the target to occupy more than one-third of the field of view. A small-size target that does not meet the requirements results in poor calibration accuracy, while an appropriate large-size target is difficult to manufacture and inconvenient to operate. In view of the above problem, we propose a flexible and accurate calibration method based on small target image splicing to calibrate the binocular vision system with a large field of view. The spliced images and virtual large targets are constructed to extend the target size, providing better flexibility for calibration. Moreover, an optimization objective function integrating two constraints in the imaging plane and measurement space is presented to improve the calibration accuracy during the parameter optimization process. The simulation experiments and actual experiments are carried out to test the performance of the proposed method. The results demonstrate that the calibration accuracy of the proposed method using a small target is equivalent to that of Zhang’s method using a large target. Additionally, when using a same-size target, the parameter error of the proposed method is less than that of Zhang’s method, and the proposed method reduces the distance measurement error from 1.169 mm to 0.208 mm compared to Zhang’s method.

Performance of risk scores in predicting major bleeding in left ventricular assist device recipients: a comparative external validation

BackgroundImplantation of a left ventricular assist device (LVAD) is a crucial therapeutic option for selected end-stage heart failure patients. However, major bleeding (MB) complications postimplantation are a significant concern. ObjectivesWe evaluated current risk scores’ predictive accuracy for MB in LVAD recipients. MethodsWe conducted an observational, single-center study of LVAD recipients (HeartWare or HeartMate-3, November 2010-December 2022) in the Netherlands. The primary outcome was the first post-LVAD MB (according to the International Society on Thrombosis and Haemostasis [ISTH] and Interagency Registry for Mechanically Assisted Circulatory Support [INTERMACS], and INTERMACS combined with intracranial bleeding [INTERMACS+] criteria). Mortality prior to MB was considered a competing event. Discrimination (C-statistic) and calibration were evaluated for the Hypertension, Abnormal Renal/Liver Function, Stroke, Bleeding History or Predisposition, Labile INR, Elderly, Drugs/Alcohol Concomitantly score, Hepatic or Renal Disease, Ethanol Abuse, Malignancy, Older Age, Reduced Platelet Count or Function, Re-Bleeding, Hypertension, Anemia, Genetic Factors, Excessive Fall Risk and Stroke score, Anticoagulation and Risk Factors in Atrial Fibrillation score, Outpatient Bleeding Risk Index, venous thromboembolism score, atrial fibrillation score, and Utah Bleeding Risk Score (UBRS). ResultsOne hundred four patients were included (median age, 64 years; female, 20.2%; HeartWare, 90.4%; HeartMate-3, 9.6%). The cumulative MB incidence was 75.7% (95% CI 65.5%-85.9%) by ISTH and INTERMACS+ criteria and 67.0% (95% CI 56.0%-78.0%) per INTERMACS criteria over a median event-free follow-up time of 1916 days (range, 59-4521). All scores had poor discriminative ability on their intended prediction timeframe. Cumulative area under the receiving operator characteristic curve ranged from 0.49 (95% CI 0.35-0.63, venous thromboembolism-BLEED) to 0.56 (95% CI 0.47-0.65, UBRS) according to ISTH and INTERMACS+ criteria and from 0.48 (95% CI 0.40-0.56, Anticoagulation and Risk Factors in Atrial Fibrillation) to 0.56 (95% CI 0.47-0.65, UBRS) per INTERMACS criteria. All models showed poor calibration, largely underestimating MB risk. ConclusionCurrent bleeding risk scores exhibit inadequate predictive accuracy for LVAD recipients. There is a need for an accurate risk score to identify LVAD patients at high risk of MB who may benefit from patient-tailored antithrombotic therapy.