Simple Recalibration Research Articles

Wavelet-based selection-and-recalibration network for Parkinson’s disease screening in OCT images

Background and Objective:Parkinson’s disease (PD) is one of the most prevalent neurodegenerative brain diseases worldwide. Therefore, accurate PD screening is crucial for early clinical intervention and treatment. Recent clinical research indicates that changes in pathology, such as the texture and thickness of the retinal layers, can serve as biomarkers for clinical PD diagnosis based on optical coherence tomography (OCT) images. However, the pathological manifestations of PD in the retinal layers are subtle compared to the more salient lesions associated with retinal diseases. Methods:Inspired by textural edge feature extraction in frequency domain learning, we aim to explore a potential approach to enhance the distinction between the feature distributions in retinal layers of PD cases and healthy controls. In this paper, we introduce a simple yet novel wavelet-based selection and recalibration module to effectively enhance the feature representations of the deep neural network by aggregating the unique clinical properties, such as the retinal layers in each frequency band. We combine this module with the residual block to form a deep network named Wavelet-based Selection and Recalibration Network (WaveSRNet) for automatic PD screening. Results:The extensive experiments on a clinical PD-OCT dataset and two publicly available datasets demonstrate that our approach outperforms state-of-the-art methods. Visualization analysis and ablation studies are conducted to enhance the explainability of WaveSRNet in the decision-making process. Conclusions:Our results suggest the potential role of the retina as an assessment tool for PD. Visual analysis shows that PD-related elements include not only certain retinal layers but also the location of the fovea in OCT images.

Recalibration of a Non-Laboratory-Based Risk Model to Estimate Pre-Diabetes/Diabetes Mellitus Risk in Primary Care in Hong Kong.

A non-laboratory-based pre-diabetes/diabetes mellitus (pre-DM/DM) risk prediction model developed from the Hong Kong Chinese population showed good external discrimination in a primary care (PC) population, but the estimated risk level was significantly lower than the observed incidence, indicating poor calibration. This study explored whether recalibrating/updating methods could improve the model's accuracy in estimating individuals' risks in PC. We performed a secondary analysis on the model's predictors and blood test results of 919 Chinese adults with no prior DM diagnosis recruited from PC clinics from April 2021 to January 2022 in HK. The dataset was randomly split in half into a training set and a test set. The model was recalibrated/updated based on a seven-step methodology, including model recalibrating, revising and extending methods. The primary outcome was the calibration of the recalibrated/updated models, indicated by calibration plots. The models' discrimination, indicated by the area under the receiver operating characteristic curves (AUC-ROC), was also evaluated. Recalibrating the model's regression constant, with no change to the predictors' coefficients, improved the model's accuracy (calibration plot intercept: -0.01, slope: 0.69). More extensive methods could not improve any further. All recalibrated/updated models had similar AUC-ROCs to the original model. The simple recalibration method can adapt the HK Chinese pre-DM/DM model to PC populations with different pre-test probabilities. The recalibrated model can be used as a first-step screening tool and as a measure to monitor changes in pre-DM/DM risks over time or after interventions.

Breast cancer polygenic risk scores derived in White European populations are not calibrated for women of Ashkenazi Jewish descent

PurposePolygenic risk scores (PRSs) are a major component of accurate breast cancer (BC) risk prediction but require ethnicity-specific calibration. Ashkenazi Jewish (AJ) population is assumed to be of White European (WE) origin in some commercially available PRSs despite differing effect allele frequencies (EAFs). We conducted a case-control study of WE and AJ women from the Predicting Risk of Cancer at Screening Study. The Breast Cancer in Northern Israel Study provided a separate AJ population–based case-control validation series. MethodsAll women underwent Illumina OncoArray single-nucleotide variation (SNV; formerly single-nucleotide polymorphism [SNP]) analysis. Two PRSs were assessed, SNV142 and SNV78. A total of 221 of 2243 WE women (discovery: cases = 111; controls = 110; validation: cases = 651; controls = 1772) and 221 AJ women (cases = 121; controls = 110) were included from the UK study; the Israeli series consisted of 2045 AJ women (cases = 1331; controls = 714). EAFs were obtained from the Genome Aggregation Database. ResultsIn the UK study, the mean SNV142 PRS demonstrated good calibration and discrimination in WE population, with mean PRS of 1.33 (95% CI 1.18-1.48) in cases and 1.01 (95% CI 0.89-1.13) in controls. In AJ women from Manchester, the mean PRS of 1.54 (1.38-1.70) in cases and 1.20 (1.08-1.32) in controls demonstrated good discrimination but overestimation of BC relative risk. After adjusting for EAFs for the AJ population, mean risk was corrected (mean SNV142 PRS cases = 1.30 [95% CI 1.16-1.44] and controls = 1.02 [95% CI 0.92-1.12]). This was recapitulated in the larger Israeli data set with good discrimination (area under the curve = 0.632 [95% CI 0.607-0.657] for SNV142). ConclusionAJ women should not be given BC relative risk predictions based on PRSs calibrated to EAFs from the WE population. PRSs need to be recalibrated using AJ-derived EAFs. A simple recalibration using the mean PRS adjustment ratio likely performs well.

Miscalibration of lung allocation models leads to inaccurate waitlist mortality predictions

The importance of waitlist (WL) mortality risk estimates will increase with the adoption of the US Composite Allocation Score (CAS) system. Calibration is rarely assessed in clinical prediction models, yet it is a key factor in determining access to lung transplant. We assessed the calibration of the WL–lung allocation score (LAS)/CAS models and developed alternative models to minimize miscalibration. Scientific Registry of Transplant Recipients data from 2015 to 2020 were used to assess the calibration of the WL model and for subgroups (age, sex, diagnosis, and race/ethnicity). Three recalibrated models were developed and compared: (1) simple recalibration model (SRM), (2) weighted recalibration model 1 (WRM1), and (3) weighted recalibration model 2 (WRM2). The current WL-LAS/CAS model underestimated risk for 78% of individuals (predicted mortality risk, <42%) and overpredicted risk for 22% of individuals (predicted mortality risk, ≥42%), with divergent results among subgroups. Error measures improved in SRM, WRM1, and WRM2. SRM generally preserved candidate rankings, whereas WRM1 and WRM2 led to changes in ranking by age and diagnosis. Differential miscalibration occurred in the WL-LAS/CAS model, which improved with recalibration measures. Further inquiry is needed to develop mortality models in which risk predictions approximate observed data to ensure accurate ranking and timely access to transplant. ImpactWith changes to the lung transplant allocation system planned in 2023, evaluation of the accuracy and precision of survival models used to rank candidates for lung transplant is important. The waitlist model underpredicts risk for 78% of US transplant candidates with an unequal distribution of miscalibration across subgroups leading to inaccurate ranking of transplant candidates. This work will serve to inform future efforts to improve modeling efforts in the US lung transplant allocation system.

Cardiovascular risk prediction in type 2 diabetes: a comparison of 22 risk scores in primary care settings

Aims/hypothesisWe aimed to compare the performance of risk prediction scores for CVD (i.e., coronary heart disease and stroke), and a broader definition of CVD including atrial fibrillation and heart failure (CVD+), in individuals with type 2 diabetes.MethodsScores were identified through a literature review and were included irrespective of the type of predicted cardiovascular outcome or the inclusion of individuals with type 2 diabetes. Performance was assessed in a contemporary, representative sample of 168,871 UK-based individuals with type 2 diabetes (age ≥18 years without pre-existing CVD+). Missing observations were addressed using multiple imputation.ResultsWe evaluated 22 scores: 13 derived in the general population and nine in individuals with type 2 diabetes. The Systemic Coronary Risk Evaluation (SCORE) CVD rule derived in the general population performed best for both CVD (C statistic 0.67 [95% CI 0.67, 0.67]) and CVD+ (C statistic 0.69 [95% CI 0.69, 0.70]). The C statistic of the remaining scores ranged from 0.62 to 0.67 for CVD, and from 0.64 to 0.69 for CVD+. Calibration slopes (1 indicates perfect calibration) ranged from 0.38 (95% CI 0.37, 0.39) to 0.74 (95% CI 0.72, 0.76) for CVD, and from 0.41 (95% CI 0.40, 0.42) to 0.88 (95% CI 0.86, 0.90) for CVD+. A simple recalibration process considerably improved the performance of the scores, with calibration slopes now ranging between 0.96 and 1.04 for CVD. Scores with more predictors did not outperform scores with fewer predictors: for CVD+, QRISK3 (19 variables) had a C statistic of 0.68 (95% CI 0.68, 0.69), compared with SCORE CVD (six variables) which had a C statistic of 0.69 (95% CI 0.69, 0.70). Scores specific to individuals with diabetes did not discriminate better than scores derived in the general population: the UK Prospective Diabetes Study (UKPDS) scores performed significantly worse than SCORE CVD (p value <0.001).Conclusions/interpretationCVD risk prediction scores could not accurately identify individuals with type 2 diabetes who experienced a CVD event in the 10 years of follow-up. All 22 evaluated models had a comparable and modest discriminative ability.Graphical abstract

Identification of the high-risk patient in primary percutaneous coronary intervention: development and validation of a novel predictive index

Abstract Background Primary percutaneous coronary intervention (PPCI) is the best treatment for patients with ST elevation myocardial infarction (STEMI). Several risk scores have been created to help risk-stratify these patients but few of these can be calculated in-lab, during the acute event. Development of a score that could be applied during PPCI could aid operators' decisions regarding adjunctive therapies and post-procedural surveillance which could improve patient outcomes. This study aimed to develop a simple, practical risk model that could be applied during PPCI to identify high-risk patients. Methods Demographic, clinical and outcome data were collected for all patients, as part of the HEAT-PPCI trial, who underwent PPCI for suspected STEMI between February 2012 and November 2013 at our hospital. Independent predictors of the composite outcome of 28-day mortality or severe impairment of LV function (ejection fraction ≤35%) were identified using multiple logistic regression. A risk model was fitted and internal validation was performed by bootstrapping. External validation was performed on a separate cohort of patients with STEMI. Results The derivation cohort included 1271 patients, with 131/1271 = 10.3% experiencing the composite outcome of 28-day mortality or poor LV function. Three variables were required to predict the outcome: age (OR:2.07, 95% CI 1.55 to 2.78), location of the culprit artery (OR:6.16, 95% CI 4.00 to 9.47), myocardial blush grade post-PPCI (OR:2.32, 95% CI 1.39 to 3.88). External validation was performed on 324 patients undergoing PPCI from a different centre. The model showed good discrimination on ROC-curve analysis (c statistic 0.79, 95% CI 0.75 to 0.83) and performed well on external validation (c statistic 0.87, 95% CI 0.72 to 0.95). Accuracy of the risk model on the validation data was improved by simple recalibration. The model was used to create a risk prediction chart that can be used in-lab during PPCI (Figure 1). Conclusions We have developed a risk model that accurately predicts 28-day mortality or poor LV function following STEMI using age, culprit location and myocardial blush grade. The model can assist operators in identifying high-risk patients during PPCI. Funding Acknowledgement Type of funding source: Public hospital(s). Main funding source(s): National Health Service, UK

Automated identification of chest radiographs with referable abnormality with deep learning: need for recalibration.

To evaluate the calibration of a deep learning (DL) model in a diagnostic cohort and to improve model's calibration through recalibration procedures. Chest radiographs (CRs) from 1135 consecutive patients (M:F = 582:553; mean age, 52.6years) who visited our emergency department were included. A commercialized DL model was utilized to identify abnormal CRs, with a continuous probability score for each CR. After evaluation of the model calibration, eight different methods were used to recalibrate the original model based on the probability score. The original model outputs were recalibrated using 681 randomly sampled CRs and validated using the remaining 454 CRs. The Brier score for overall performance, average and maximum calibration error, absolute Spiegelhalter's Z for calibration, and area under the receiver operating characteristic curve (AUROC) for discrimination were evaluated in 1000-times repeated, randomly split datasets. The original model tended to overestimate the likelihood for the presence of abnormalities, exhibiting average and maximum calibration error of 0.069 and 0.179, respectively; an absolute Spiegelhalter's Z value of 2.349; and an AUROC of 0.949. After recalibration, significant improvements in the average (range, 0.015-0.036) and maximum (range, 0.057-0.172) calibration errors were observed in eight and five methods, respectively. Significant improvement in absolute Spiegelhalter's Z (range, 0.809-4.439) was observed in only one method (the recalibration constant). Discriminations were preserved in six methods (AUROC, 0.909-0.949). The calibration of DL algorithm can be augmented through simple recalibration procedures. Improved calibration may enhance the interpretability and credibility of the model for users. • A deep learning model tended to overestimate the likelihood of the presence of abnormalities in chest radiographs. • Simple recalibration of the deep learning model using output scores could improve the calibration of model while maintaining discrimination. • Improved calibration of a deep learning model may enhance the interpretability and the credibility of the model for users.

High-resolution wavelength tuning technique for a widely tunable MG-Y lasers

The wavelength tuning characteristics of a widely tunable modulated grating Y-branch laser operating in a large temperature range was investigated experimentally. The linear fine-tuning was realized by sweeping the phase region current, and the fine lookup table was created with a high resolution of less than 0.1 pm. We found experimentally that the tuned wavelength has an excellent linear correspondence and there are small intercepts among the parallel tuning lines of different temperatures. Based on this special characteristic, an online simple wavelength re-calibration method of ‘one-wavelength bias pick-up and full wavelength range re-calibration’ was proposed and verified experimentally. The verification experiments proved its feasibility, and a high-tuned wavelength precision and accuracy of ∼0.1 pm within a temperature range of −25 °C to +75 °C was achieved. The tuning accuracy improvement is about two orders for magnitude.

Addressing practical issues of predictive models translation into everyday practice and public health management: a combined model to predict the risk of type 2 diabetes improves incidence prediction and reduces the prevalence of missing risk predictions

IntroductionMany predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of...

Individual participant data validation of the PICNICC prediction model for febrile neutropenia

BackgroundRisk-stratified approaches to managing cancer therapies and their consequent complications rely on accurate predictions to work effectively. The risk-stratified management of fever with neutropenia is one such very common area...

Equalization of four cardiovascular risk algorithms after systematic recalibration: individual-participant meta-analysis of 86 prospective studies.

AimsThere is debate about the optimum algorithm for cardiovascular disease (CVD) risk estimation. We conducted head-to-head comparisons of four algorithms recommended by primary prevention guidelines, before and after ‘recalibration’, a method that adapts risk algorithms to take account of differences in the risk characteristics of the populations being studied.Methods and resultsUsing individual-participant data on 360 737 participants without CVD at baseline in 86 prospective studies from 22 countries, we compared the Framingham risk score (FRS), Systematic COronary Risk Evaluation (SCORE), pooled cohort equations (PCE), and Reynolds risk score (RRS). We calculated measures of risk discrimination and calibration, and modelled clinical implications of initiating statin therapy in people judged to be at ‘high’ 10 year CVD risk. Original risk algorithms were recalibrated using the risk factor profile and CVD incidence of target populations. The four algorithms had similar risk discrimination. Before recalibration, FRS, SCORE, and PCE over-predicted CVD risk on average by 10%, 52%, and 41%, respectively, whereas RRS under-predicted by 10%. Original versions of algorithms classified 29–39% of individuals aged ≥40 years as high risk. By contrast, recalibration reduced this proportion to 22–24% for every algorithm. We estimated that to prevent one CVD event, it would be necessary to initiate statin therapy in 44–51 such individuals using original algorithms, in contrast to 37–39 individuals with recalibrated algorithms.ConclusionBefore recalibration, the clinical performance of four widely used CVD risk algorithms varied substantially. By contrast, simple recalibration nearly equalized their performance and improved modelled targeting of preventive action to clinical need.

Shortwave infrared (1.0–2.5 μm) hyperspectral imaging of the Athabasca West Grand Rapids Formation oil sands

Previous work on the reflectance spectroscopy of oil sands has focused exclusively on the McMurray Formation of the Athabasca deposit. However, as industrial development expands into other formations, spectral research on Alberta’s other bitumen reservoirs is warranted. This study presents the first investigation of shortwave infrared (SWIR; 1.0–2.5 μm) hyperspectral imaging of Grand Rapids Formation oil sands. Comparing the spectral properties of Grand Rapids oil sands to that of the McMurray Formation reveals some key differences. Grand Rapids oil sands have lower overall reflectance and deeper 1.4- and 1.9-μm absorption features than McMurray oil sands of the same ore grade. At ore grades greater than 10 wt. %, Grand Rapids oil sands also have shallower bitumen absorption features. These spectral differences may be attributed to the mineralogical differences between the two formations, namely, structural water in K-feldspar. Hyperspectral analysis techniques that have been previously developed for McMurray oil sands were tested on a Grand Rapids Formation core. Using contrast stretching and the three-band combination of R e 2.05 μm, G e 2.13 μm, and B e 2.28 μm (where R, G, and B refer to red, green, and blue channels), the visibility of sedimentological features was enhanced in SWIR imagery. With a simple recalibration, an established spectral model for the estimation of total bitumen content (TBC) produced results in close agreement with Dean–Stark analysis data (coefficient of determination [R2] e 0.92, ±1.1 wt. %) and of similar accuracy to what has been previously demonstrated for the McMurray Formation. The results of this study suggest that with minor adjustments, oil sands spectral analysis techniques are transferable between different geological formations.

Optical and Electrical Characterization of Biocompatible Polymeric Lines for Hemodialysis Applications.

During hemodialysis (HD), blood is circulated through an extracorporeal tubing system (bloodline) made of medical-grade polymeric material. Sensors of various types that do not come into contact with blood (optical, electromagnetic, etc.) are applied directly across the bloodline for clinical purposes and for therapy customization. Thus, a detailed knowledge of the bloodline’s physical properties is useful for the development of next-generation HD sensors. In this work, we performed a novel comparative analysis of the materials used by the manufacturers of the bloodlines. We focused on signals and characterization techniques matching those of the abovementioned sensors; consequently, this is an application-specific study of the optical and electrical characterization of bloodline material. Such properties are analyzed and compared for bloodlines from seven different manufacturers by optical absorbance spectroscopy and electrical impedance spectroscopy (EIS). Absorbance spectrum measurements are carried out in the VIS-NIR range. Absorbance spectra are pre-processed and data from both types of analyses are normalized with respect to sample thickness. Optical analysis shows that all bloodlines except one have similarly shaped spectra with slight quantitative differences. In all optical spectra, we find a decreasing trend of specific absorption from 0.14 mm−1 at 400 nm to 0.06 mm−1 at 1000 nm, with an absorption peak at 915 nm. In one case, a large absorption peak centered at ≃600 nm is found. Electrical analysis shows that all bloodlines have the electrical properties of a constant-phase element (CPE), with statistically significant differences in parameters’ values. Estimation of electrical CPE parameters for all bloodline returns a range of 0.942–0.957 for parameter n and a range of 12.41–16.64 for parameter Q0’. In conclusion, we find that, although some statistically significant differences are present, bloodlines from a representative group of manufacturers share similar electrical and optical properties. Therefore, contactless sensing devices developed for HD will work on different bloodlines if a simple recalibration is performed.

A Simple Community-Based Risk-Prediction Score for Sudden Cardiac Death

BackgroundAlthough sudden cardiac death is a leading cause of death in the United States, most victims of sudden cardiac death are not identified as at risk prior to death. We sought to derive and validate a population-based risk score that predicts sudden cardiac death. MethodsThe Atherosclerosis Risk in Communities (ARIC) Study recorded clinical measures from men and women aged 45-64 years at baseline; 11,335 white and 3780 black participants were included in this analysis. Participants were followed over 10 years and sudden cardiac death was physician adjudicated. Cox proportional hazards models were used to derive race-specific equations to estimate the 10-year sudden cardiac death risk. Covariates for the risk score were selected from available demographic and clinical variables. Utility was assessed by calculating discrimination (Harrell's C-index) and calibration (Hosmer-Lemeshow chi-squared test). The white-specific equation was validated among 5626 Framingham Heart Study participants. ResultsDuring 10 years' follow-up among ARIC participants (mean age 54.4 years, 52.4% women), 145 participants experienced sudden cardiac death; the majority occurred in the highest quintile of predicted risk. Model covariates included age, sex, total cholesterol, lipid-lowering and hypertension medication use, blood pressure, smoking status, diabetes, and body mass index. The score yielded very good internal discrimination (white-specific C-index 0.82; 95% confidence interval [CI], 0.78-0.85; black-specific C-index 0.75; 95% CI, 0.68-0.82) and very good external discrimination among Framingham participants (C-index 0.82; 95% CI, 0.79-0.86). Calibration plots indicated excellent calibration in ARIC (white-specific chi-squared 5.3, P = .82; black-specific chi-squared 4.1, P = .77), and a simple recalibration led to excellent fit within Framingham (chi-squared 2.1, P = 0.99). ConclusionsThe proposed risk scores may be used to identify those at risk for sudden cardiac death within 10 years and particularly classify those at highest risk who may merit further screening.

Discrimination slope and integrated discrimination improvement – properties, relationships and impact of calibration

Discrimination slope, defined as the slope of a linear regression of predicted probabilities of event derived from a prognostic model on the binary event status, has recently gained popularity as a measure of model performance. It is as a building block for the integrated discrimination improvement that equals the difference in discrimination slopes between the two models being compared. Several authors have pointed out that it does not make sense to apply the integrated discrimination improvement and discrimination slope when working with mis-calibrated models, whereas others have raised concerns about the ability of improving discrimination slope without adding new information. In this paper, we show that under certain assumptions the discrimination slope is asymptotically related to two other R-squared measures, one of which is a rescaled version of the Brier score, known to be proper. Furthermore, we illustrate how a simple recalibration makes the slope equal to the rescaled Brier R-squared metric. We also show that the discrimination slope can be interpreted as a measure of reduction in expected regret for the Gini-Brier regret function. Using theoretical and practical examples, we illustrate how all of these metrics are affected by different levels of model mis-calibration. In particular, we demonstrate that simple recalibration ascertaining calibration in-the-large and calibration slope equal to 1 are not sufficient to correct for some forms of mis-calibration. We conclude that R-squared metrics, including the discrimination slope, offer an attractive choice for quantifying model performance as long as one accounts for their sensitivity to model calibration. Copyright © 2016 John Wiley & Sons, Ltd.

Best Practices for Postprocessing Ensemble Climate Forecasts. Part I: Selecting Appropriate Recalibration Methods

Abstract This study describes a systematic approach to selecting optimal statistical recalibration methods and hindcast designs for producing reliable probability forecasts on seasonal-to-decadal time scales. A new recalibration method is introduced that includes adjustments for both unconditional and conditional biases in the mean and variance of the forecast distribution and linear time-dependent bias in the mean. The complexity of the recalibration can be systematically varied by restricting the parameters. Simple recalibration methods may outperform more complex ones given limited training data. A new cross-validation methodology is proposed that allows the comparison of multiple recalibration methods and varying training periods using limited data. Part I considers the effect on forecast skill of varying the recalibration complexity and training period length. The interaction between these factors is analyzed for gridbox forecasts of annual mean near-surface temperature from the CanCM4 model. Recalibration methods that include conditional adjustment of the ensemble mean outperform simple bias correction by issuing climatological forecasts where the model has limited skill. Trend-adjusted forecasts outperform forecasts without trend adjustment at almost 75% of grid boxes. The optimal training period is around 30 yr for trend-adjusted forecasts and around 15 yr otherwise. The optimal training period is strongly related to the length of the optimal climatology. Longer training periods may increase overall performance but at the expense of very poor forecasts where skill is limited.

Improving patient prostate cancer risk assessment: Moving from static, globally-applied to dynamic, practice-specific risk calculators

Clinical risk calculators are now widely available but have generally been implemented in a static and one-size-fits-all fashion. The objective of this study was to challenge these notions and show via a case study concerning risk-based screening for prostate cancer how calculators can be dynamically and locally tailored to improve on-site patient accuracy. Yearly data from five international prostate biopsy cohorts (3 in the US, 1 in Austria, 1 in England) were used to compare 6 methods for annual risk prediction: static use of the online US-developed Prostate Cancer Prevention Trial Risk Calculator (PCPTRC); recalibration of the PCPTRC; revision of the PCPTRC; building a new model each year using logistic regression, Bayesian prior-to-posterior updating, or random forests. All methods performed similarly with respect to discrimination, except for random forests, which were worse. All methods except for random forests greatly improved calibration over the static PCPTRC in all cohorts except for Austria, where the PCPTRC had the best calibration followed closely by recalibration. The case study shows that a simple annual recalibration of a general online risk tool for prostate cancer can improve its accuracy with respect to the local patient practice at hand.

The validation of cardiovascular risk scores for patients with type 2 diabetes mellitus

ObjectiveVarious cardiovascular prediction models have been developed for patients with type 2 diabetes. Their predictive performance in new patients is mostly not investigated. This study aims to quantify the predictive...

Simple dichotomous updating methods improved the validity of polytomous prediction models

ObjectivesPrediction models may perform poorly in a new setting. We aimed to determine which model updating methods should be applied for models predicting polytomous outcomes, which often suffer from one or more categories with low prevalence. Study Design and SettingWe used case studies on testicular and ovarian tumors. The original regression models were based on 502 and 2,037 patients and validated on 273 and 1,107 patients, respectively. The polytomous models combined dichotomous models for category A vs. B + C and B vs. C (sequential dichotomous modeling). Simple recalibration, revision, and redevelopment methods were considered. To assess discrimination (using dichotomous and polytomous c-statistics) and calibration (by comparing observed and expected prevalences) of these methods, the validation data were divided into updating and test parts. Five hundred such divisions were randomly generated, and the average test set results reported. ResultsNone of the updating methods could improve discrimination of the original models, but recalibration, revision, and redevelopment strongly improved calibration. Redevelopment was unstable with respect to overfitting and performance. ConclusionSimple dichotomous updating methods behaved well when applied to polytomous models. Our results suggest that recalibration is preferred, but larger validation sets may make revision or redevelopment a sensible alternative.

DEMOCRATISATION OF DEMOCRACY? ON THE DISCONTINUITY BETWEEN EMPIRICAL AND NORMATIVE THEORIES OF DEMOCRACY

The article considers the gap that exists between between normative and empirical theories of democracy. Empirical theories usually stop in their aspirations where normative theories get off the ground, that is, they take the model of liberal democracy as their normative horizon. This is a confusing situation especially with regard to the possibilities of enhancing the quality of existing liberal democracies. We argue that a simple recalibration of democracy indexes, so as to include normatively more demanding considerations, is impossible, due to certain recurring features of normative models. We conclude that if we want to be serious about bridging the gap, concessions should be made on both sides.