Hosmer-Lemeshow Chi-square Statistic Research Articles

Development and Validation of a Risk-Prediction Nomogram for Chronic Low Back Pain Using a National Health Examination Survey: A Cross-Sectional Study.

Several prognostic factors have been reported for chronic low back pain (CLBP). However, there are no studies on the prediction of CLBP development in the general population using a risk prediction model. This cross-sectional study aimed to develop and validate a risk prediction model for CLBP development in the general population, and to create a nomogram that can help a person at risk of developing CLBP to receive appropriate counseling on risk modification. Data on CLBP development, demographics, socioeconomic history, and comorbid health conditions of the participants were obtained through a nationally representative health examination and survey from 2007 to 2009. Prediction models for CLBP development were derived from a health survey on a random sample of 80% of the data and validated in the remaining 20%. After developing the risk prediction model for CLBP, the model was incorporated into a nomogram. Data for 17,038 participants were analyzed, including 2693 with CLBP and 14,345 without CLBP. The selected risk factors included age, sex, occupation, education level, mid-intensity physical activity, depressive symptoms, and comorbidities. This model had good predictive performance in the validation dataset (concordance statistic = 0.7569, Hosmer-Lemeshow chi-square statistic = 12.10, p = 0.278). Based on our model, the findings indicated no significant differences between the observed and predicted probabilities. The risk prediction model presented by a nomogram, which is a score-based prediction system, can be incorporated into the clinical setting. Thus, our prediction model can help individuals at risk of developing CLBP to receive appropriate counseling on risk modification from primary physicians.

Estimation of 10-Year Risk of Death from Coronary Heart Disease, Stroke, and Cardiovascular Disease in a Pooled Analysis of Japanese Cohorts: EPOCH-JAPAN.

Aims: We aimed to develop and validate risk prediction models to estimate the absolute 10-year risk of death from coronary heart disease (CHD), stroke, and cardiovascular disease (CVD). Methods: We evaluated a total of 44,869 individuals aged 40–79 years from eight Japanese prospective cohorts to derive coefficients of risk equations using cohort-stratified Cox proportional hazard regression models. Discrimination (C-index) of the equation was examined in each cohort and summarised using random-effect meta-analyses. Calibration of the equation was assessed using Hosmer-Lemeshow chi-squared statistic. Results: Within a median follow-up of 12.7 years, we observed 765 deaths due to CVD (276 CHDs and 489 strokes). After backward selection, age, sex, current smoking, systolic blood pressure (SBP), proteinuria, prevalent diabetes mellitus, the ratio of total cholesterol to high-density lipoprotein cholesterol (TC/HDLC), interaction terms of age by SBP, and age by current smoking were retained as predictors for CHD. Sex was excluded in the stroke equation. We did not consider TC/HDLC as a risk factor for the stroke and CVD equations. The pooled C-indices for CHD, stroke, and CVD were 0.83, 0.80, and 0.81, respectively, and the corresponding p -values of the Hosmer-Lemeshow tests were 0.18, 0.003, and 0.25, respectively. Conclusions: Risk equations in the present study can adequately estimate the absolute 10-year risk of death from CHD, stroke, and CVD. Future work will evaluate the system as an education and risk communication tool for primary prevention of CHD and stroke.

NOMOGRAM FOR THE CUMULATIVE LIVE BIRTH RATE IN LOW PROGNOSIS PATIENTS ACCORDING TO POSEIDON CRITERIA: A RETROSPECTIVE COHORT STUDY OF 4494 PATIENTS IN CHINA

A small number of prediction model have been previously developed to predict the success of infertility treatment. However, the studies of prediction model for low prognosis patients are limited. The aim of this study was to develop and validate a nomogram for the cumulative live birth rate (CLBR) in low prognosis patients using a single center database in China. Retrospective cohort study. We reviewed the clinical data of 4494 low prognosis patients according to POSEIDON criteria who underwent in vitro fertilization (IVF)/intracytoplasmic sperm injection (ICSI) treatment between 2014 and 2018 at Assisted Reproduction Center of Northwest Women’s and Children’s Hospital, Northwest China. Of the eligible participants, 70% were randomly assigned to the training set, while the remaining 30% were assigned to the external validation set. Multivariate analysis using the Logistic regression model was performed. Multivariate analyses revealed that female age, female body mass index (BMI), antral follicle count (AFC), male infertility, uterine factor infertility, and basal serum follicle stimulating hormone (FSH) level were significant factors for CLBR in low prognosis patients. An area under the receiver operating characteristic curve (AUC) in the prediction model was 0.70 (95% CI 0.68 to 0.72) in the training cohort. The validation set showed good discrimination with an AUC of 0.68 (95% CI 0.65 to 0.71). Additionally, Hosmer-Lemeshow chi-square statistic was 5.931 (P = 0.655). We developed and validated a nomogram to predict CLBR in low prognosis patients using a single center database in China. The validated nomogram to predict CLBR could be a potential tool for IVF counselling in low prognosis patients.

Validating the Framingham Hypertension Risk Score: A 4-year follow-up from the Brazilian Longitudinal Study of the Adult Health (ELSA-Brasil).

The Framingham Heart Study published an equation that permits to estimate the 4-year incidence of hypertension among adults. In Brazil, only the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil) of 15105 men and women aged 35-74years enrolled in 2008-2010 has data that can validate the Framingham Risk Score for Hypertension and create a new equation according to the Brazilian population. We examined the predictive performance of the Framingham Risk Score for Hypertension in the ELSA-Brasil using as an outcome variable, the 4-year incidence of hypertension. We split randomly the 8027 participants who participated in the second visit (2012-2014) and without hypertension at baseline in derivation data set (n=4825; 60%) and a validation data set (n=3202 participants; 40%). The area under the curve for Framingham Risk Score for Hypertension and ELSA-Brasil Risk Score was relatively similar. Hosmer-Lemeshow chi-squared statistic applied for the Framingham Risk Score was 3.78 (P-value=.876) and for our model was 8.22 (P-value=.41), disclosing good discrimination and calibration for both models. Even with these classification intervals, our model presents more underestimation of the risk, classifying 15% of the participants with new onset of hypertension in low risk vs 9% of the Framingham model and less overestimation of the risk, classifying 17% of the participants without hypertension as high risk vs 24% of the Framingham model. We concluded that the Framingham Risk Score for Hypertension has an acceptable performance when applied in the ELSA-Brasil population with good discrimination and calibration.

Mortality Prediction in Rural Kenya: A Cohort Study of Mechanical Ventilation in Critically Ill Patients.

Importance:Critical care is expanding in low- and middle-income countries. Yet, due to factors such as missing data and different disease patterns, predictive scores often fail to adequately predict the high rates of mortality observed.Objectives:We evaluated multiple prognostic models for the outcome of mortality in critically ill, mechanically ventilated patients in rural Kenya and examined factors contributing to mortality in our setting.Design, Setting, and Participants:A prospective cohort study was conducted on mechanically ventilated patients in rural Kenya. Consecutive patients 16 years old and older initiated on mechanical ventilation between January 1, 2016, and April 30, 2017, at Tenwek Hospital were included. Demographic data, clinical characteristics, and patient outcomes were collected during routine clinical care.Main Outcomes and Measures:We assessed the discrimination and calibration of multiple previously-described models for mortality: Acute Physiology and Chronic Health Evaluation II, Sequential Organ Failure Assessment, quick Sequential Organ Failure Assessment, Simplified Acute Physiology Score II, Modified Early Warning Score, Tropical Intensive Care Score, Rwanda-Mortality Predictive Model, Vitals score (validated in Tanzania), and Vitals score for sepsis (validated in Uganda). Factors most associated with mortality were analyzed in our cohort utilizing stepwise regression.Results:Among the final cohort of 300 patients, the overall mortality rate was 60.7%, the average age was 39.9 years, 65% were male, and 33% were seen at an outside facility prior to admission to the critical care unit. Missing variables occurred in patients for numerous models but were complete in most adapted to resource-limited settings. Models displayed moderate prediction of mortality and variable discrimination area under the receiver operating characteristic curves (and Hosmer-Lemeshow chi-square statistic) of 0.77 (22.4) for Acute Physiology and Chronic Health Evaluation II, 0.70 (3.4) for Modified Early Warning Score, 0.65 (0.16) for quick Sequential Organ Failure Assessment, 0.55 (18.4) for Simplified Acute Physiology Score II and 0.74 (9.2) for Rwanda-Mortality Predictive Model, 0.72 (0.12) for Vitals Tanzania, 0.68 (14.7) for Vitals Uganda, and 0.65 (13.9) for Tropical Intensive Care Score. Variables associated with increased mortality in our population were hypotension, infection, traumatic brain injury, and hematocrit.Conclusions and Relevance:Overall, survival for critically ill patients in rural Kenya was poor, but predictable with contributing factors. Models designed for resource-constrained settings had favorable discrimination and better calibration for mortality prediction than high-resource models in our population of mechanically ventilated, critically ill patients in rural Kenya.

4325Change in left atrial function predicts incident atrial fibrillation: the multi-ethnic study of atherosclerosis

Abstract Background Longitudinal change in left atrial (LA) structure and function could be helpful in predicting risk for incident AF. We used cardiac magnetic resonance (CMR) imaging to explore the relationship between change in LA structure and function and incident AF in a multi-ethnic population free of clinical cardiovascular disease at baseline. Methods and results In the Multi-Ethnic Study of Atherosclerosis (MESA), 2338 participants, free at baseline of clinically recognized AF and cardiovascular disease, had LA volume and function assessed with CMR imaging, at baseline (2000–02), and at Exam 4 (2005–07) or 5 (2010–12). Free of AF, 124 participants developed AF over 3.8±0.9 years (to 2015) following the second imaging. In adjusted Cox regression models, an average annualized change in all LA parameters were significantly associated with an increased risk of AF. An annual decrease of 1–SD unit in total LA emptying fractions (LAEF) was most strongly associated with risk of AF after adjusting for AF clinical risk factors, baseline LA parameters and left ventricular mass-to-volume ratio (hazard ratio per SD=1.91,95% confidence interval=1.53–2.38, P<0.001). The addition of change in total LAEF to an AF risk score improved model discrimination and reclassification (net reclassification improvement=0.107, P=0.017; integrative discrimination index=0.049, P<0.001). Model discrimination, NRI and IDI Model: CHARGE-AF risk factors# + Baseline LA variable + ΔLA variable CHARGE-AF ΔLAVImin (mL/m2/y) ΔTotal LAEF (%/y) ΔPeak LA Strain (%/y) C-statistic (95% CI) 0.757 (0.721–0.794) 0.787 (0.749–0.824) 0.779 (0.737–0.820) 0.770 (0.732–0.808) NRI† (p-value) – 0.000 (0.991) 0.107 (0.017) 0.017 (0.634) IDI (p-value) – 0.049 (<0.001) 0.049 (<0.001) 0.018 (<0.001) Calibration chi2* (p-value) 19.3 (0.02) 11.68 (0.232) 5.751 (0.765) 3.673 (0.932) AF: atrial fibrillation; LA: left atrium; EF: emptying fractions; Δ: annual change; VImin: minimum indexed volume; CI: confidence interval; NRI: net reclassification improvement; IDI: integrative discrimination index #CHARGE-AF risk factors: age, race, height, weight, systolic and diastolic blood pressure, use of antihypertensive medication, smoking status, diabetes, MI and CHF by the second imaging. † Categories for NRI: <2.5%, 2.5%-5% and >5%. *Model calibration: Grønnesby and Borgan's modified Hosmer-Lemeshow chi-square statistic for survival analysis. Kaplan-Meier curves of ΔTotal LAEF Conclusion In this multi-ethnic study population free of clinical cardiovascular disease at baseline, a greater increase in LA volumes and decrease in LA function were associated with incident AF. The addition of change in total LAEF to risk prediction models for AF improved model discrimination and reclassification of AF risk. Acknowledgement/Funding This research was supported by contracts HHSN2682015ehz745.01623I,N01-HC-95159-69 from the NHLBI and UL1-TR-ehz745.016240,UL1-TR-001079,R01-HL-127659 from NIH

1515-P: Thai Diabetes Risk Score Performed Less Well in Predicting Incident Diabetes in a General Population: External Validation in the HCUR Cohort

Background: An existing Thai diabetes risk score was developed using data from a worker cohort and has been advocated as a tool to predict diabetes risk in a Thai population. However, the risk score has not been validated in an external cohort, so its accuracy in risk prediction remained uncertain. Aim: To validate the Thai diabetes risk score in an external cohort of Thai general population in the Health Checks Ubon Ratchathani (HCUR) cohort. Methods: The Thai diabetes risk score was externally validated against incident cases of diabetes developed over 6 years in a cohort of 332,038 individuals aged >=30 years free of clinically and/or biochemically diagnosed diabetes. Published coefficients for the Thai diabetes risk score were used to compute risk score for each individual. Incident diabetes was defined as physician diagnosed diabetes. We computed standard measures of predictive ability including discriminatory ability measured by the area under the receiver operating characteristic curves (aROC) and calibration, as well as the sensitivity, specificity, positive/negative predictive values, and false negative for cut-off points indicating high risk groups. Results: Over a 6-year period between 2007 and 2012, 7,372 new cases of diabetes developed (an incidence of 2.2%). The Thai risk score showed low discriminative ability and was poorly calibrated for predicting diabetes, with the aROC of 0.574 (95% CI 0.567 - 0.581) and Hosmer-Lemeshow chi-square statistic of 70.04 (p<0.001). A cut-off point for very high risk group (risk score>=8) had low sensitivity and high specificity (18% and 90% respectively), albeit a satisfactorily low false negative of 2%. Discussion: The current Thai diabetes risk score performed poorly in predicting incident diabetes in a general population and hence requires re-calibration. Disclosure P. Chamnan: None. P. Warasook: None. O. Srisawang: None. N. Phrommachart: None. B. Wettana: None.

Factors Associated with Intimate Partner Violence: Logistic Regression Approach

Background: Intimate Partner Violence is a major health and human rights problem. Globally, intimate partner violence includes physical act of violence together with sexual coercion, physical threats, psychological abuse and controlling actions by the intimate partner.Objective: This study aims to identify the factors influencing intimate partner violence of married women in Pokhara, Lekhnath Metropolitan City ward number 27, Kaski, Nepal.Materials and Methods: This study is entirely based on the primary data. The sample contains 303 married women of 15-49 years selected by systematic sampling. Data was collected using structured questionnaire and face to face interview. The questionnaire used is modified version of the questionnaire designed and recommended by World Health Organization (WHO) and Program for Appropriate Technology in Health ‘PATH’ (2005). Data was analyzed using descriptive and inferential statistical methods. To determine the significant factors, Logistic regression model was used for establishing linkage between intimate partner violence and several explanatory variables.Results: Fitted model showed the variables, affair with other women and alcohol consumption habit were highly significant predictors for the intimate partner violence. Occupation of husband was significant predictor for physical violence and occupation of respondent was significant for sexual violence. Goodness of fit tests (Hosmer-Lemeshow chi-square statistic, Negelkerke R2) showed that all three models fit well to the proposed Logistic regression model.Conclusion: Although violence is a multidimensional concept, this study has attempted to capture three types of violence namely psychological, physical and sexual called IPV. The IPV was collapsed to create a dichotomous variable on the basis of whether the woman has IPV or not. A number of independent variables are taken into account based on scientific literature review.Nepalese Journal of Statistics, Vol. 2, 37-52

Factors Affecting Maternal Health Care Services Utilization in Nepal: Insight from the Nepal Demographic Health Survey 2006 and 2011

Background: In this study, a maternal health care service is analyzed under the components of place of delivery (POD). POD is highlighted as ‘home delivery’ and ‘institutional delivery’.Objective: The objective of this study is to analyze factors associated with the utilization of health institute as delivery and estimate the probabilities of institutional delivery using some selected independent variables.Materials and Methods: Data used were extracted from individual recode of a data file of Nepal Demography Health Survey (NDHS) 2006 and 2011. The unit of analysis for this study is Ever Married Women (EMW) who had at least one live birth in the five years preceding the survey. Sample of this study consists of 4182 EMW for 2006 NDHS and 4079 EMW for 2011 NDHS. The dependent variable is place of delivery. The independent variables are mixture of categorical and continuous variables. For building of suitable statistical models, various types of models were explored and different measure of models adequacy test were applied. Finally, two logistic regression models were developed separately for 2006 and 2011 NDHS data.Results: Fitted model showed several variables: education level, wealth index, birth order, residence and ANC by provider were highly significant predictors for the selection of place of delivery for 2006 NDHS but in 2011 NDHS age was also highly significant. Goodness of fit tests (Hosmer-Lemeshow chi-square statistic), multicollinearity diagnostics, residual analysis and outliers (leverage value) showed that both models fit well to the proposed logistic regression model.Conclusion: A Comparative assessment of model coefficients between 2006 and 2011 NDHS, it is found that the values are only slightly different for most of the predictors under consideration demonstrating consistency of associations found in the two surveys.Nepalese Journal of Statistics, 2017, Vol. 1, 55-72

Predicting Mortality in Low-Income Country ICUs: The Rwanda Mortality Probability Model (R-MPM).

IntroductionIntensive Care Unit (ICU) risk prediction models are used to compare outcomes for quality improvement initiatives, benchmarking, and research. While such models provide robust tools in high-income countries, an ICU risk prediction model has not been validated in a low-income country where ICU population characteristics are different from those in high-income countries, and where laboratory-based patient data are often unavailable. We sought to validate the Mortality Probability Admission Model, version III (MPM0-III) in two public ICUs in Rwanda and to develop a new Rwanda Mortality Probability Model (R-MPM) for use in low-income countries.MethodsWe prospectively collected data on all adult patients admitted to Rwanda’s two public ICUs between August 19, 2013 and October 6, 2014. We described demographic and presenting characteristics and outcomes. We assessed the discrimination and calibration of the MPM0-III model. Using stepwise selection, we developed a new logistic model for risk prediction, the R-MPM, and used bootstrapping techniques to test for optimism in the model.ResultsAmong 427 consecutive adults, the median age was 34 (IQR 25–47) years and mortality was 48.7%. Mechanical ventilation was initiated for 85.3%, and 41.9% received vasopressors. The MPM0-III predicted mortality with area under the receiver operating characteristic curve of 0.72 and Hosmer-Lemeshow chi-square statistic p = 0.024. We developed a new model using five variables: age, suspected or confirmed infection within 24 hours of ICU admission, hypotension or shock as a reason for ICU admission, Glasgow Coma Scale score at ICU admission, and heart rate at ICU admission. Using these five variables, the R-MPM predicted outcomes with area under the ROC curve of 0.81 with 95% confidence interval of (0.77, 0.86), and Hosmer-Lemeshow chi-square statistic p = 0.154.ConclusionsThe MPM0-III has modest ability to predict mortality in a population of Rwandan ICU patients. The R-MPM is an alternative risk prediction model with fewer variables and better predictive power. If validated in other critically ill patients in a broad range of settings, the model has the potential to improve the reliability of comparisons used for critical care research and quality improvement initiatives in low-income countries.

Validation of the Pooled Cohort equations in a long-term cohort study of Hong Kong Chinese

The 2013 American College of Cardiology and the American Heart Association guidelines recommended the Pooled Cohort equations for evaluation of cardiovascular (CV) risk of individuals. We investigated the usefulness of the Pooled Cohort equations in Chinese by validating this risk prediction model using the Hong Kong Cardiovascular Risk Factor Prevalence Study (CRISPS) cohort. The Hong Kong CRISPS is a population-based prospective cohort study of CV risk factors among 2895 Chinese men and women (aged 25-74years) initiated in 1995. CV events were ascertained until December 2012. The discrimination and calibration of the Pooled Cohort equations were evaluated and compared with the Framingham CV risk equation. A Hosmer-Lemeshow chi-square statistic (X(2)) of <20 indicated good calibration. The discrimination power of the 2 models in both men and women was moderate. The calibration score of both models were unacceptable in men (Pooled Cohort X(2), 24.1; Framingham X(2), 20.1), but was satisfactory in women (10.1 and 12.1, respectively). In men, with recalibration of the model using the CRISPS data, the accuracy of prediction improved. Recalibration, however, could not be applied to the Pooled Cohort model because the degree of miscalibration varied across the different risk categories. The Pooled Cohort equations provide poor calibration and moderate discrimination in Hong Kong Chinese, especially in men. In contrast, the Framingham CV risk equation can be applied to the Chinese population but requires recalibration in men.

Survival nomogram for curatively resected Korean gastric cancer patients: multicenter retrospective analysis with external validation.

BackgroundA small number of nomograms have been previously developed to predict the individual survival of patients who undergo curative resection for gastric cancer. However, all were derived from single high-volume centers. The aim of this study was to develop and validate a nomogram for gastric cancer patients using a multicenter database.MethodsWe reviewed the clinicopathological and survival data of 2012 patients who underwent curative resection for gastric cancer between 2001 and 2006 at eight centers. Among these centers, six institutions were randomly assigned to the development set, and the other two centers were assigned to the validation set. Multivariate analysis using the Cox proportional hazard regression model was performed, and discrimination and calibration were evaluated by external validation.ResultsMultivariate analyses revealed that age, tumor size, lymphovascular invasion, depth of invasion, and metastatic lymph nodes were significant prognostic factors for overall survival. In the external validation, the concordance index was 0.831 (95% confidence interval, 0.784–0.878), and Hosmer-Lemeshow chi-square statistic was 3.92 (P = 0.917).ConclusionsWe developed and validated a nomogram to predict 5-year overall survival after curative resection for gastric cancer based on a multicenter database. This nomogram can be broadly applied even in general hospitals and is useful for counseling patients, and scheduling follow-up.

Abstract 122: Impact of Using Clinical Data to Profile Hospitals for 30-day Stroke Mortality: Data from the Ontario Stroke Audit

Background: Risk-adjustment models that use administrative data may not fully account for case-mix differences between hospitals. Supplementing these models with clinical data may improve their accuracy. We examined the impact of using data from a clinical registry in addition to administrative data on profiling Ontario hospitals for 30-day stroke mortality. Methods: Ontario Stroke Audit (OSA) data on 7006 acute stroke cases admitted to 54 Ontario hospitals between 2010-11 were obtained. We calculated 30-day risk-standardized mortality (RSMR) from an established risk adjustment model that relied on only hospital discharge data. This model (termed ADMIN) included age, sex, past medical history (including atrial fibrillation, stroke, diabetes, hypertension, heart disease), ambulance arrival, and stroke-type. We then modified this model to create a CLINICAL model by adding statistically significant clinical variables from the OSA data. In addition to the ADMIN model variables, the CLINICAL model included dysphagia, cognitive status, pre-stroke function, and stroke severity. We compared model performance (c-statistics, calibration [Hosmer-Lemeshow chi-square statistic (HLCS], Integrated Discrimination Improvement [IDI]), and hospital rankings (based on RSMR) using Spearman rank correlation. Results: Model performance was noticeably better for the CLINICAL model compared to the ADMIN model; discrimination was higher (c-statistic 0.82 vs. 0.70), calibration better (HLCS p= 0.08 vs &lt;0.01) and predictive accuracy greater (IDI= 0.13, p &lt;0.001). Comparison of RSMRs between the two models showed only moderate correlation (r = 0.653) with substantial variation for some hospitals (Figure). Conclusions: The addition of variables from clinical data to an established administrative data model had a marked effect on hospital rankings. These results suggest that clinical variables could have a meaningful impact on the accuracy of hospital report cards.

Evaluation of the Revised Trauma Score in Predicting Outcomes of Trauma Patients

Introduction The revised trauma score (RTS) was developed more than 20 years ago. Few studies investigated its usefulness in predicting trauma outcomes. This is especially true for the weighted version of RTS (RTS-w). The aim of this study was to test the predicting power of RTS-w for the trauma outcomes including mortality, admission to intensive care unit (ICU), hospital length of stay and ICU length of stay through a comparison with Injury Severity Score (ISS). Methods Descriptive data, variables related to the trauma scores and outcomes were collected. The statistical performance of RTS-w and ISS in predicting the trauma outcomes using receiver operating characteristics (ROC) curves and the area under the curve (AUC) with 95% confidence interval and p value were calculated. The Hosmer-Lemeshow chi-squared statistic was performed to measure its calibration. Results A total of 3323 patients were enrolled in the study. RTS-w was significantly better than ISS in predicting mortality of trauma patients (AUC: 0.934 vs.0.880, p&lt;0.0001). However, for the other three outcomes, i.e. admission to ICU, hospital length of stay and intensive care unit length of stay, the performance of RTS-w was inferior to ISS. Conclusions The RTS-w is a better predictor of mortality than ISS. But its ability to predict other trauma outcomes is not as good as ISS. More studies are needed to identify the predictive ability of RTS-w for the outcomes other than mortality. Besides, updating the coefficients of the formula may make RTS-w more accurate.

External validation of the 2008 Framingham cardiovascular risk equation for CHD and stroke events in a European population of middle-aged men. The PRIME study

ObjectiveTo test the applicability of the sex-specific 2008 Framingham general cardiovascular risk equation for coronary heart disease (CHD) and stroke in European middle-aged men from Ireland and France. MethodsIn the PRIME study, 9638 healthy middle-aged men recruited in France and Ireland were surveyed for 10years for the occurrence of first CHD and stroke events. The original Framingham equation, the partially calibrated Framingham equation (using the PRIME baseline survival at 10years), and the completely calibrated Framingham equation (additionally using risk factor means calculated in PRIME) were assessed. Model fit (expected versus observed events) and discrimination ability were assessed using a modified Hosmer–Lemeshow Chi-square statistic and Harrell's c-index respectively. ResultsThe original (uncalibrated) Framingham equation overestimated by 1.94-fold the risk of CHD and stroke combined in PRIME, and by 2.23 and 1.42-fold in PRIME-France and PRIME-Ireland respectively. Adequate fit was found after complete calibration. However, discrimination ability of the Framingham equation was poor as shown by Harrell's c-index lower than 0.70. ConclusionThe (completely) calibrated 2008 Framingham equation predicted accurate number of CHD and stroke events but discriminated poorly individuals at higher from those at lower event risk in a European population of middle-aged men.

Breast cancer risk assessment across the risk continuum: genetic and nongenetic risk factors contributing to differential model performance

IntroductionClinicians use different breast cancer risk models for patients considered at average and above-average risk, based largely on their family histories and genetic factors. We used longitudinal cohort data from women whose breast cancer risks span the full spectrum to determine the genetic and nongenetic covariates that differentiate the performance of two commonly used models that include nongenetic factors - BCRAT, also called Gail model, generally used for patients with average risk and IBIS, also called Tyrer Cuzick model, generally used for patients with above-average risk.MethodsWe evaluated the performance of the BCRAT and IBIS models as currently applied in clinical settings for 10-year absolute risk of breast cancer, using prospective data from 1,857 women over a mean follow-up length of 8.1 years, of whom 83 developed cancer. This cohort spans the continuum of breast cancer risk, with some subjects at lower than average population risk. Therefore, the wide variation in individual risk makes it an interesting population to examine model performance across subgroups of women. For model calibration, we divided the cohort into quartiles of model-assigned risk and compared differences between assigned and observed risks using the Hosmer-Lemeshow (HL) chi-squared statistic. For model discrimination, we computed the area under the receiver operator curve (AUC) and the case risk percentiles (CRPs).ResultsThe 10-year risks assigned by BCRAT and IBIS differed (range of difference 0.001 to 79.5). The mean BCRAT- and IBIS-assigned risks of 3.18% and 5.49%, respectively, were lower than the cohort's 10-year cumulative probability of developing breast cancer (6.25%; 95% confidence interval (CI) = 5.0 to 7.8%). Agreement between assigned and observed risks was better for IBIS (HL X42 = 7.2, P value 0.13) than BCRAT (HL X42 = 22.0, P value <0.001). The IBIS model also showed better discrimination (AUC = 69.5%, CI = 63.8% to 75.2%) than did the BCRAT model (AUC = 63.2%, CI = 57.6% to 68.9%). In almost all covariate-specific subgroups, BCRAT mean risks were significantly lower than the observed risks, while IBIS risks showed generally good agreement with observed risks, even in the subgroups of women considered at average risk (for example, no family history of breast cancer, BRCA1/2 mutation negative).ConclusionsModels developed using extended family history and genetic data, such as the IBIS model, also perform well in women considered at average risk (for example, no family history of breast cancer, BRCA1/2 mutation negative). Extending such models to include additional nongenetic information may improve performance in women across the breast cancer risk continuum.

Abstract PR-02: Breast Cancer Risk Assessment: Genetic and Non-Genetic Risk Factors contributing to Differential Model Performance

Abstract Introduction: Clinicians use different breast cancer risk models for patients considered at average and above-average risk, based largely on their family histories and genetic factors. To explore the feasibility of a single model appropriate for all women, we used longitudinal cohort data from women whose breast cancer risks span the full spectrum to determine the covariates that differentiate the performance of two commonly used models, BCRAT, or also called GAIL model, (used for patients with average risk) and IBIS, or also called Tyrer Cuzick model, (for patients with above-average risk). Methods: We evaluated the performance of BCRAT and IBIS for ten-year absolute risk of breast cancer, using prospective data from 1,857 women over a mean follow-up length of 8.1 years, of whom 83 developed cancer. This cohort spans the continuum of breast cancer risk, with some subjects at lower than average population risk. Therefore, the wide-variation in individual risk makes it an interesting population to examine model performance across subgroups of women. For model calibration, we divided the cohort into quartiles of model-assigned risk and comparing differences between assigned and observed risks using the Hosmer-Lemeshow (HL) chi-squared statistic. For model discrimination, we computed the area under the receiver operator curve (AUC) and the case risk percentiles (CRPs). Results: The ten-year risks assigned by BCRAT and IBIS differed (range of difference 0.001 to 79.5). The mean BCRAT- and IBIS-assigned risks of 3.18% and 5.59%, respectively, were lower than the cohort's 10-year cumulative probability of developing breast cancer (6.25%; 95% confidence interval (CI) = 5.0-7.8%). Agreement between assigned and observed risks was better for IBIS (HL X42 =7.2, P-value 0.13) than BCRAT (HL X42 = 22.0, P-value &amp;lt; 0.001). The IBIS model also showed better discrimination (AUC = 69.5%, CI = 63.8% - 75.2%) than did the BCRAT model (AUC = 63.2%, CI = 57.6% - 68.9%). In almost all covariate-specific subgroups, BCRAT mean risks were significantly lower than the observed risks, while IBIS risks showed generally good agreement with observed risks, even in the subgroups of women considered at average risk (e.g., no family history of breast cancer, BRCA1/2 mutation negative). Conclusions: Models developed using extended family history and genetic data, such as the IBIS model, also perform well in women considered at average risk (e.g. no family history of breast cancer, BRCA1/2 mutation negative). Extending such models to include nongenetic information (e.g., breast biopsy history and race/ethnicity) may improve their performance among all women across the breast cancer risk continuum. This abstract is also presented as Poster A109. Citation Format: Anne S. Quante, Alice S. Whittemore, Mary Beth Terry. Breast cancer risk assessment: Genetic and nongenetic risk factors contributing to differential model performance. [abstract]. In: Proceedings of the Eleventh Annual AACR International Conference on Frontiers in Cancer Prevention Research; 2012 Oct 16-19; Anaheim, CA. Philadelphia (PA): AACR; Cancer Prev Res 2012;5(11 Suppl):Abstract nr PR-02.

Performance of the Seattle Heart Failure Model in Implantable Defibrillator Patients Treated With Cardiac Resynchronization Therapy

The Seattle Heart Failure Model (SHFM) is a validated multivariate risk prediction model for mortality in patients with heart failure, using widely available clinical variables. The aim of this study was to assess the performance of the SHFM when applied to patients with heart failure who received cardiac resynchronization therapy devices with defibrillation. A total of 413 patients were identified from 2 prospective implantable cardioverter-defibrillator registries who received cardiac resynchronization therapy devices with defibrillation for the primary prevention of sudden death. Baseline laboratory and clinical data were entered in the SHFM to calculate predicted survival. The end point was all-cause mortality. During a median follow-up period of 2.8 years, 78 patients died and 9 underwent heart transplantation. Observed versus predicted 5-year mortality rates were 11.6% versus 11.4%, 21.5% versus 22.1%, and 41.4% versus 46.1% by ascending tertile of Seattle Heart Failure Score, respectively. No systematic or substantial errors of risk estimation were observed. Discrimination was excellent; the C-statistic ranged from 0.78 at 1-year follow-up to 0.70 at 5-year follow-up, and the Hosmer-Lemeshow chi-square statistic was 0.87 (p = 0.65). In conclusion, in patients with heart failure who received cardiac resynchronization therapy devices with defibrillation, the SHFM offers adequate discrimination of risk for all-cause mortality and estimation of mortality risk without substantial or systematic errors.

Abstract MP049: Evaluating Hypertension Risk Prediction Models in Young Adults: The Coronary Artery Risk Development in Young Adults (CARDIA) Study

Background: A hypertension risk prediction model was developed in the Framingham Heart Study (FHS), but its use among young adults has not been validated. The purpose of this study was to determine the predictive ability of the FHS hypertension model in a cohort of young adults and compare its performance with a model with pre-hypertension as the only predictor. Methods: This study included 4453 participants, aged 18–30 years at enrollment, from the Coronary Artery Risk Development in Young Adults (CARDIA) Study who attended 2 consecutive exams between the baseline (1985-86) and seventh examinations (2005–06). Data were pooled such that the interval between any 2 consecutive exams was 1 time period (median=2.9 years); participants contributed up to 6 time periods at risk for hypertension. Participants with prevalent hypertension or diabetes at the start of each risk period were excluded from that and subsequent risk periods. The FHS hypertension model includes the following variables: age, sex, systolic blood pressure (SBP), diastolic blood pressure (DBP), body mass index, parental hypertension, current smoking, and an age-DBP interaction term at the baseline for each risk period. Incident hypertension was defined as SBP≥140 mmHg, DBP≥ 90 mmHg, or new use of antihypertensive medications. Pre-hypertension was defined as 120≤SBP≤139 mmHg or 80≤SBP≤89 among those not meeting hypertension criteria. The c-statistic was used to assess model discrimination and the Hosmer-Lemeshow chi-square statistic was used to assess model calibration based on quintiles of risk for hypertension. Results: The crude hypertension incidence rate was 13.2 per 1,000 person-years. The FHS hypertension model performed well in identifying those who did and did not develop hypertension (c-statistic=0.87, 95% CI=0.86, 0.88) and was better than a model with pre-hypertension as the only predictor (c-statistic=0.73, 95% CI=0.71, 0.75). The predicted risk from the FHS hypertension model was systematically lower than the observed hypertension incidence in each quintile (Hosmer-Lemeshow χ2= 2649.35; p&lt;0.001). Using CARDIA’s hypertension incidence rate and mean levels of its risk factors to recalibrate the model, the differences between predicted risk and observed hypertension incidence improved but the predicted risk overestimated observed risk in the bottom 4 quintiles and underestimated observed risk in the top quintile (Hosmer-Lemeshow χ2= 47.31; p&lt;0.001). Conclusions: The FHS model performed well in identifying those who developed hypertension in this cohort study of young adults. This model performed better than a pre-hypertension only model, but calibration was poor. The FHS hypertension model may be a useful tool for discriminating individuals with and without a high risk for hypertension and could lead to improvements in targeted prevention strategies for young adults.

Improving American College of Surgeons National Surgical Quality Improvement Program Risk Adjustment: Incorporation of a Novel Procedure Risk Score

Risk-adjusted evaluation is a key component of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). The purpose of this study was to improve standard ACS NSQIP risk adjustment using a novel procedure risk score. Current Procedural Terminology codes (CPTs) represented in ACS NSQIP data were assigned to 136 procedure groups. Log odds predicted risk from preliminary logistic regression modeling generated a continuous risk score for each procedure group, used in subsequent modeling. Appropriate subsets of 271,368 patients in the 2008 ACS NSQIP were evaluated using logistic models for overall 30-day morbidity, 30-day mortality, and surgical site infection (SSI). Models were compared when including either work Relative Value Unit (RVU), RVU and the standard ACS NSQIP CPT range variable (CPT range), or RVU and the newly constructed CPT risk score (CPT risk), plus routine ACS NSQIP predictors. When comparing the CPT risk models with the CPT range models for morbidity in the overall general and vascular surgery dataset, CPT risk models provided better discrimination through higher c statistics at earlier steps (0.81 by step 3 vs 0.81 by step 46), more information through lower Akaike's information criterion (127,139 vs 130,019), and improved calibration through a smaller Hosmer-Lemeshow chi-square statistic (48.76 vs 116.79). Improved model characteristics of CPT risk over CPT range were most apparent for broader patient populations and outcomes. The CPT risk and standard CPT range models were moderately consistent in identification of outliers as well as assignment of hospitals to quality deciles (weighted kappa ≥ 0.870). Information from focused, clinically meaningful CPT procedure groups improves the risk estimation of ACS NSQIP models.