Hosmer-Lemeshow Χ2 Research Articles

Predicting complications of percutaneous coronary intervention using a novel support vector method

To explore the feasibility of a novel approach using an augmented one-class learning algorithm to model in-laboratory complications of percutaneous coronary intervention (PCI). Data from the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) multicenter registry for the years 2007 and 2008 (n=41 016) were used to train models to predict 13 different in-laboratory PCI complications using a novel one-plus-class support vector machine (OP-SVM) algorithm. The performance of these models in terms of discrimination and calibration was compared to the performance of models trained using the following classification algorithms on BMC2 data from 2009 (n=20 289): logistic regression (LR), one-class support vector machine classification (OC-SVM), and two-class support vector machine classification (TC-SVM). For the OP-SVM and TC-SVM approaches, variants of the algorithms with cost-sensitive weighting were also considered. The OP-SVM algorithm and its cost-sensitive variant achieved the highest area under the receiver operating characteristic curve for the majority of the PCI complications studied (eight cases). Similar improvements were observed for the Hosmer-Lemeshow χ(2) value (seven cases) and the mean cross-entropy error (eight cases). The OP-SVM algorithm based on an augmented one-class learning problem improved discrimination and calibration across different PCI complications relative to LR and traditional support vector machine classification. Such an approach may have value in a broader range of clinical domains.

Development and Evaluation of a Self-Report Tool to Predict Low Pharmacy Refill Adherence in Elderly Patients with Uncontrolled Hypertension

To develop and evaluate a short self-report tool to predict low pharmacy refill adherence in older patients with uncontrolled hypertension. Cross-sectional analysis of survey and administrative data from the Cohort Study of Medication Adherence Among Older Adults (CoSMO). A total of 394 adults with uncontrolled blood pressure; mean ± SD age was 76.6 ± 5.6 years, 33.0% were black, 66.0% were women, and 23.4% had a low medication possession ratio (MPR). We considered 164 self-reported candidate items for development of a prediction rule for low (less than 0.8) versus high (0.8 or more) MPR from pharmacy refill data. Risk prediction models were evaluated by using best subsets analyses, and the final model was chosen based on clinical relevance and model parsimony. Bootstrap simulations assessed internal validity. The performance of the final four-item model was compared to the eight-item Morisky Medication Adherence Scale (MMAS-8) and the nine-item Hill-Bone Compliance Scale. The four-item self-report tool for predicting pharmacy refill adherence showed moderate discrimination (C statistic 0.704, 95% confidence interval [CI], 0.683-0.714) and good model fit (Hosmer-Lemeshow χ² = 1.238, p=0.743). Sensitivity and specificity were 67.4% and 67.8%, respectively. The concordance (C) statistics for MMAS-8 and the Hill-Bone Compliance Scale were lower at 0.665 (95% CI 0.632-0.683) and 0.660 (95% CI 0.622-0.674), respectively. A four-item self-report tool moderately discriminated low from high pharmacy refill adherers, and its test performance was comparable with existing eight- and nine-item adherence scales. Parsimonious self-report tools predicting low pharmacy refill in patients with uncontrolled blood pressure could facilitate hypertension management in the elderly.

Relationship between hospital volume and risk-adjusted mortality rate following percutaneous coronary intervention in Korea, 2003 to 2004

There have been a large number of studies that have investigated the relationship between outcomes and provider volume for a wide variety of medical conditions and surgical conditions. The objective of this study was to explore the relation between hospital volume and risk-adjusted mortality following percutaneous coronary intervention between 2003 and 2004 in Korea. This is a retrospective analysis of database in National Health Insurance Review & Assessment Service and Korean National Statistical Office. The study data set confined to the ICD-10 diagnosis and procedure codes that were recorded in the National Health Insurance Review Agency. Risk modeling was performed through logistic regression and validated with cross-validation. The statistical performance of the developed model was evaluated using c-statistics, R², and Hosmer-Lemeshow statistic. Crude and risk-adjusted 30-day mortality was evaluated among patients who underwent Percutaneous Coronary Intervention (PCI) between 2003 and 2004 at low (less 200 cases/year), medium (200~399 cases/year), and high (400 cases or more/year) PCI volume hospitals. The final risk-adjustment model consisted of ten risk factors for 30-day mortality. These factors were found to have statistically significant effects on patient mortality. The c-statistic and Hosmer-Lemeshow χ² goodness-of-fit test and the model's performance were good [R²=0.147, c-statistic 0.823, 4.1037 (p=0.8476)]. A total number of 60 low-volume hospitals (9.071 patients) and 27 medium-volume hospitals (15.623 patients) and 15 high-volume hospitals (19.669 patients) were included. Crude 30-day mortality rate was 1.4%, 1.1%, and 1.0% (p=0.0106) in each volume hospitals. But risk-adjusted mortality rate was not significantly different among three groups (1.3%, 1.0%, and 1.1% in each volume hospitals). Although we found a significant different crude 30-day mortality rates according to hospital PCI volume, but did not find a relationship between hospital volume and 30-day risk-adjusted mortality rates following PCI in Korea.

Beta-Cell Age Calculator, a Translational Yardstick to Communicate Diabetes Risk with Patients: Tehran Lipid and Glucose Study

Aims. To provide a yardstick for physicians/patients to efficiently communicate/measure incident diabetes risk. Methods. We included data on 5,960 (3,438 women) diabetes-free adults, aged ≥20 years at baseline who either developed diabetes during two consecutive examinations or completed the followup. Age, systolic blood pressure, family history of diabetes, waist-to-height ratio (WHtR), triglyceride-to-high-density lipoprotein cholesterol ratio (TG/HDLD-C), and fasting plasma glucose (FPG) were introduced into an accelerated failure time regression model. Results. Annual diabetes incidence rate was 0.85/1000-person (95% CIs 0.77–0.94). Point-score-system incorporated age (1 point for >65 years), family history of diabetes (4 points), systolic blood pressure (−1 to 3 points), WHtR (−4 to 6 points), TG/HDL-C (1 point for ≥1.5), and FPG (0 to 27 points). Harrell's C statistic = 0.830 (95% CIs 0.808–0.852) and Hosmer-Lemeshow χ 2 = 9.7 (P for lack of fitness = 0.462) indicated good discrimination and calibration. We defined beta-cell age as chronological age of a person with the same predicted risk but all risk factors at the normal levels (i.e., WHtR 0.50, no family history of diabetes, Ln (TG/HDL-C) = 0.531, and FPG = 4.9 (mmol·L−1)). Conclusion. Hereby, we have made it also possible to estimate wide ranges of “beta-cell age” for most chronological ages to assist clinician with risk communication.

Abstract MP049: Evaluating Hypertension Risk Prediction Models in Young Adults: The Coronary Artery Risk Development in Young Adults (CARDIA) Study

Background: A hypertension risk prediction model was developed in the Framingham Heart Study (FHS), but its use among young adults has not been validated. The purpose of this study was to determine the predictive ability of the FHS hypertension model in a cohort of young adults and compare its performance with a model with pre-hypertension as the only predictor. Methods: This study included 4453 participants, aged 18–30 years at enrollment, from the Coronary Artery Risk Development in Young Adults (CARDIA) Study who attended 2 consecutive exams between the baseline (1985-86) and seventh examinations (2005–06). Data were pooled such that the interval between any 2 consecutive exams was 1 time period (median=2.9 years); participants contributed up to 6 time periods at risk for hypertension. Participants with prevalent hypertension or diabetes at the start of each risk period were excluded from that and subsequent risk periods. The FHS hypertension model includes the following variables: age, sex, systolic blood pressure (SBP), diastolic blood pressure (DBP), body mass index, parental hypertension, current smoking, and an age-DBP interaction term at the baseline for each risk period. Incident hypertension was defined as SBP≥140 mmHg, DBP≥ 90 mmHg, or new use of antihypertensive medications. Pre-hypertension was defined as 120≤SBP≤139 mmHg or 80≤SBP≤89 among those not meeting hypertension criteria. The c-statistic was used to assess model discrimination and the Hosmer-Lemeshow chi-square statistic was used to assess model calibration based on quintiles of risk for hypertension. Results: The crude hypertension incidence rate was 13.2 per 1,000 person-years. The FHS hypertension model performed well in identifying those who did and did not develop hypertension (c-statistic=0.87, 95% CI=0.86, 0.88) and was better than a model with pre-hypertension as the only predictor (c-statistic=0.73, 95% CI=0.71, 0.75). The predicted risk from the FHS hypertension model was systematically lower than the observed hypertension incidence in each quintile (Hosmer-Lemeshow χ2= 2649.35; p<0.001). Using CARDIA’s hypertension incidence rate and mean levels of its risk factors to recalibrate the model, the differences between predicted risk and observed hypertension incidence improved but the predicted risk overestimated observed risk in the bottom 4 quintiles and underestimated observed risk in the top quintile (Hosmer-Lemeshow χ2= 47.31; p<0.001). Conclusions: The FHS model performed well in identifying those who developed hypertension in this cohort study of young adults. This model performed better than a pre-hypertension only model, but calibration was poor. The FHS hypertension model may be a useful tool for discriminating individuals with and without a high risk for hypertension and could lead to improvements in targeted prevention strategies for young adults.

Abstract 18064: Prognostic Risk Index for One Year Morbimortality in Hospitalized Patients with Peripheral Artery Disease

Background. Risk stratification in peripheral arterial disease (PAD) may provide the basis for optimal management strategies in patients. This study aimed to develop a prognostic risk index for one-year morbimortality, in patients with severe PAD. Methods and results. Factors associated with morbimortality (total mortality, non-fatal MI or non-fatal stroke) were determined using data from the prospective, multicentre COPART register, including consecutive patients hospitalized for PAD in 3 university hospitals in Southwestern France. A risk score was constructed in a derivation cohort using the beta regression coefficient obtained in the final model of a descending step-by-step logistic regression analysis. The score was then tested on a validation cohort. We analyzed the data of 1,157 patients at hospital discharge: 640 in the derivation cohort and 517 in the validation cohort. At one year, morbimortality rate were respectively 19.2% and 20.3%. The index included the following factors (weight points): age ≥75<85 (+2), ≥85 years (+3); previous history of myocardial infarction (+1); eGFR according to MDRD equation ≤30 (+1.5), >30≤60 ml/min -1 1.73m -2 (+1); ankle-brachial index <0.3 (+2), ≥0.3<0.5 (+1.5) and >1.3 (+2); CRP ≥69mg/l (+2); association of statins, antiplatelet agents, renin-angiotensin system inhibitors (-1.5). Morbimortality rates increased significantly with the risk index in the derivation cohort: low risk (≤0 point): 2%; medium (>0≤2): 12.8%; high (>2<4.5): 23%; very high (≥4.5): 42.2%. The prognostic risk index appeared to be discriminating in both cohorts (C-statistic 0.76 and 0.74). Calibration was satisfying, no significant differences were found between the observed and calculated morbimortality rates in the validation cohort, using our prognostic risk index (Hosmer-Lemeshow χ2 p= 0.65). Conclusions. This prognostic risk index is a tool for risk stratification in patients with PAD and may be helpful for patient counselling and medical decision making.

303 Can Nursing Staff and Junior Physicians Accurately Predict the Risk of In-Hospital Mortality of Acutely Admitted Medical Patients: An Observational Cohort Study

Most acutely admitted medical patients are initially assessed and treated by junior staff, both nursing staff and as well as physicians. However, at presentation, many decisions have to be made and in order to make the right decisions, the staff will have to decide how ill they think the patient really is. We therefore performed the present study to assess the ability of the staff to accurately estimate the in-hospital mortality of acutely admitted medical patients. Design: A prospective observational cohort study. Upon contact with the patient, the first member of the nursing staff to assess the patient was asked to complete a form. This form included their initial estimation of the risk of in-hospital mortality (based solely on the information available to them at the present time). The first doctor to treat the patient was asked to complete a similar form. The discriminatory power (ability to discriminate between survivors and non-survivors) of the estimations of the staff was estimated using the area under the receiver-operating characteristics curve (AUROC). Values above 0.8 represent good discriminatory power. The discriminatory power of the nursing staff and physicians were compared using χ2 test. Calibration (accuracy of the prediction) was assessed using Hosmer-Lemeshow χ2 goodness of fit test with a p-value above 0.05 indicating acceptable calibration. Setting: The general internal medical and cardiology admission units at a regional teaching hospital.Type of participants: 2,855 consecutively admitted acute medical patients. Median age of the patients was 62 years. Sixty-five patients died while admitted (2.3 % in-hospital mortality). The nursing staff estimated the risk of in-hospital mortality for 1,822 patients (63.8 %). We found a good discriminatory power with an AUROC 0.838. Calculating the goodness of fit, we found a χ2 (8 degrees of freedom) of 61.6, p < 0.001.The physicians estimated the risk of in-hospital mortality for 733 patients (25.7 %). We found a good discriminatory power with an AUROC of 0.811. Calculating the goodness of fit, we found a χ2 (8 degrees of freedom) of 32.6, p < 0.001. We were only able to include 507 (17.8 %) patients where comparing the discriminatory power of the nursing staff and the physicians. We were not able to show a statistical significant difference as χ2 was 1.76, p = 0.18. Both nursing staff and physicians were able to distinguish between survivors and non-survivors with an acceptable discriminatory power. However, both groups were unable to show an acceptable calibration.

P1-31 Clinical usefulness of Framingham cardiovascular risk profile during a 10-year follow-up in Iran

IntroductionClinical usefulness of a risk function is a new concept that should be considered beyond the traditional checking of discrimination and calibration for performance of a model. Framingham risk score...

The Model for End-Stage Liver Disease Accurately Predicts 90-Day Liver Transplant Wait-List Mortality in Atlantic Canada

To determine the generalizability of the predictions for 90-day mortality generated by Model for End-stage Liver Disease (MELD) and the serum sodium augmented MELD (MELDNa) to Atlantic Canadian adults with end-stage liver disease awaiting liver transplantation (LT). The predictive accuracy of the MELD and the MELDNa was evaluated by measurement of the discrimination and calibration of the respective models' estimates for the occurrence of 90-day mortality in a consecutive cohort of LT candidates accrued over a five-year period. Accuracy of discrimination was measured by the area under the ROC curves. Calibration accuracy was evaluated by comparing the observed and model-estimated incidences of 90-day wait-list failure for the total cohort and within quantiles of risk. The area under the ROC curve for the MELD was 0.887 (95% CI 0.705 to 0.978) - consistent with very good accuracy of discrimination. The area under the ROC curve for the MELDNa was 0.848 (95% CI 0.681 to 0.965). The observed incidence of 90-day wait-list mortality in the validation cohort was 7.9%, which was not significantly different from the MELD estimate of 6.6% (95% CI 4.9% to 8.4%; P=0.177) or the MELDNa estimate of 5.8% (95% CI 3.5% to 8.0%; P=0.065). Global goodness-of-fit testing found no evidence of significant lack of fit for either model (Hosmer-Lemeshow c2 [df=3] for MELD 2.941, P=0.401; for MELDNa 2.895, P=0.414). Both the MELD and the MELDNa accurately predicted the occurrence of 90-day wait-list mortality in the study cohort and, therefore, are generalizable to Atlantic Canadians with end-stage liver disease awaiting LT.

External validation of the UK Prospective Diabetes Study (UKPDS) risk engine in patients with type 2 diabetes

Aims/hypothesisTreatment guidelines recommend the UK Prospective Diabetes Study (UKPDS) risk engine for predicting cardiovascular risk in patients with type 2 diabetes, although validation studies showed moderate performance. The methods used in these validation studies were diverse, however, and sometimes insufficient. Hence, we assessed the discrimination and calibration of the UKPDS risk engine to predict 4, 5, 6 and 8 year cardiovascular risk in patients with type 2 diabetes.MethodsThe cohort included 1,622 patients with type 2 diabetes. During a mean follow-up of 8 years, patients were followed for incidence of CHD and cardiovascular disease (CVD). Discrimination and calibration were assessed for 4, 5, 6 and 8 year risk. Discrimination was examined using the c-statistic and calibration by visually inspecting calibration plots and calculating the Hosmer–Lemeshow χ2 statistic.ResultsThe UKPDS risk engine showed moderate to poor discrimination for both CHD and CVD (c-statistic of 0.66 for both 5 year CHD and CVD risks), and an overestimation of the risk (224% and 112%). The calibration of the UKPDS risk engine was slightly better for patients with type 2 diabetes who had been diagnosed with diabetes more than 10 years ago compared with patients diagnosed more recently, particularly for 4 and 5 year predicted CVD and CHD risks. Discrimination for these periods was still moderate to poor.Conclusions/interpretationWe observed that the UKPDS risk engine overestimates CHD and CVD risk. The discriminative ability of this model is moderate, irrespective of various subgroup analyses. To enhance the prediction of CVD in patients with type 2 diabetes, this model should be updated.Electronic supplementary materialThe online version of this article (doi:10.1007/s00125-010-1960-0) contains supplementary material, which is available to authorised users.

Mild renal dysfunction in patients undergoing cardiac surgery as a new risk factor for EuroSCORE

BackgroundIt has recently been suggested that mild renal dysfunction is associated with increased mortality in cardiac surgery; however, this risk factor is not accounted for in the European System for...

Evaluation of acid–base balance in ST-elevation myocardial infarction in the early phase: a prognostic tool?

Metabolic acidosis has been described after myocardial infarction, but little data are available on the acid-base imbalance in ST-elevation myocardial infarction (STEMI) submitted to mechanical revascularization, and earlier studies on this topic differ with respect to patients' selection criteria, treatment and evaluated parameters. We assessed admission base excess, anion gap, and lactate in 445 consecutive patients with STEMI submitted to primary percutaneous coronary intervention and whether its evaluation could help in identifying patients at a higher risk for in-hospital mortality and complications (acute pulmonary edema and arrhythmias). At backward stepwise logistic regression analysis, the following variables were independently associated with intra-ICCU mortality: age [odds ratio (OR) 1.05; 95% confidence interval (CI) 1.02-1.10; P = 0.006], estimated glumerular filtration rate (OR 0.98; 95% CI 0.96-0.99; P= 0.010), Tn I (OR 1.006; 95% CI 1.004-1.008; P <0.001), and base excess (OR 0.90; 95% CI 0.82-0.99; P = 0.038); Hosmer-Lemeshow v2: 5.69, P = 0.681. At backward stepwise logistic regression analysis, the following variables were independently associated with intra-ICCU complications: left ventricular ejection fraction (OR 0.95; 95% CI 0.91-0.98; P = 0.005) and lactic acid (OR 1.31; 95% CI 1.10-1.57; P =0.003); Hosmer-Lemeshow v2: 4.11, P = 0.847. According to our findings, the evaluation of base excess and lactate in the early phase of STEMI provides the bedside clinicians with useful tools for early risk stratification. In fact, base excess proved to be an independent predictor for intra-ICCU mortality, whereas lactate represented an independent marker for intra-ICCU complications.

A population-based risk algorithm for the development of diabetes: development and validation of the Diabetes Population Risk Tool (DPoRT)

BackgroundNational estimates of the upcoming diabetes epidemic are needed to understand the distribution of diabetes risk in the population and to inform health policy.ObjectiveTo create and validate a population-based risk...

014 Risk score to predict serious bleeding in stable outpatients with atherothrombosis

Assessment of bleeding risk is often crucial for making decisions regarding chronic use of antithrombotics. To develop a risk score to quantify bleeding risk in outpatients. Design, Setting, and Participants We studied patients enrolled in the Reduction of Atherothrombosis for Continued Health (REACH) Registry, a cohort of 68 236 patients with or at risk of atherothrombosis. The outcome of interest was serious bleeding (non-fatal hemorrhagic stroke or bleeding requiring hospitalization and transfusion) over 2 years. Risk factors for bleeding were assessed using modified regression analysis. Multiple potential scoring systems based on the least complex models were constructed. Competing scores were compared on their discriminative ability via logistic regression. The score was validated externally using the CHARISMA population. From a final cohort of 56 616 patients, 804 (1.42%) experienced serious bleeding between baseline and 2 years. A nine-item bleeding risk score, ranging from 0 to 23 points, was constructed including age, peripheral arterial disease, congestive heart failure, diabetes, hypertension, smoking, antiplatelets, oral anticoagulants, and hypercholesterolemia. Observed incidence of bleeding at 2 years was: 0.46% in patients with a score ≤6; 0.95% for scores 7-8; 1.25% for scores 9-10; and 2.76% for scores ≥11. The score's discriminative performance was consistent in CHARISMA and REACH (c-statistics 0.63 and 0.68, respectively); calibration in the CHARISMA population was very good (modified Hosmer-Lemeshow χ2=4.74; P=.69). Bleeding risk increases dramatically with a score >10. This score can assist clinicians in predicting the risk of serious bleeding and making decisions on antithrombotic therapy in outpatients with atherothrombosis.

Validating the Framingham Hypertension Risk Score

A promising hypertension risk prediction score using data from the US Framingham Offspring Study has been developed, but this score has not been tested in other cohorts. We examined the predictive performance of the Framingham hypertension risk score in a European population, the Whitehall II Study. Participants were 6704 London-based civil servants aged 35 to 68 years, 31% women, free from prevalent hypertension, diabetes mellitus, and coronary heart disease. Standard clinical examinations of blood pressure, weight and height, current cigarette smoking, and parental history of hypertension were undertaken every 5 years for a total of 4 times. We recorded a total of 2043 incident (new-onset) cases of hypertension in three 5-year baseline follow-up data cycles. Both discrimination (C statistic: 0.80) and calibration (Hosmer-Lemeshow chi(2): 11.5) of the Framingham hypertension risk score were good. Agreement between the predicted and observed hypertension incidences was excellent across the risk score distribution. The overall predicted:observed ratio was 1.08, slightly better among individuals >50 years of age (0.99 in men and 1.02 in women) than in younger participants (1.16 in men and 1.18 in women). Reclassification with a modified score on the basis of our study population did not improve the prediction (net reclassification improvement: -0.5%; 95% CI: -2.5% to 1.5%). These data suggest that the Framingham hypertension risk score provides a valid tool with which to estimate near-term risk of developing hypertension.

A New Goodness-of-Fit Test for Event Forecasting and Its Application to Credit Defaults

We develop a new goodness-of-fit test for validating the performance of probability forecasts. Our test statistic is particularly powerful under sparseness and dependence in the observed data. To build our test statistic, we start from a formal definition of calibrated forecasts, which we operationalize by introducing two components. The first component tests the level of the estimated probabilities; the second validates the shape, measuring the differentiation between high and low probability events. After constructing test statistics for both level and shape, we provide a global goodness-of-fit statistic, which is asymptotically χ2 distributed. In a simulation exercise, we find that our approach is correctly sized and more powerful than alternative statistics. In particular, our shape statistic is significantly more powerful than the Kolmogorov–Smirnov test. Under independence, our global test has significantly greater power than the popular Hosmer–Lemeshow's χ2 test. Moreover, even under dependence, our global test remains correctly sized and consistent. As a timely and important empirical application of our method, we study the validation of a forecasting model for credit default events. This paper was accepted by Wei Xiong, finance.

Microsatellite Instability in Interval Colon Cancers

Colon cancers that develop after a complete colonoscopy may be the result of "failure of colonoscopy" or rapid tumor growth. Tumors that develop via the mismatch repair gene pathway demonstrate rapid tumor growth. The aim of this study was to determine if interval colon cancers were more likely than noninterval cancers to result from the loss of function of mismatch repair genes and hence demonstrate microsatellite instability (MSI). We searched our institution's cancer registry for interval cancers, defined as colon cancers that developed within 5 years of a complete colonoscopy. These were frequency matched in a 1:2 ratio by age and sex to patients with noninterval cancers (defined as colon cancers diagnosed on a patient's first recorded colonoscopy). Archived cancer specimens for all subjects were retrieved and tested for MSI. Of the 993 colon cancers diagnosed during the study period, 51 (5.1%) were identified as an interval cancer, and 112 subjects with noninterval cancer served as a comparison group. Study subjects were almost all men. MSI was found in 30.4% of interval cancers compared with 10.3% of noninterval cancers (P = .003). After adjusting for age, interval cancers were 3.7 times more likely to show MSI than noninterval cancers (95% confidence interval, 1.5-9.1). This association was strongest for tumors located in the distal colon (odds ratio, 17.5; P = .008). No difference in TNM stage at diagnosis, histologic type or grade, or 5-year survival was found between groups. Interval colon cancers were almost 4 times as likely as noninterval colon cancers to be associated with mismatch repair gene dysfunction.

Functional status decline as a measure of adverse events in home health care: an observational study.

BackgroundResearch that examines the quality of home health care is complex because no gold standard exists for measuring adverse outcomes, and because the patient and clinician populations are highly heterogeneous. The objectives in this study are to develop models to predict functional decline for three indices of functional status as measures of adverse events in home health care and determine which index is most appropriate for risk-adjusting for future quality research.MethodsData come from the Outcomes and Assessment Information Set (OASIS) from a large urban home health care agency and other agency data. Prognostic data yields 49,437 episodes, while follow-up data yields 47,684 episodes. We tested three indices defined as substantial decline in three or more (gt3_ADLs), two or more (gt2_ADLs), and one or more (gt1_ADLs) ADLs. Multivariate logistic regression determines the performance of the models for each index as measured by the c-statistic and Hosmer-Lemeshow chi square (χ2).ResultsFrequencies for gt3_ADLs, gt2_ADLs, and gt1_ADLs are 212 (0.43%), 783 (1.58%), and 4,271 (8.64%) respectively. Follow-up results are comparable with frequencies of 218 (0.46%), 763 (1.60%), and 3,949 (8.28%) for each index. Gt3_ADLs does not produce valid models. The model for gt2_ADLs consistently yields a higher c-statistic compared to gt1_ADLs (0.754 vs. 0.679, respectively). Both indices' models yield non-significant Hosmer-Lemeshow chi square indicating reasonable model fit. Findings for gt2_ADLs and gt1_ADLs are consistent over time as indicated by follow-up data results.ConclusionGt2_ADLs yields the best models as indicated by a high c-statistic and a non-significant Hosmer-Lemeshow χ2, both of which exhibit exceptional consistency. We conclude that gt2_ADLs may be preferable in defining ADL adverse events in the context of home health care.

Daily Assessment of Organ Dysfunction and Survival in Intensive Care Unit Cardiac Surgical Patients

The purpose of this study was to develop a specific postoperative score for intensive care unit (ICU) cardiac surgical patients for assessment of organ dysfunction and survival. This prospective study consisted of all consecutive adult patients admitted after cardiac surgery to our ICU over a period of 3 years. Evaluation of variables was performed using the first year patients who stayed in the ICU for at least 24 hours. The reproducibility was then tested in two validation sets using all patients. Performance was assessed with the Hosmer-Lemeshow (chi2 statistics) goodness-of-fit test and receiver operating characteristic (ROC) curves and compared with the Acute Physiology and Chronic Health Evaluation (APACHE II) and Multiple Organ Dysfunction Score (MODS). A total of 3,230 patients were admitted to the ICU after cardiac surgery. Mean chi2 values for the new score were 5.8 (APACHE II, 11.3; MODS, 9.7) for the construction set, 7.2 (APACHE II, 8.0; MODS, 4.5) for the validation set I, and 5.9 for the validation set II. The mean area under the ROC curve was 0.91 (APACHE II, 0.86; MODS, 0.84) for the new score in the construction set, 0.88 (APACHE II, 0.84; MODS, 0.84) in the validation set I, and 0.92 in the validation set II. Our new 10-variable risk index performs very well, with calibration and discrimination very high, better than general severity systems; and it is an appropriate tool for daily risk stratification in ICU cardiac surgery patients. Thus, it may serve as an "expert system" for diagnosing organ failure, decision making, resource evaluation, and predicting mortality among ICU cardiac surgical patients.

Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example

To examine how the selections of different risk adjustors and statistical approaches affect the profiles of physician groups on patient satisfaction. Mailed patient surveys. Patients with asthma were selected randomly from each of 20 California physician groups between July 1998 and February 1999. A total of 2515 patients responded. A cross-sectional study. Patient satisfaction with asthma care was the performance indicator for physician group profiling. Candidate variables for risk-adjustment model development included sociodemographic, clinical characteristics, and self-reported health status. Statistical strategies were the ratio of observed-to-expected rate (OE), fixed effects (FE), and the random effects (RE) approaches. Model performance was evaluated using indicators of discrimination (C-statistic) and calibration (Hosmer-Lemeshow chi2). Ranking impact of using different risk adjustors and statistical approaches was based on the changes in absolute ranking (AR) and quintile ranking (QR) of physician group performance and the weighted kappa for quintile ranking. Variables that added significantly to the discriminative power of risk-adjustment models included sociodemographic (age, sex, prescription drug coverage), clinical (asthma severity), and health status (SF-36 PCS and MCS). Based on an acceptable goodness-of-fit (P > 0.1)and higher C-statistics, models adjusting for sociodemographic, clinical, and health status variables (Model S-C-H) using either the FE or RE approach were more favorable. However, the C-statistic (=0.68) was only fair for both models. The influence of risk-adjustor selection on change of performance ranking was more salient than choice of statistical strategy (AR: 50%-80% v. 20%-55%; QR: 10%-30% v. 0%-10%). Compared to the model adjusting for sociodemographic and clinical variables only and using OE approach, the Model S-C-H using RE approach resulted in 70% of groups changing in AR and 25% changing in QR (weighted kappa: 0.88). Compared to the Consumer Assessment of Health Plans model, the Model S-C-H using RE approach resulted in 65% of groups changing in AR and 20% changing in QR (weighted kappa: 0.88). In comparing the performance of physician groups on patient satisfaction with asthma care, the use of sociodemographic, clinical, and health status variables maximized risk-adjustment model performance. Selection of risk adjustors had more influence on ranking profiles than choice of statistical strategies. Stakeholders employing provider profiling should pay careful attention to the selection of both variables and statistical approach used in risk-adjustment.