External Validation Population Research Articles

External validation of prognostic models: what,why, how, when and where?

Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model’s reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.

MRI-based radiomics to predict lipomatous soft tissue tumors malignancy: a pilot study

ObjectivesTo develop and validate a MRI-based radiomic method to predict malignancies in lipomatous soft tissue tumors.MethodsThis retrospective study searched in the database of our pathology department, data from patients with lipomatous soft tissue tumors, with histology and gadolinium-contrast enhanced T1w MR images, obtained from 56 centers with non-uniform protocols. For each tumor, 87 radiomic features were extracted by two independent observers to evaluate the inter-observer reproducibility. A reduction of learning base dimension was performed from reproducibility and relevancy criteria. A model was subsequently prototyped using a linear support vector machine to predict malignant lesions.ResultsEighty-one subjects with lipomatous soft tissue tumors including 40 lipomas and 41 atypical lipomatous tumors or well-differentiated liposarcomas with fat-suppressed T1w contrast enhanced MR images available were retrospectively enrolled. Based on a Pearson’s correlation coefficient threshold at 0.8, 55 out of 87 (63.2%) radiomic features were considered reproducible. Further introduction of relevancy finally selected 35 radiomic features to be integrated in the model. To predict malignant tumors, model diagnostic performances were as follow: AUROC = 0.96; sensitivity = 100%; specificity = 90%; positive predictive value = 90.9%; negative predictive value = 100% and overall accuracy = 95.0%.ConclusionThis work demonstrates that radiomics allows to predict malignancy in soft tissue lipomatous tumors with routinely used MR acquisition in clinical oncology. These encouraging results need to be further confirmed in an external validation population.

Unravelling gender-specific factors that link obesity to albuminuria.

Obesity is a major public health problem, which continues to be diagnosed and classified by BMI, excluding the most elemental concepts of the precision medicine approach. Obesity does not equally affect males and females, even with the same BMI. Microalbuminuria is a risk marker of cardiovascular disease closely related to obesity. The aim of this study was to evaluate the gender-dependent differences in the development of early obesity-related disease, focusing on pathologic microalbuminuria (PMA). We developed a single-centre cross-sectional study including 1068 consecutive adults from May 2016 to January 2018, divided into two groups: one including the first 787 patients attended, evaluated as a description population; the second group included 281 subjects analysed as an external validation population. Collected data included medical history, anthropometric measures, abdominal bioimpedance and routine laboratory tests. First, we confirmed the lack of accuracy of classic obesity measures in predicting microalbuminuria. Second, we tested the utility of a tailored evaluation to predict PMA, with an area under the ROC curve of 0.78 for females and 0.82 for males. We also confirmed the different physiology of visceral adiposity for males when compared to females, in which small variations of fat mass entail major changes in the clinical repercussion. Third, we performed an external validation of our results, achieving a 77% accuracy rate. Our findings support that there is an individual threshold of fat amount necessary to develop obesity-dependent PMA and that gender plays a major role in the interplay between PMA and adiposity.

Predicting the Outcome of Limb Revascularization in Patients With Lower-extremity Arterial Trauma: Development and External Validation of a Supervised Machine-learning Algorithm to Support Surgical Decisions.

Estimating the likely success of limb revascularization in patients with lower-extremity arterial trauma is central to decisions between attempting limb salvage and amputation. However, the projected outcome is often unclear at the time these decisions need to be made, making them difficult and threatening sound judgement. The objective of this study was to develop and validate a prediction model that can quantify an individual patient's risk of failed revascularization. A BN prognostic model was developed using domain knowledge and data from the US joint trauma system. Performance (discrimination, calibration, and accuracy) was tested using ten-fold cross validation and externally validated on data from the UK Joint Theatre Trauma Registry. BN performance was compared to the mangled extremity severity score. Rates of amputation performed because of nonviable limb tissue were 12.2% and 19.6% in the US joint trauma system (n = 508) and UK Joint Theatre Trauma Registry (n = 51) populations respectively. A 10-predictor BN accurately predicted failed revascularization: area under the receiver operating characteristic curve (AUROC) 0.95, calibration slope 1.96, Brier score (BS) 0.05, and Brier skill score 0.50. The model maintained excellent performance in an external validation population: AUROC 0.97, calibration slope 1.72, Brier score 0.08, Brier skill score 0.58, and had significantly better performance than mangled extremity severity score at predicting the need for amputation [AUROC 0.95 (0.92-0.98) vs 0.74 (0.67-0.80); P < 0.0001]. A BN (https://www.traumamodels.com) can accurately predict the outcome of limb revascularization at the time of initial wound evaluation. This information may complement clinical judgement, support rational and shared treatment decisions, and establish sensible treatment expectations.

Risk Prediction for Poor Outcome and Death in Hospital In-Patients with COVID-19: Derivation in Wuhan, China and External Validation in London, UK

Background: Accurate risk prediction of clinical outcome would usefully inform clinical decisions and intervention targeting in COVID-19. The aim of this study was to derive and validate risk prediction models for poor outcome and death in adult inpatients with COVID-19. Methods: Model derivation using data from Wuhan, China used logistic regression with death and poor outcome (death or severe disease) as outcomes. Predictors were demographic, comorbidity, symptom and laboratory test variables. The best performing models were externally validated in data from London, UK. Findings: 4.3% of the derivation cohort (n=775) died and 9.7% had a poor outcome, compared to 34.1% and 42.9% of the validation cohort (n=226). In derivation, prediction models based on age, sex, neutrophil count, lymphocyte count, platelet count, C-reactive protein and creatinine had excellent discrimination (death c-index=0.91, poor outcome c-index=0.88), with good-to-excellent calibration. Using two cut-offs to define low, high and very-high risk groups, derivation patients were stratified in groups with observed death rates of 0.34%, 15.0% and 28.3% and poor outcome rates 0.63%, 8.9% and 58.5%. External validation discrimination was good (c-index death=0.74, poor outcome=0.72) as was calibration. However, observed rates of death were 16.5%, 42.9% and 58.4% and poor outcome 26.3%, 28.4% and 64.8% in predicted low, high and very-high risk groups. Interpretation: Our prediction model using demography and routinely-available laboratory tests performed very well in internal validation in the lower-risk derivation population, but less well in the much higher-risk external validation population. Further external validation is needed. Collaboration to create larger derivation datasets, and to rapidly externally validate all proposed prediction models in a range of populations is needed, before routine implementation of any risk prediction tool in clinical care. Funding Statement: HW and HZ are supported by Medical Research Council and Health Data Research UK Grant (MR/S004149/1), Industrial Strategy Challenge Grant (MC_PC_18029) and Wellcome Institutional Translation Partnership Award (PIII054). RD is supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. DMB is funded by a UKRI Innovation Fellowship as part of Health Data Research UK MR/S00310X/1 (https://www.hdruk.ac.uk). KD is supported by LifeArc STOPCOVID award. This work uses data provided by patients and collected by the NHS as part of their care and support. XW is supported by National Natural Science Foundation of China (grant number:81700006). QL is supported by National Key R&D Program (2018YFC1313700), National Natural Science Foundation of China (grant number: 81870064) and the “Gaoyuan” project of Pudong Health and Family Planning Commission (PWYgy2018-06).Declaration of Interests: The authors declare no competing interests.Ethics Approval Statement: The derivation study was approved by the Research Ethics Committee of Shanghai Dongfang Hospital and Taikang Tongji Hospital. The external validation study operated under London South East Research Ethics Committee (reference 18/LO/2048) approval granted to the King’s Electronic Records Research Interface (KERRI).

Comparison of prognostic models to predict the occurrence of colorectal cancer in asymptomatic individuals: a systematic literature review and external validation in the EPIC and UK Biobank prospective cohort studies

ObjectiveTo systematically identify and validate published colorectal cancer risk prediction models that do not require invasive testing in two large population-based prospective cohorts.DesignModels were identified through an update of a...

Development of a risk prediction model for lung cancer: The Japan Public Health Center-based Prospective Study.

Although the impact of tobacco consumption on the occurrence of lung cancer is well‐established, risk estimation could be improved by risk prediction models that consider various smoking habits, such as quantity, duration, and time since quitting. We constructed a risk prediction model using a population of 59 161 individuals from the Japan Public Health Center (JPHC) Study Cohort II. A parametric survival model was used to assess the impact of age, gender, and smoking‐related factors (cumulative smoking intensity measured in pack‐years, age at initiation, and time since cessation). Ten‐year cumulative probability of lung cancer occurrence estimates were calculated with consideration of the competing risk of death from other causes. Finally, the model was externally validated using 47 501 individuals from JPHC Study Cohort I. A total of 1210 cases of lung cancer occurred during 986 408 person‐years of follow‐up. We found a dose‐dependent effect of tobacco consumption with hazard ratios for current smokers ranging from 3.78 (2.00‐7.16) for cumulative consumption ≤15 pack‐years to 15.80 (9.67‐25.79) for >75 pack‐years. Risk decreased with time since cessation. Ten‐year cumulative probability of lung cancer occurrence estimates ranged from 0.04% to 11.14% in men and 0.07% to 6.55% in women. The model showed good predictive performance regarding discrimination (cross‐validated c‐index = 0.793) and calibration (cross‐validated χ2 = 6.60; P‐value = .58). The model still showed good discrimination in the external validation population (c‐index = 0.772). In conclusion, we developed a prediction model to estimate the probability of developing lung cancer based on age, gender, and tobacco consumption. This model appears useful in encouraging high‐risk individuals to quit smoking and undergo increased surveillance.

Validity of a Risk Prediction Equation for CKD After 10 Years of Follow-up in a Japanese Population: The Ibaraki Prefectural Health Study

Chronic kidney disease (CKD) is an important health problem for which risk equations have been developed for Western populations. This study aimed to develop and validate a risk prediction equation for CKD in a Japanese population. Observational cohort study. The study included 135,007 participants who completed an annual health checkup in 1993 to 1996 in the Ibaraki Prefecture in Japan. Participants were initially free of CKD (defined as stage 3, 4, or 5 CKD or proteinuria [2+ or 3+] by dipstick). Follow-up information was available from health checkups 10 years after the initial evaluation. We used data from 40,963 women and 17,892 men in the northern region of the prefecture for the development of risk prediction equations and 53,042 women and 23,110 men in the southern region for external validation. Age, estimated glomerular filtration rate (eGFR), body mass index, proteinuria, hematuria, hypertension, diabetes mellitus, smoking, and drinking. Occurrence of CKD (defined as eGFR<60mL/min/1.73m2 and/or proteinuria [2+ or 3+] by dipstick). Logistic regression analysis to estimate risk for CKD stratified by sex. During follow-up, 7,500 cases of CKD developed in the northern region and 8,964, in the southern region. Older age, proteinuria (1+), higher systolic blood pressure, medication for hypertension, and current smoking were associated with increased risk for CKD in both sexes, whereas higher eGFR and daily alcohol intake were associated with lower risk. C statistics of risk estimation equations for CKD at 10 years were >0.8 for both the development and external validation populations, and discrimination of the riskestimation was fairly good in women and men. Fluctuations in variables were not evaluated because the study used annual health checkups. This study excluded a large number of people for whom a 10-year health checkup was not available. Estimations of risk for CKD after 10 years of follow-up in a general Japanese population can be achieved with a high level of validity.

Abstract PR15: Projecting individualized absolute risk of developing gastric cancer in Koreans

Abstract Background: Despite the rapid decline of the incidence and mortality over the past few decades, gastric cancer still remains to be the third leading cause of cancer death and the fifth most common cancer worldwide. Furthermore, the absolute number of gastric cancer cases is increasing globally due to the aging of the world population. With available methods to detect and treat precancerous and early stage lesions, development of a prediction model to estimate the probability of developing gastric cancer for various age intervals and risk profiles have important public health implications. Methods: Candidate predictors were selected combining expert opinion and literature search. Using a case-control study with 4,603 Korean subjects, a logistic regression model was used for estimating relative risks, separately for men and women. The discriminatory ability of the models was assessed by receiver operator characteristic area under the curve (AUC). Absolute risk parameters were then calculated combined with the relative risk estimates with baseline age-specific cancer hazard rates from Korean Cancer Registry data. The developed models were validated using an independent prospective cohort data, a Japanese population-based cohort study, which includes 40,173 subjects, by calculating the ratio of expected number (E) over observed number (O) and 95% confidence interval (CI). Results: The models for both men and women included low education, past medical history of diabetes mellitus, no regular use of nonsteroidal inflammatory drugs intake, smoking, no regular exercising, and consumption of high salted food, less fruit, less nonstarchy vegetables, and less nonfermented soy food. The AUC for the relative risk model was 0.73 (95% CI 0.71-0.75) for men and 0.76 (95% CI 0.74-0.79) for women. The overall E/O ratio was 1.05 (95% CI 0.98-1.12) in men, and 0.93 (95% CI 0.83-1.04) in women in the external validation population. Conclusions: To our knowledge this is the first study to estimate the absolute risk of gastric cancer in Korean population. The mathematical models developed in the present study will help predict the occurrence of gastric cancer for an individual considering combined risk factors which will help at a personalized level by enabling early detection and preventive efforts. A further development of a model incorporating biomarkers can provide strategies to select individuals at high risk, for screening for gastric cancer. This abstract is also being presented as Poster B11. Citation Format: Minkyo Song, Jae Jeong Yang, Hyuna Sung, Seong-Ho Kong, Hyuk-Joon Lee, Hyung-Ho Kim, Sang Gyun Kim, Han-Kwang Yang, Norie Sawada, Shoichiro Tsugane, Manami Inoue, Daehee Kang. Projecting individualized absolute risk of developing gastric cancer in Koreans. [abstract]. In: Proceedings of the AACR Special Conference: Improving Cancer Risk Prediction for Prevention and Early Detection; Nov 16-19, 2016; Orlando, FL. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2017;26(5 Suppl):Abstract nr PR15.

Impact of correlation of predictors on discrimination of risk models in development and external populations

BackgroundThe area under the ROC curve (AUC) of risk models is known to be influenced by differences in case-mix and effect size of predictors. The impact of heterogeneity in correlation among predictors has however been under investigated. We sought to evaluate how correlation among predictors affects the AUC in development and external populations.MethodsWe simulated hypothetical populations using two different methods based on means, standard deviations, and correlation of two continuous predictors. In the first approach, the distribution and correlation of predictors were assumed for the total population. In the second approach, these parameters were modeled conditional on disease status. In both approaches, multivariable logistic regression models were fitted to predict disease risk in individuals. Each risk model developed in a population was validated in the remaining populations to investigate external validity.ResultsFor both approaches, we observed that the magnitude of the AUC in the development and external populations depends on the correlation among predictors. Lower AUCs were estimated in scenarios of both strong positive and negative correlation, depending on the direction of predictor effects and the simulation method. However, when adjusted effect sizes of predictors were specified in the opposite directions, increasingly negative correlation consistently improved the AUC. AUCs in external validation populations were higher or lower than in the derivation cohort, even in the presence of similar predictor effects.ConclusionsDiscrimination of risk prediction models should be assessed in various external populations with different correlation structures to make better inferences about model generalizability.

NORRISK 2: A Norwegian risk model for acute cerebral stroke and myocardial infarction.

Background Guidelines for the prevention of cardiovascular disease recommend the estimation of an individual's total risk. We have developed a new model for the prediction of the 10-year risk of incident acute myocardial infarction or cerebral stroke based on Norwegian data, NORRISK 2. Design The model was based on 10-year follow-up of a large population-based cohort (CONOR) through linkage to the CVDNOR project, a database of cardiovascular disease hospital discharge diagnoses and mortality in Norway in 1994-2009. Methods We used the Fine and Gray regression model to estimate the 10-year risk adjusting for competing risk. The model population consisted of participants in 1994-1999 and the external validation population of participants in 2000-2003. We validated the model by area under the receiver operating characteristic curves, calibration plots and analyses of sensitivity and specificity. Results The model population consisted of 31,445 men and 35,267 women aged 40-79 years with 3658 endpoints in men and 2459 in women. The external validation population consisted of 19,980 men and 19,309 women, of whom 1858 men and 874 women had an endpoint during follow-up. The area under the curve was 0.79 (0.79-0.80) in men and 0.84 (0.83-0.85) in women in the model population and was slightly lower in the external validation population. Calibration plots showed good agreement between observed and predicted risk. The sum of sensitivity and specificity was greatest around the suggested risk thresholds. Conclusion The NORRISK 2 model showed good validity in an external dataset and will be a valuable tool to guide decisions about preventive interventions in people without known previous cardiovascular disease.

The ORBIT bleeding score: a simple bedside score to assess bleeding risk in atrial fibrillation

BackgroundTherapeutic decisions in atrial fibrillation (AF) are often influenced by assessment of bleeding risk. However, existing bleeding risk scores have limitations.ObjectivesWe sought to develop and validate a novel bleeding risk score using routinely available clinical information to predict major bleeding in a large, community-based AF population.MethodsWe analysed data from Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF), a prospective registry that enrolled incident and prevalent AF patients at 176 US sites. Using Cox proportional hazards regression, we identified factors independently associated with major bleeding among patients taking oral anticoagulation (OAC) over a median follow-up of 2 years (interquartile range = 1.6–2.5). We also created a numerical bedside risk score that included the five most predictive risk factors weighted according to their strength of association with major bleeding. The predictive performance of the full model, the simple five-item score, and two existing risk scores (hypertension, abnormal renal/liver function, stroke, bleeding history or predisposition, labile INR, elderly, drugs/alcohol concomitantly, HAS-BLED, and anticoagulation and risk factors in atrial fibrillation, ATRIA) were then assessed in both the ORBIT-AF cohort and a separate clinical trial population, Rivaroxaban Once-daily oral direct factor Xa inhibition compared with vitamin K antagonism for prevention of stroke and embolism trial in atrial fibrillation (ROCKET-AF).ResultsAmong 7411 ORBIT-AF patients taking OAC, the rate of major bleeding was 4.0/100 person-years. The full continuous model (12 variables) and five-factor ORBIT risk score (older age [75+ years], reduced haemoglobin/haematocrit/history of anaemia, bleeding history, insufficient kidney function, and treatment with antiplatelet) both had good ability to identify those who bled vs. not (C-index 0.69 and 0.67, respectively). These scores both had similar discrimination, but markedly better calibration when compared with the HAS-BLED and ATRIA scores in an external validation population from the ROCKET-AF trial.ConclusionsThe five-element ORBIT bleeding risk score had better ability to predict major bleeding in AF patients when compared with HAS-BLED and ATRIA risk scores. The ORBIT risk score can provide a simple, easily remembered tool to support clinical decision making.

GFR Estimation Using β-Trace Protein and β2-Microglobulin in CKD

β-Trace protein (BTP) and β2-microglobulin (B2M) are novel glomerular filtration markers that have stronger associations with adverse outcomes than creatinine. Comparisons of BTP and B2M to creatinine and cystatin C are limited by the absence of rigorously developed glomerular filtration rate (GFR) estimating equations for the novel markers. Study of diagnostic test accuracy. Pooled database of 3 populations with chronic kidney disease (CKD) with mean measured GFR of 48 mL/min/1.73 m2 (N=3,551; MDRD [Modification of Diet in Renal Disease] Study, AASK [African American Study of Kidney Disease and Hypertension], and CRIC [Chronic Renal Insufficiency Cohort] Study). GFR estimated using creatinine, cystatin C, BTP, or B2M level. GFR measured as the urinary clearance of iothalamate. For BTP and B2M, coefficients for age, sex, and race were smaller than for creatinine and were similar or smaller than for cystatin C. For B2M, coefficients for sex, age, and race were smaller than for creatinine and were similar (age and race) or smaller (sex) than for cystatin C. The final equations with BTP (BTP, age, and sex) or B2M (B2M alone) were less accurate than either the CKD-EPI (CKD Epidemiology Collaboration) creatinine or cystatin C equations. The combined BTP-B2M equation (BTP and B2M alone) had similar accuracy to the CKD-EPI creatinine or cystatin C equation. The average of the BTP-B2M equation and the CKD-EPI creatinine-cystatin C equation was not more accurate than the CKD-EPI creatinine-cystatin C equation. No external validation population, study population was restricted to CKD, few participants older than 65 years, or nonblack nonwhite race. BTP and B2M are less influenced by age, sex, and race than creatinine and less influenced by race than cystatin C, but provide less accurate GFR estimates than the CKD-EPI creatinine and cystatin C equations. The CKD-EPI BTP and B2M equation provides a methodological advance for their study as filtration markers and in their associations with risk and adverse outcomes, but further study is required before clinical use.

Differentiation between Cystic Pituitary Adenomas and Rathke Cleft Cysts: A Diagnostic Model Using MRI.

Cystic pituitary adenomas may mimic Rathke cleft cysts when there is no solid enhancing component found on MR imaging, and preoperative differentiation may enable a more appropriate selection of treatment strategies. We investigated the diagnostic potential of MR imaging features to differentiate cystic pituitary adenomas from Rathke cleft cysts and to develop a diagnostic model. This retrospective study included 54 patients with a cystic pituitary adenoma (40 women; mean age, 37.7 years) and 28 with a Rathke cleft cyst (18 women; mean age, 31.5 years) who underwent MR imaging followed by surgery. The following imaging features were assessed: the presence or absence of a fluid-fluid level, a hypointense rim on T2-weighted images, septation, an off-midline location, the presence or absence of an intracystic nodule, size change, and signal change. On the basis of the results of logistic regression analysis, a diagnostic tree model was developed to differentiate between cystic pituitary adenomas and Rathke cleft cysts. External validation was performed for an additional 16 patients with a cystic pituitary adenoma and 8 patients with a Rathke cleft cyst. The presence of a fluid-fluid level, a hypointense rim on T2-weighted images, septation, and an off-midline location were more common with pituitary adenomas, whereas the presence of an intracystic nodule was more common with Rathke cleft cysts. Multiple logistic regression analysis showed that cystic pituitary adenomas and Rathke cleft cysts can be distinguished on the basis of the presence of a fluid-fluid level, septation, an off-midline location, and the presence of an intracystic nodule (P = .006, .032, .001, and .023, respectively). Among 24 patients in the external validation population, 22 were classified correctly on the basis of the diagnostic tree model used in this study. A systematic approach using this diagnostic tree model can be helpful in distinguishing cystic pituitary adenomas from Rathke cleft cysts.

Validation of a geriatric screening test to detect frailty in elderly cancer patients: Improving the G8 test.

e12604 Background: The G8 is a geriatric screening test to identify cancer patients older than 70 years who require comprehensive geriatric assessment (CGA) (Sensitivity (Se) = 76%, Specificity (Sp) = 64%). The main objective of our study was to design and evaluate the performance of alternative tests taking into account other predictive domains for frailty. Methods: We included patients recruited in the original ONCODAGE study between 2008 and 2010 and patients recruited in the Aquitaine Geriatric Oncology cohort. We conducted a literature review to identify factors of frailty in elderly cancer patients. Using a Delphi consensus method, we collected 24 European experts’ opinions to validate the most relevant items to improve the G8. Three alternative G8 tests were created and performance assessed on a development population (ONCODAGE cohort). The highest performing test was then compared to the G8, and validated on both internal validation population (ONCODAGE cohort) and external validation population (...

A permutation method to assess heterogeneity in external validation for risk prediction models.

The value of a developed prediction model depends on its performance outside the development sample. The key is therefore to externally validate the model on a different but related independent data. In this study, we propose a permutation method to assess heterogeneity in external validation for risk prediction models. The permutation p value measures the extent of homology between development and validation datasets. If p < 0.05, the model may not be directly transported to the external validation population without further revision or updating. Monte-Carlo simulations are conducted to evaluate the statistical properties of the proposed method, and two microarray breast cancer datasets are analyzed for demonstration. The permutation method is easy to implement and is recommended for routine use in external validation for risk prediction models.

Inflammation-related genetic variations and survival in patients with advanced non-small cell lung cancer receiving first-line chemotherapy.

Backgroundaccurate prognostic prediction is challenging for advanced-stage non-small cell lung cancer (NSCLC) patients.Methodswe systematically investigated genetic variants within inflammation pathway as potential prognostic markers for advanced-stage NSCLC patients treated with first-line chemotherapy. A discovery phase in 502 patients and an internal validation in 335 patients were completed at MD Anderson Cancer Center. External validation was performed in 371 patients at Harvard University.Resultsa missense SNP (HLA-DOB:rs2071554) predicted to influence protein function was significantly associated with poor survival in the discovery (HR:1.46, 95% CI:1.02-2.09), internal validation (HR:1.51, 95% CI:1.02-2.25), and external validation (HR:1.52, 95% CI:1.01-2.29) populations. KLRK1:rs2900420 was associated with a reduced risk in the discovery (HR:0.76, 95% CI:0.60-0.96), internal validation (HR:0.77, 95% CI:0.61-0.99), and external validation (HR:0.80, 95% CI:0.63-1.02) populations. A strong cumulative effect was observed for these SNPs on overall survival.ConclusionsGenetic variations in inflammation-related genes could have potential to complement prediction of prognosis.

A Comparative Assessment of Non-Laboratory-Based versus Commonly Used Laboratory-Based Cardiovascular Disease Risk Scores in the NHANES III Population

BackgroundNational and international primary CVD risk screening guidelines focus on using total CVD risk scores. Recently, we developed a non-laboratory-based CVD risk score (inputs: age, sex, smoking, diabetes, systolic blood pressure, treatment of hypertension, body-mass index), which can assess risk faster and at lower costs compared to laboratory-based scores (inputs include cholesterol values). We aimed to assess the exchangeability of the non-laboratory-based risk score to four commonly used laboratory-based scores (Framingham CVD [2008, 1991 versions], and Systematic COronary Risk Evaluation [SCORE] for low and high risk settings) in an external validation population.Methods and FindingsAnalyses were based on individual-level, score-specific rankings of risk for adults in the Third National Health and Nutrition Examination Survey (NHANES III) aged 25–74 years, without history of CVD or cancer (n = 5,999). Risk characterization agreement was based on overlap in dichotomous risk characterization (thresholds of 10-year risk >10–20%) and Spearman rank correlation. Risk discrimination was assessed using receiver operator characteristic curve analysis (10-year CVD death outcome). Risk characterization agreement ranged from 91.9–95.7% and 94.2–95.1% with Spearman correlation ranges of 0.957–0.980 and 0.946–0.970 for men and women, respectively. In men, c-statistics for the non-laboratory-based, Framingham (2008, 1991), and SCORE (high, low) functions were 0.782, 0.776, 0.781, 0.785, and 0.785, with p-values for differences relative to the non-laboratory-based score of 0.44, 0.89, 0.68 and 0.65, respectively. In women, the corresponding c-statistics were 0.809, 0.834, 0.821, 0.792, and 0.792, with corresponding p-values of 0.04, 0.34, 0.11 and 0.09, respectively.ConclusionsEvery score discriminated risk of CVD death well, and there was high agreement in risk characterization between non-laboratory-based and laboratory-based risk scores, which suggests that the non-laboratory-based score can be a useful proxy for Framingham or SCORE functions in resource-limited settings. Future external validation studies can assess whether the sex-specific risk discrimination results hold in other populations.

Lymph node (LN) ratio (LNR) risk classification (RC) in stage III colon cancer (CC): A pooled analysis of 16,425 patients from the ACCENT database.

3556 Background: The degree of LN involvement is one of the most important prognostic factors in CC. Current staging does not consider the number of total LN examined (TLN) and its predictive ability of nodal associated disease RC. Our aim was to develop a RC model based on LNR with internal and external validation. Methods: Individual patient (pt) data from 18 randomized clinical trials were used. LNR is defined as number of positive LNs divided by number of TLN. The primary endpoint was disease-free survival (DFS). The development dataset consisted of pts receiving 5FU based chemotherapy only. This dataset was randomly split into a training set (6848 pt) and an internal validation set (4952 pt). Pts receiving surgery alone (951 pt) and 5FU plus other agents (oxaliplatin, irinotecan, or oral-based) (3674 pt) comprised two external validation populations. Optimal cutoffs were selected based on Cox model goodness of fit. Validation was based on two key aspects of a predication model - discrimination (c sta...

Prognosis of Patients With Nonspecific Neck Pain

Reanalysis of data from 3 randomized controlled trials. Development and validation of a prediction rule that estimates the probability of complaints persisting for at least 6 months in patients presenting with nonspecific neck pain in primary care. A substantial proportion of the neck pain patients will develop chronic neck pain. An important question is whether we can identify patients at risk of persistent complaints at the first consultation with the physician, based on their personal characteristics? A prediction model that quantifies prognosis in patients with nonspecific neck pain is not available. The study population consisted of a sample (n = 468) from the adult primary care population (18-70 years) in The Netherlands presenting with nonspecific neck pain. The primary outcome measure was global perceived recovery measured at 6 months of follow-up. Seventeen baseline characteristics of the patients were included in the analysis. Significant predictors were identified by multivariable backward stepwise logistic regression analysis. A score chart was constructed by using the regression coefficient estimates. The score chart was externally validated in a cohort of patients with nonspecific neck pain (n = 315), who participated in a randomized controlled trial in the United Kingdom (PANTHER-trial). The multivariable analysis resulted in a set of 9 predictors. The score chart has a discriminative ability of 0.66. External validation of the score chart showed a discriminative ability of 0.65, an adequate calibration, a good fit, and a low explained variation. We developed a score chart, estimating the probability of persistent complaints at 6 months follow-up for patients with nonspecific neck pain. This chart performed well in the study population and external validation population. The prediction which patients are more likely to develop persistent complaints is significantly improved by the score chart.