Measure Of Discriminative Ability Research Articles

The calibrated model-based concordance improved assessment of discriminative ability in patient clusters of limited sample size

BackgroundDiscriminative ability is an important aspect of prediction model performance, but challenging to assess in clustered (e.g., multicenter) data. Concordance (c)-indexes may be too extreme within small clusters. We aimed to define a new approach for the assessment of discriminative ability in clustered data.MethodsWe assessed discriminative ability of a prediction model for the binary outcome mortality after traumatic brain injury within centers of the CRASH trial. With multilevel logistic regression analysis, we estimated cluster-specific calibration slopes which we used to obtain the recently proposed calibrated model-based concordance (c-mbc) within each cluster. We compared the c-mbc with the naïve c-index in centers of the CRASH trial and in simulations of clusters with varying calibration slopes.ResultsThe c-mbc was less extreme in distribution than the c-index in 19 European centers (internal validation; n = 1716) and 36 non-European centers (external validation; n = 3135) of the CRASH trial. In simulations, the c-mbc was biased but less variable than the naïve c-index, resulting in lower root mean squared errors.ConclusionsThe c-mbc, based on multilevel regression analysis of the calibration slope, is an attractive alternative to the c-index as a measure of discriminative ability in multicenter studies with patient clusters of limited sample size.

Features of the Area under the Receiver Operating Characteristic (ROC) Curve. A Good Practice

The area under the receiver operating characteristic (ROC) curve is a measure of discrimination ability used in diagnostic and prognostic research. The ROC plot is usually represented without additional information about decision thresholds used to generate the graph. In our article, we show that adding at least one or more informative cutoff points on the ROC graph facilitates the characterization of the test and the evaluation of the discriminatory capacities, which can result in more informed medical decisions. We use the rocreg and rocregplot commands.

Genetic and Clinical Predictors for Breast Cancer Risk Assessment and Stratification Among Chinese Women

Most of the genetic variants identified from genome-wide association studies of breast cancer have not been validated in Asian women. No risk assessment model that incorporates both genetic and clinical predictors is currently available to predict breast cancer risk in this population. We analyzed 12 single-nucleotide polymorphisms (SNPs) identified in recent genome-wide association studies mostly of women of European ancestry as being associated with the risk of breast cancer in 3039 case patients and 3082 control subjects who participated in the Shanghai Breast Cancer Study. All participants were interviewed in person to obtain information regarding known and suspected risk factors for breast cancer. The c statistic, a measure of discrimination ability with a value ranging from 0.5 (random classification) to 1.0 (perfect classification), was estimated to evaluate the contribution of genetic and established clinical predictors of breast cancer to a newly established risk assessment model for Chinese women. Clinical predictors included in the model were age at menarche, age at first live birth, waist-to-hip ratio, family history of breast cancer, and a previous diagnosis of benign breast disease. The utility of the models in risk stratification was evaluated by estimating the proportion of breast cancer patients in the general population that could be accounted for above a given risk threshold as predicted by the models. All statistical tests were two-sided. Eight SNPs (rs2046210, rs1219648, rs3817198, rs8051542, rs3803662, rs889312, rs10941679, and rs13281615), each of which reflected a genetically independent locus, were found to be associated with the risk of breast cancer. A dose-response association was observed between the risk of breast cancer and the genetic risk score, which is an aggregate measure of the effect of these eight SNPs (odds ratio for women in the highest quintile of genetic risk score vs those in the lowest = 1.85, 95% confidence interval = 1.58 to 2.18, P(trend) = 2.5 x 10(-15)). The genetic risk score, the waist-to-hip ratio, and a previous diagnosis of benign breast disease were the top three predictors of the risk of breast cancer, each contributing statistically significantly (P < .001) to the full risk assessment model. The model, with a c statistic of 0.6295 after adjustment for overfitting, showed promise for stratifying women into different risk groups; women in the top 30% risk group accounted for nearly 50% of the breast cancers diagnosed in the general population. A risk assessment model that includes both genetic markers and clinical predictors may be useful to classify Asian women into relevant risk groups for cost-efficient screening and other prevention programs.

An evaluation of the utility of SSR loci as molecular markers in maize (Zea mays L.): comparisons with data from RFLPS and pedigree

The utility of 131 simple sequence repeat (SSR) loci to characterize and identify maize inbred lines, validate pedigree, and show associations among inbred lines was evaluated using a set of 58 inbred lines and four hybrids. Thirteen sets of inbred parent-progeny triplet pedigrees together with four hybrids and their parental lines were used to quantify incidences of scoring that departed from expectations based upon simple Mendelian inheritance. Results were compared to those obtained using 80 restriction fragment length polymorphism (RFLP) probes. Over all inbred triplets, 2.2% of SSRs and 3.6% of RFLP loci resulted in profiles that were scored as having segregated in a non-Mendelian fashion. Polymorphic index content (PIC, a measure of discrimination ability) values ranged from 0.06 to 0.91 for SSRs and from 0.10 to 0.84 for RFLPs. Mean values for PIC for SSRs and RFLPs were similar, approximately 0.62. However, PIC values for nine SSRs exceeded the maximum PIC for RFLPs. Di-repeats gave the highest mean PIC scores for SSRs but this class of repeats can result in “stutter” bands that complicate accurate genotyping. Associations among inbreds were similar for SSR and RFLP data, closely approximating expectations from known pedigrees. SSR technology presents the potential advantages of reliability, reproducibility, discrimination, standardization and cost effectiveness over RFLPs. SSR profiles can be readily interpreted in terms of alleles at mapped loci across a broad range of maize germ plasm. Consequently, SSRs represent the optimum approach for the identification and pedigree validation of maize genotypes compared to other currently available methods.

Texture discrimination in carpal tunnel syndrome.

The ability to discriminate textured surfaces was measured in patients with carpal tunnel syndrome (CTS). Patients were initially diagnosed using clinical and electrophysiological criteria. The textured surfaces were gratings of alternating ridges and grooves. The gratings differed in their spatial period only, with the ratio of the ridge width to the groove width remaining constant (1:5). A two-alternative forced-choice paradigm was employed in which subjects, both patients and age-matched controls, rubbed their index finger (D2) or little finger (D5) back and forth across the surfaces. The grating spatial period at which subjects could discriminate a difference between the standard grating (spatial period = 2000 microns) and the comparison grating (spatial period in the range 2000-2900 microns) with a probability of 0.75 was taken as the measure of discriminative ability. Statistical comparison of the mean 75% values showed that: (i) when patients used D2 their discriminative ability was significantly impaired in comparison with the discriminative ability of controls using either D2 or D5; (ii) there was no significant difference in discriminative ability between patients and controls when patients used D5 to discriminate the textures; (iii) the 75% values for patients using D2 or D5 did not differ significantly. The degree of abnormality of each patient's sensory evoked potential did not allow us to predict their subsequent performance on the discrimination task.