Applying Kraemer’s Q (Positive Sign Rate): Some Implications for Diagnostic Test Accuracy Study Results

Abstract

Background/Aims: Sensitivity and specificity (Sens, Spec) are not invariant properties of diagnostic and screening tests, but vary in different patient samples. Kraemer [Evaluating medical tests. Objective and quantitative guidelines. 1992] used the level of test, Q, also known as “positive sign rate” (sum of true and false positives divided by sample size), to calculate quality sensitivity and specificity (QSN, QSP). These scaled indices may be more comparable across different patient samples, but have been little studied hitherto. Methods: The dataset of a pragmatic test accuracy study of the Mini-Addenbrooke’s Cognitive Examination (MACE) was re-interrogated to calculate values of QSN and QSP and other paired and unitary test outcome measures based on them, and comparison was made with outcomes previously calculated by standard methods. Results: QSN and QSP values in this cohort (n = 755; overall prevalence of dementia and mild cognitive impairment [MCI] 0.15 and 0.29, respectively) were inferior to Sens and Spec, as were all other outcome measures for MACE for the diagnosis of both dementia and MCI. QSN was relatively preserved, indicating the sensitivity of MACE. Conclusion: Indices of test outcome scaled according to Kraemer’s Q, the positive sign rate, are less impressive than outcomes calculated by standard methods. These discrepancies may have implications for test evaluation.