QROC curves and kappa functions: new methods for evaluating the quality of clinical decisions

Abstract

The application of statistical methods for measuring the ability of clinical judgements and tests to discriminate between normal and diseased patients is increasing in optometry and ophthalmology. This paper presents two new methods for assessing the quality of such diagnostic decisions. Both are based on measurement of the kappa coefficient of association and offer theoretical and practical advantages over traditional methods. The paper first introduces the family of kappa coefficients with examples of their calculation and application. The use of weighted kappa values for measuring test sensitivity and specificity is explained, and it is shown how the results may be plotted in the form of the QROC curve which enables different aspects of test discriminability to be represented graphically. Kappa functions are then proposed as an alternative method for representing and predicting test performance, having the advantage of identifying optimal test criteria in addition to providing measures of discriminabilty. The clinical application of the new methods is illustrated using data for the detection of glaucoma from measurements of intraocular pressure.