Receiver Operating Characteristic Analysis for Classification Based on Various Prior Probabilities of Groups with an Application to Breath Analysis

Abstract

HE PRIOR PROBABILITIES are independent of measured data and known before taking any observations. The change of this information changes the probability of a correct output of a diagnostic test [5]. In this paper we show how radical these changes are in classification of measured concentrations of volatile organic compounds of smokers and non-smokers. The ROC (receiver operating characteristic) curve is a metric for comparing predicted and actual target values in a classification model. The ROC curve plots sensitivity and (1 specificity) of the diagnostic test. The sensitivity measures the proportion of actual positives which are correctly identified as such (i.e. the percentage of sick people who are identified as having the condition); and the specificity measures the proportion of negatives which are correctly identified (i.e. the percentage of healthy people who are identified as not having the condition). Different classification algorithms use different techniques for finding relationships between the measured values of subjects (e.g. concentrations of selected volatile organic compounds, VOCs, of breath profile) and the known targets (association with groups, e.g. smokers or non-smokers). We use the discriminant function g(X) with a threshold (the decision point used by the model for classification) dependent on prior probabilities of groups, [5]. The ROC curve measures the impact of changes in the threshold. For the ROC curve related to changes of prior probabilities we constructed the asymptotic pointwise confidence interval, [4] (CI describes the range where the true ROC curve lies with some specific probability, e.g. 95% CI). To evaluate effectiveness of classification based on different prior probabilities of discriminated classes we use the Youden index [3]. This index ranges between 0 and 1, with a value close to 1 indicating that the effectiveness of algorithm is relatively large and a value close to 0 indicating limited effectiveness. For the Youden index we constructed the asymptotic pointwise confidence interval, too. We apply the classification on breath analysis data. Breath analysis as a non-invasive technique is very attractive because it can be easily applied to sick patients, including children and elderly people. It offers potential for detection of some aaaaaaa