Quantitative scores for binary qualitative proficiency testing

Abstract

While z-scores provide participants with easy-to-interpret scores for quantitative proficiency tests, there is no universally accepted equivalent scoring method available for qualitative testing. Under the assumption that these tests follow a binomial distribution, it is possible to calculate scores that mimic the widely used z-scores and provide participants with insight into their performance level. We show that these scores, which we term a-scores, can be combined to provide a single score for multiple tests so that participants can monitor their performance over time, and discuss the use of the exact binomial test in place of uncertainty when there is no clear consensus.