The statistical evaluation of binary tests without gold standard: Robustness of latent variable approaches

Abstract

Binary tests classify items into two categories such as reject/accept or positive/negative. Such tests are usually evaluated in terms of their misclassification probabilities FAP (false acceptance probability) and FRP (false rejection probability). A common complication arises when there is no gold standard or reference standard. Various methods based on latent variable modelling have been proposed for this situation. We present the results of a simulation study in which these methods are tried in a range of scenarios, to study how robust they are to departures from the assumptions on which they are based. The study convincingly shows that in general, the ambition of estimating FAP and FRP without gold standard is unattainable, since all methods easily derail when assumptions are not precisely met. The study also shows that the random components of the FAP and FRP can be reliably estimated by a straightforward modification of one of the tested methods.