Summary receiver operating characteristics (SROC) and hierarchical SROC models for analysis of diagnostic test evaluations of antibody ELISAs for paratuberculosis

Abstract

Critical, systematic reviews of available diagnostic test evaluations are a meticulous approach to synthesize evidence about a diagnostic test. However, often the review finds that data quality is poor due to deficiencies in design and reporting of the test evaluations and formal statistical comparisons are discouraged. Even when only simple summary measures are appropriate, the strong correlation between sensitivity and specificity and their dependence on differences in diagnostic threshold across studies, creates the need for tools to summarise properties of the diagnostic test under investigation.This study presents summary receiver operating characteristics (SROC) analysis as a means to synthesize information from diagnostic test evaluation studies. Using data from a review of diagnostic tests for ante mortem diagnosis of paratuberculosis as an illustration, SROC and hierarchical SROC (HSROC) analysis were used to estimate overall diagnostic accuracies of antibody ELISAs for bovine paratuberculosis while accounting for covariates: the target condition (infectious or infected) used in the test evaluation (one for the evaluation of Se and one for Sp); and the type of test (serum vs. milk). The methods gave comparable results (regarding the estimated diagnostic log odds ratio), considering the small sample size and the quality of data. The SROC analysis found a difference in the performance of tests when the target condition for evaluation of Se was infected rather than infectious, suggesting that ELISAs are not suitable for detecting infected cattle. However, the SROC model does not take differences in sample size between study units into account, whereas the HSROC allows for both between and within study variation. Considering the small sample size, more credibility should be given to the results of the HSROC. For both methods the area under the (H)SROC curve was calculated and results were comparable.The conclusion is that while the SROC is simpler and easier to implement, analyse and interpret, the HSROC does have properties encourage the extra effort involved in the analysis.