Buyer beware: inflated claims of sensitivity for rapid COVID-19 tests

Abstract

Widespread COVID-19 testing is paramount for the receipt of timely medical care and for curtailing transmission. The USA continues to face formidable challenges in making testing accessible for all because efforts to scale up COVID-19 testing have fallen short.1Tromberg BJ Schwetz TA Pérez-Stable EJ et al.Rapid scaling up of COVID-19 diagnostic testing in the United States—the NIH RADx Initiative.N Engl J Med. 2020; 383: 1071-1077Crossref PubMed Scopus (114) Google Scholar RT-PCR testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which is considered to be the gold standard for identifying cases, is limited by processing time, exacerbating delays that are generated by surging demand. To expand testing capacity and accelerate diagnosis, the US Food and Drug Administration has issued an emergency use authorisation (EUA) for several diagnostic products, including six rapid antigen tests.2Office of the CommissionerEmergency use authorization.https://www.fda.gov/emergency-preparedness-and-response/mcm-legal-regulatory-and-policy-framework/emergency-use-authorizationDate: 2020Date accessed: October 20, 2020Google Scholar Although EUA for the rapid antigen tests provides essential countermeasures during this public health crisis, we outline the causes for concern regarding claims by manufacturers about performance metrics that might engender misinterpretation. The sensitivity and specificity of these tests have been presented by manufacturers in a way that inflates these performance characteristics. For example, the manufacturers of the BinaxNOW test (Abbott; Chicago, IL, USA) claimed a sensitivity of 97·1% (95% CI 85·1–99·9)3AbbottUpping the ante on COVID-19 antigen testing.https://www.abbott.com/corpnewsroom/product-and-innovation/upping-the-ante-on-COVID-19-antigen-testing.htmlDate: Aug 26, 2020Date accessed: October 14, 2020Google Scholar and the manufacturers of BD Veritor tests (Becton Dickinson; Franklin Lakes, NJ, USA) have claimed a sensitivity of 83·9% (66·3–94·5). However, the reported accuracy of these rapid antigen tests is actually the percent positive agreement (PPA) and not sensitivity. The PPA is measured relative to an RT-PCR test, which is imperfect itself.4Miller TE Garcia Beltran WF Bard AZ et al.Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital.FASEB J. 2020; (published online Aug 28.)https://doi.org/10.1096/fj.202001700RRCrossref Scopus (74) Google Scholar Due to variation in the diagnostic sensitivity of different RT-PCR tests, which are evaluated by the Foundation for Innovative New Diagnostics (FIND), understanding the accuracy of a rapid antigen test requires knowledge of the exact RT-PCR test that is selected as the comparator. However, manufacturers of most rapid antigen tests have not specified the test comparator. Compounding this uncertainty, the minimum sample size that is required to apply for EUA is 30 positive cases. Such small sample sizes have led to large CIs for the PPA. For example, the BD Veritor EUA study had 31 positive cases (PPA 83·9%, 95% CI 66·3–94·5). Combining the effect of small sample size with the reported sensitivity that is typical of RT-PCR (92·1%, 95% CI 86·6–95·9; over the first 7 days after symptom onset)4Miller TE Garcia Beltran WF Bard AZ et al.Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital.FASEB J. 2020; (published online Aug 28.)https://doi.org/10.1096/fj.202001700RRCrossref Scopus (74) Google Scholar would correspond to diagnostic sensitivities of 89·4% (81·7–94·7) for BinaxNOW and 77·3% (63·5–87·8) for BD Veritor. Furthermore, the real-world use of these antigen tests has extended beyond the EUA for postsymptom diagnosis to encompass routine screening. Screening is fundamental to the control of COVID-19, particularly because silent infections (ie, asymptomatic and presymptomatic infections) are major drivers of transmission.5Moghadas SM Fitzpatrick MC Sah P et al.The implications of silent transmission for the control of COVID-19 outbreaks.Proc Natl Acad Sci U S A. 2020; 117: 17513-17515Crossref PubMed Scopus (269) Google Scholar However, the performance of rapid antigen tests has not been evaluated for detection of asymptomatic infections or during the incubation phase. The dangers of disregarding or misunderstanding the imperfections in test sensitivity are evidenced by the outbreak that unfolded in the White House, which relied exclusively on rapid antigen screening as a sufficient measure to prevent transmission. Policy optimisation and implementation requires an accurate understanding of testing sensitivity. Numerous universities rely on antigen testing to screen students in congregate living facilities and identify infectious individuals for isolation. Many schools are examining testing as a pathway for safe instruction in person, despite high incidence in the community. University and school decisions about testing frequency and closing or isolation criteria often rely on risk tolerance for missing infections, the probabilities of which depend on test sensitivity. Adjusting from the reported test performance to real-world diagnostic sensitivity shows that such decision makers could be substantially underestimating the number of missed infections. For example, organisations relying on BinaxNOW miss three times as many infections as they have been led to believe. If rapid testing is going to become a viable, trusted screening strategy for control of COVID-19, then performance characteristics should be well understood and screening strategies should be designed with test imperfections clearly in mind. We declare no competing interests.