Valid and Reliable Assessment of Upper Respiratory Tract Specimen Collection Skills during the COVID-19 Pandemic.

Tobias Todsen,Thomas Benfield,Jacob Melchiors,Rebekka Consuelo Eið,Nikolai Kirkby,Lisette Hvid Hovgaard,Lars Konge,Morten B S Svendsen,Christian Von Buchwald,Anne Bohr,Martin Tolsgaard

doi:10.3390/diagnostics11111987

Abstract

Proper specimen collection is the most important step to ensure accurate testing for the coronavirus disease 2019 (COVID-19) and other infectious diseases. Assessment of healthcare workers’ upper respiratory tract specimen collection skills is needed to ensure samples of high-quality clinical specimens for COVID-19 testing. This study explored the validity evidence for a theoretical MCQ-test and checklists developed for nasopharyngeal (NPS) and oropharyngeal (OPS) specimen collection skills assessment. We found good inter-item reliability (Cronbach’s alpha = 0.76) for the items of the MCQ-test and high inter-rater reliability using the checklist for the assessment of OPS and NPS skills on 0.86 and 0.87, respectively. The MCQ scores were significantly different between experts (mean 98%) and novices (mean 66%), p < 0.001, and a pass/fail score of 91% was established. We found a significant discrimination between checklist scores of experts (mean 95% score for OPS and 89% for NPS) and novices (mean 50% score for OPS and 36% for NPS), p < 0.001, and a pass/fail score was established of 76% for OPS and 61% for NPS. Further, the results also demonstrated that a group of non-healthcare educated workers can perform upper respiratory tract specimen collection comparably to experts after a short and focused simulation-based training session. This study, therefore, provides validity evidence for the use of a theoretical and practical test for upper respiratory specimens’ collection skills that can be used for competency-based training of the workers in the COVID-19 test centers.

Highlights

Testing for the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)causing the coronavirus disease 2019 (COVID-19) is an essential part of the pandemic control [1]
A multiple-choice test (MCQ) and checklists following the World Health Organization (WHO) recommendations to assess nasopharyngeal swabbing (NPS) and OPS performance were constructed, and experts and novices were invited to participate in a validation study
Results from 64 upper respiratory tract specimens’ collection skills (URTS) MCQ tests and 128 videos of OPS and NPS performance were available for assessment with the URTS checklist

Summary

Introduction

Testing for the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)causing the coronavirus disease 2019 (COVID-19) is an essential part of the pandemic control [1]. For initial SARS-CoV-2 infections, an upper respiratory tract specimen is recommended to be collected for molecular or antigen testing. The World Health Organization (WHO) recommends performing oropharyngeal (OPS) and nasopharyngeal swabbing (NPS) to ensure the collection of representative upper respiratory tract specimens for COVID-19 testing [3]. The test procedures can be technically difficult to perform, and due to the significant and unmet need for healthcare workers at the many community-based COVID-19 test sites, non-healthcare workers may be used to collect upper respiratory tract specimens [5,6]. A pre-requisite for competency-based training is an assessment tool with established validity evidence for evaluating upper respiratory specimen sampling techniques at the COVID-19 test sites

Objectives

Methods

Results

Discussion

Conclusion