Inter- and Intra-Rater Reliability of a Virtual Concussion Assessment During the Era of the COVID-19 Pandemic (P8-1.004)

Abstract

<h3>Objective:</h3> To assess inter-examiner agreement of virtual concussion telemedicine examinations and to determine agreement of virtual and in-person concussion examinations performed by the same physician. <h3>Background:</h3> We developed a virtual concussion telemedicine examination by adapting several in-office examination methods. The virtual exam was compared to the in-person concussion exam to examine reliability and to validate the use of telemedicine via audio-video conferencing technology. <h3>Design/Methods:</h3> A virtual concussion examination form with instructions on how to perform this was developed. The standardized examination included 29 elements, such as orthostatic tolerance, ocularmotor exam, and balance tests. We enrolled 21 participants referred for an initial concussion evaluation at the NYU Concussion Center. Two virtual concussion telemedicine examinations were conducted in the office setting following study enrollment; one examination was perfomed by the treating physician and another was compled by another physician. The in-person concussion examination was then performed by the treating physician. We used Cohen’s Kappa to determine inter-modality and inter-examiner agreement. <h3>Results:</h3> We determined Cohen’s kappa to assess agreement on dichotomous examination ratings for each of 29 exam elements across 21 participants. Kappa values for inter-examiner agreement ranged from 0.31–1.0, with median kappa=0.76; 45% of exam elements had excellent inter-examiner agreement between the two telemedicine examiners (kappa&gt;0.75), while 75% of exam elements had at least intermediate inter-examiner agreement for telemedicine examiners (kappa&gt;0.40). Within the same examiner for telemedicine vs. in-person examinations, Cohen’s kappa values were even higher, with values ranging from 0.48–1.0 overall. <h3>Conclusions:</h3> We found that the treating physician’s virtual and in-person concussion examination findings largely agreed. However, there was less agreement between virtual concussion examinations performed by two different physicians. This suggests that expertise and experience of examining physicians contribute more to variability than do modality or area of the examination. This study sets the stage for investigations of reliability between in-person and teleneurology examinations. <b>Disclosure:</b> Ms. Jack has nothing to disclose. Miss Digney has nothing to disclose. Mr. Bell has nothing to disclose. Binu Joseph has nothing to disclose. Ms. Hyman has nothing to disclose. Dr. Galetta has nothing to disclose. An immediate family member of Dr. Balcer has received personal compensation in the range of $5,000-$9,999 for serving as a Consultant for Children’s Hospital of Philadelphia. Dr. Balcer has received personal compensation in the range of $50,000-$99,999 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for North American Neuro-Ophthalmology Society. Prof. Willer has nothing to disclose. Mr. Haider has received personal compensation in the range of $10,000-$49,999 for serving as a Consultant for BlinkCNS. The institution of Mr. Haider has received research support from NIH. Dr. Saleem has nothing to disclose. Dr. Grossman has nothing to disclose. Dr. Leddy has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Neurolign. Dr. Leddy has received personal compensation in the range of $500-$4,999 for serving on a Scientific Advisory or Data Safety Monitoring board for Stage 2 Contract Engineering. Dr. Leddy has received stock or an ownership interest from Highmark Innovations. The institution of Dr. Leddy has received research support from NIH. The institution of Dr. Leddy has received research support from DoD. The institution of Dr. Leddy has received research support from AMSSM. Dr. Busis has received personal compensation in the range of $500-$4,999 for serving as a Consultant for American Academy of Neurology. Dr. Busis has received personal compensation in the range of $0-$499 for serving as an Editor, Associate Editor, or Editorial Advisory Board Member for Neurology Today. Dr. Torres has received publishing royalties from a publication relating to health care.