Validation of a Machine Learning Brain Electrical Activity–Based Index to Aid in Diagnosing Concussion Among Athletes

Abstract

An objective, reliable indicator of the presence and severity of concussive brain injury and of the readiness for the return to activity has the potential to reduce concussion-related disability. To validate the classification accuracy of a previously derived, machine learning, multimodal, brain electrical activity-based Concussion Index in an independent cohort of athletes with concussion. This prospective diagnostic cohort study was conducted at 10 clinical sites (ie, US universities and high schools) between February 4, 2017, and March 20, 2019. A cohort comprising a consecutive sample of 207 athletes aged 13 to 25 years with concussion and 373 matched athlete controls without concussion were assessed with electroencephalography, cognitive testing, and symptom inventories within 72 hours of injury, at return to play, and 45 days after return to play. Variables from the multimodal assessment were used to generate a Concussion Index at each time point. Athletes with concussion had experienced a witnessed head impact, were removed from play for 5 days or more, and had an initial Glasgow Coma Scale score of 13 to 15. Participants were excluded for known neurologic disease or history within the last year of traumatic brain injury. Athlete controls were matched to athletes with concussion for age, sex, and type of sport played. Classification accuracy of the Concussion Index at time of injury using a prespecified cutoff of 70 or less (total range, 0-100, where ≤70 indicates it is likely the individual has a concussion and >70 indicates it is likely the individual does not have a concussion). Of 580 eligible participants with analyzable data, 207 had concussion (124 male participants [59.9%]; mean [SD] age, 19.4 [2.5] years), and 373 were athlete controls (187 male participants [50.1%]; mean [SD] age, 19.6 [2.2] years). The Concussion Index had a sensitivity of 86.0% (95% CI, 80.5%-90.4%), specificity of 70.8% (95% CI, 65.9%-75.4%), negative predictive value of 90.1% (95% CI, 86.1%-93.3%), positive predictive value of 62.0% (95% CI, 56.1%-67.7%), and area under receiver operator characteristic curve of 0.89. At day 0, the mean (SD) Concussion Index among athletes with concussion was significantly lower than among athletes without concussion (75.0 [14.0] vs 32.7 [27.2]; P < .001). Among athletes with concussion, there was a significant increase in the Concussion Index between day 0 and return to play, with a mean (SD) paired difference between these time points of -41.2 (27.0) (P < .001). These results suggest that the multimodal brain activity-based Concussion Index has high classification accuracy for identification of the likelihood of concussion at time of injury and may be associated with the return to control values at the time of recovery. The Concussion Index has the potential to aid in the clinical diagnosis of concussion and in the assessment of athletes' readiness to return to play.