Convergent and discriminant validity of the ImPACT with traditional neuropsychological measures.

Robert J. Thoma,Jessica Pommy,Ronald A. Yeo,Julia A. Cook,Richard A. Campbell,Dalin T. Pulsipher,John H. King,Christopher McGrew,Mollie A. Monnig,Andrew Mayer

doi:10.1080/23311908.2018.1430199

Abstract

Neuropsychological assessment of cognitive sequelae secondary to sports concussion is limited by lengthy administration times and lack of readily available neuropsychologists. Brief computerized test batteries are now under development to address this, but the validity of these measures is not yet established. The validity of one such computerized test battery, the Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), was administered to 93 healthy NCAA Division I athletes, aged 18–24, along with a battery of traditional, well-described neuropsychological tests. Convergent and discriminant validity between the ImPACT and traditional measures was investigated using multitrait-multimethod matrix (MTMM) analysis. As an example, the ImPACT Visual Motor Speed composite demonstrated reasonably good convergent validity secondary to moderate correlations with traditional measures of processing speed, but it demonstrated relatively poor discriminant validity as it significantly correlated with the Reaction Time composite score. MTMM results were variable across ImPACT composites and data for each are presented. The ImPACT composite’s validity was further investigated using exploratory factor analysis (EFA). Six principal components were termed processing speed, visual memory, verbal memory, attention & working memory, and verbal fluency, based upon traditional test loadings, and a sixth loaded only on the ImPACT Reaction Time composite. EFA indicated content validity of moderate strength for the Visual Motor Speed and Visual Memory composites, but revealed problems with specificity for the other composites. Based upon the present findings, validity problems render the interpretability of the ImPACT composites somewhat questionable, and more research is necessary prior to using the ImPACT for assessment of clinical populations.

Highlights

The summary from the Fifth International Conference on Concussion in Sport defines sports related concussion as “... a traumatic brain injury induced by biomechanical forces” (McCrory et al, 2017)
Multitrait-multimethod matrix Table 3 shows the partial correlations between Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) Composite Scores and all traditional neuropsychological test scores organized in terms of an MTMM
Further evidence for poor discriminant validity were the correlations found between the Visual Motor Composite score and other ImPACT Composite scores

Summary

Introduction

The summary from the Fifth International Conference on Concussion in Sport defines sports related concussion as “... a traumatic brain injury induced by biomechanical forces” (McCrory et al, 2017). A traumatic brain injury induced by biomechanical forces” (McCrory et al, 2017) Such a definition is inclusive but very imprecise, leading to substantial clinical heterogeneity. Due to this heterogeneity, concussion/mild traumatic brain injury (mTBI) in sports has received increasing attention in neuropsychology and sport medicine in recent years (Schatz, Pardini, Lovell, Collins, & Podell, 2006). Traditional neuropsychological tests are important diagnostic tools used by clinicians to assess neurocognitive impairment because neuropsychological tests may be sensitive to the subtle cognitive deficits associated with sports concussions in otherwise healthy, athletes ranging from elementary school age to the professional level (Collins, Lovell, & McKeag, 1999; Echemendia, Putukian, Mackin, Julian, & Shoss, 2001; Kelly, 2001; Macciocchi, Barth, Alves, Rimel, & Jane, 1996; McCrea, Kelly, Randolph, Cisler, & Berger, 2002; Randolph, McCrea, & Barr, 2005; Register-Mihalik et al, 2012). Two additional concerns are that (1) athletic trainers who administer the computer assessments may lack both the background to effectively utilize and evaluate results of the computer-based tests; and (2) trainers’ access to a neuropsychologist as a consultant may be limited (Randolph et al, 2005)

Methods

Results

Conclusion