Interpreting reliability of early motor function measurement following head injury

Abstract

Reliable measurement of motor function in adults following head injury (HI) is necessary to plan and evaluate the effectiveness of treatment interventions, and essential to the understanding of the process of recovery following HI. The purpose of this report is to compare reliability estimates obtained using two methods of assessment that can be used in the acute care setting. Reliability estimates associated with live assessments of motor function were compared to those associated with videotaped assessments of motor function in the same group of HI subjects. The data are compared to illustrate possible sources of error associated with the clinical measurement of motor function. Twelve adults with a severe HI were assessed, using standardised testing protocols, on eight motor skills and two primitive reflexes. Subjects' live and videotaped assessments provided data to estimate intra- and inter-rater reliability. Intra-rater kappa values ranged from 0.61 to 1.0 for the live assessments. Reliability of motor skills (videotape) ranged from 0.62 to 1.0 slightly exceeding that of motor skills (live). Lowest kappa values were found for primitive reflexes (videotape) that ranged from 0.18 to 1.0. Inter-rater reliability kappa values exceeded 0.62 for motor skills and ranged from 0.37 to 1.0. for primitive reflexes. Kappa values for motor skills (live and videotape) were of similar magnitude. Lower kappa values were associated with the measurement of primitive reflexes, ranging from 0.53 to 0.63 for live assessments and from 0.37 to 1.0 for videotaped assessments. Comparison of these data revealed that a number of sources of error (including motor recovery) may mask any true estimate of reliability obtained with this complex patient population. Caution is therefore recommended when interpreting reliability studies in this population in the acute care setting.