Measuring group social interactions following acquired brain injury: an inter-rater reliability evaluation

Abstract

ABSTRACT Background Social communication impairments following acquired brain injury (ABI) are well-documented. There is evidence that group interventions are beneficial but research into validated instruments to measure group outcomes is a new field of investigation. Aims This study reports on the inter-rater reliability of three established social communication measures for use with group interaction data: the Profile of Pragmatic Impairment in Communication (PPIC), the Behaviorally Referenced Rating System of Intermediate Social Skills (BRISS-R), the Adapted Measure of Participation in Conversation (MPC). Inter-rater reliability of the Interactional Network Tool (INT), a new digital tool designed for group interactional behaviours, is also evaluated. Method Thirty-one video samples of ABI group interactions were independently rated by two rater pairs using the four outcome measures. Inter-rater reliability was calculated using intra-class correlations (ICC). Results ICC estimates and their 95% confidence intervals were calculated for the different measures. The measures showed differential sensitivity. Rater agreement on the MPC interaction (ICC = 0.77) and transaction (ICC = 0.74) scales was moderate to good. The INT initiation frequencies (ICC = 0.83) were moderate to excellent and the INT response frequencies (ICC = 0.69) were poor to good. Poor to moderate reliability was achieved on the BRISS-R PCSS (ICC = 0.49) and PDBS (ICC = 0.50) scale and PPIC findings were moderate but showed presence of skew. Conclusion Acceptable reliability was achieved on two measures of participation (MPC and INT). The INT shows promise as a new method to characterise interactions and detect change in group communication behaviour.