Interrater Reliability of Functional Status Scores for Patients Transferred From One Rehabilitation Setting to Another

Abstract

Kohler F, Redmond H, Dickson H, Connolly C, Estell J. Interrater reliability of functional status scores for patients transferred from one rehabilitation setting to another. Objective To report the interrater reliability of FIM total score, FIM motor subscore, and FIM cognitive subscore from scoring that occurred in routine clinical practice in 2 closely linked inpatient rehabilitation services in Sydney, Australia. Design A natural-experiment blind clinical interrater reliability cohort study of the FIM across 2 rehabilitation units. Setting This study is set in 2 inpatient rehabilitation units immediately adjacent to each other in southwestern Sydney, New South Wales, Australia. Participants All patients (N=143) who were transferred between the 2 rehabilitation units between August 2006 and October 2007 were included in the study. Intervention Discharge FIMs were scored by the first unit and an admission FIM was scored independently by the second unit within a few days. The FIM scores were analyzed for agreement and systematic bias. Main Outcome Measure Intraclass correlation coefficients, kappa statistic, weighted kappa statistic, and Bland-Altman plots were used. Results There were 143 sets of scores identified. The range of differences between the 2 FIM totals was −32 to 50, between the FIM motor subscores was −22 to 43, and between the FIM cognitive subscores was −14 to 21. Bland-Altman plots demonstrated poor agreement. Few FIM totals were perfectly matched. The intraclass correlation coefficients ranged from .872 for the FIM total to .830 for the cognitive subscales. Values for kappa ranged from −.007 (FIM motor subscore) to .123 (FIM cognitive subscore). Values for weighted kappa ranged from .465 (FIM cognitive subscore) to .521 (FIM total). Conclusions There was no systematic scoring bias evident. Intraclass correlation coefficients were high, but tests of agreement demonstrated poor agreement. These findings have implications for the use of the FIM and any patient classification or funding system based on the FIM, especially if poor levels of agreement were found in the presence of all staff being FIM credentialed and standardization of methods of assessment. This study indicates that further investigation of agreement of both FIM totals and FIM item scores in the clinical setting is warranted.