Assessing engagement in rehabilitation: development, validity, reliability, and responsiveness to change of the Rehabilitation Observation Measure of Engagement (ROME)

Abstract

Purpose We describe the development of an observational video coding tool, the Rehabilitation Observation Measure of Engagement (ROME), to quantify engagement in rehabilitative settings at the person (internal state of an individual) and between-system (interaction between individuals) level. Methods Forty-nine children with unilateral spastic cerebral palsy (29 males; Age: M = 9.28 yrs, SD = 3.08 yrs) and their interventionists were videotaped during different activities. Construct validity was examined by correlating the ROME with the Engagement vs. Disaffection with Learning Survey and the Pediatric Rehabilitation Intervention Measure of Engagement – Observation questionnaire. Inter- and intra-rater reliability were examined using two independent raters. The ROME’s responsiveness to change was examined by comparing scores across activities. Results For construct validity, results showed a positive correlation for person-level engagement (r = 0.444, p = 0.003). No relationship was found between-system-level engagement. High intrarater (91.8%) and interrater (96.1%) reliability was found. The ROME’s responsiveness to change was supported by children exhibiting lower engagement scores during repetitive shaping activities. Conclusion These findings provide evidence that the ROME is a reliable tool to objectively examine the construct of engagement within rehabilitation and is valid for quantifying person-level engagement. It provides information that cannot be extracted from questionnaires and can help guide intervention decisions.