Excellent reliability of the ALSFRS-R administered via videoconferencing: A study of people with motor neuron disease in Scotland

Abstract

•The ALSFRS-R can reliably be administered via videoconferencing•For some people with motor neuron disease it is easier to communicate via videoconferencing than telephone•People with motor neuron disease have a positive attitude towards videoconferencing Motor neuron disease is characterised by progressive weakness and disability, meaning that travel to specialist MND care and research participation can quickly become difficult. Many pwMND, particularly those living in remote areas, face troublesome commutes to access care and to participate in research. Travel-related restrictions have been shown to impact on attrition in clinical trials [[1]Atassi N. Yerramilli-Rao P. Szymonifka J. et al.Analysis of start-up, retention, and adherence in ALS clinical trials.Neurology. 2013; 81 (2013/09/04): 1350-1355https://doi.org/10.1212/WNL.0b013e3182a823e0Crossref PubMed Scopus (18) Google Scholar] and are likely to contribute to drop-out of clinic-based assessment. By improving accessibility, reducing costs and mitigating geographical distances, videoconferencing has the potential to improve clinical care for pwMND, increase retention in clinical trials and capture a more diverse cohort of pwMND [[2]Hobson E.V. Baird W.O. Bradburn M. et al.Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study.BMJ Open. 2019; 9e028525https://doi.org/10.1136/bmjopen-2018-028525Crossref PubMed Scopus (11) Google Scholar]. Our aim is to enhance the incorporation of telehealth in research and clinical care by demonstrating that the amyotrophic lateral sclerosis functional rating scale (ALSFRS-R), a frequently used clinical instrument, can be reliably administered using videoconferencing. We also hope to show that videoconferencing has additional advantages over telephone-based assessments; for example, facilitating non-verbal communication and supporting pwMND with bulbar impairment to communicate remotely by typing free text. The ALSFRS-R can be reliably administered face-to-face or via telephone [[3]Kaufmann P. Levy G. Montes J. et al.Excellent inter-rater, intra-rater, and telephone-administered reliability of the ALSFRS-R in a multicenter clinical trial.Amyotroph. Lateral Scler. 2007; 8 (2007/03/17): 42-46https://doi.org/10.1080/17482960600888156Crossref PubMed Scopus (93) Google Scholar,[4]Mannino M. Cellura E. Grimaldi G. et al.Telephone follow-up for patients with amyotrophic lateral sclerosis.Eur. J. Neurol. 2007; 14 (2007/01/16): 79-84https://doi.org/10.1111/j.1468-1331.2006.01559.xCrossref PubMed Scopus (26) Google Scholar] but administration via videoconferencing has not been investigated. We completed a service evaluation to determine the reliability of videoconferencing-based administration of the ALSFRS-R by comparing it to face-to-face administration. To distinguish between inter-rater and inter-method reliability, we assessed inter-rater reliability separately by comparing ALSFRS-R scores between videoconferencing assessments by different raters. We recruited a convenience sample of pwMND between August 2018 and August 2019, residing in mainland Scotland and remote islands, with a range of disability. Participants were recruited from the Scottish MND register, the Clinical Audit Research and Evaluation of MND (CARE-MND) platform. One group of participants completed assessments with the ALSFRS-R via face-to-face appointments and videoconferencing (n = 20) and another group completed videoconferencing assessments scored by two independent raters (n = 20); two participants underwent both. Face-to-face and videoconferencing administration of the ALSFRS-R occurred within a 15-day period and were performed by different raters. Raters used the European Network to Cure ALS (ENCALS) standards for administration [[5]European Network to Cure ALS TW. ENCALS SOP for ALSFRS-R v1.2. Access via URL: https://www.encals.eu/wp-content/uploads/2016/09/ENCALS-SOP-for-ALSFRS-R-v1.2.pdf.Google Scholar]. Videoconferencing was undertaken via the NHS Attend Anywhere platform (attendanywhere®), which is approved for use by NHS Scotland. Participants views on the use of videoconferencing were also ascertained in a subset of participants (n = 20). Internal consistency of ALSFRS-R was determined using Cronbach's coefficient alpha. Inter-rater and inter-method reliability were assessed using a two-way random effects model with single measures for absolute agreement (ICC) and graphically displayed using a Bland-Altman plot. The minimal detectable change (MDC) was calculated as follows: MDC = 1.96 × SD × √ (1 − ICC) × √ 2. Effect measures are displayed with 95% confidence intervals (CIs) and summary statistics as mean and standard deviation (SD) or median and interquartile range (IQR). Analysis was undertaken in SPSS (version 25). 38 pwMND were included. All had ALS, 81.6% (31/38) were male, mean age was 62.6 years (SD ± 12.2). 81.6% had spinal onset (31/38) and 18.4% (7/38), had bulbar onset. The mean ALSFRS-R was 30.6 (SD 13.0, range 0–45). There was excellent reliability and internal consistency in ALSFRS-R scores between videoconference and face-to-face administration by two independent raters (ICC = 0.99, 95%CIs 0.99, 0.99; Cronbach's alpha = 0.99, Fig. 1). The limits of agreement were ± 3 and the MDC was 3.6, that is, the smallest change in ALSFRS-R score that is not due to chance. Assessments took place within a median of 0 days (IQR 0–6). The inter-rater reliability and internal consistency between videoconference administration of ALSFRS-R by two independent raters was also excellent (ICC = 0.99, 95% CI 0.99, 0.99; Cronbach's alpha = 0.99). Two participants utilised Eyegaze® software in conjunction with their PC and participated in ALSFRS-R scoring by typing responses in the chat box through the videoconferencing software. 90% (18/20) of pwMND described the set up as simple and 85% (17/20) rated the call quality as “good” or “excellent”; none felt that any disruption adversely affected the consultation. Participants positively commented on the convenience, reduction in travel time and flexibility of timing and location associated with videoconferencing. All participants were keen to use videoconferencing for future consultations although 35% (7/20) wished to continue face-to-face consultations. Our results support the incorporation of videoconferencing-based ALSFRS-R assessments into research and clinical practice. By reducing burden of participation, telehealth facilitates involvement in trials by pwMND who may not otherwise be able to participate, for example, those with severe disability and those living remotely. A recent telehealth versus usual care trial reports that several commonly identified barriers to research participation, time, fatigue and the impact of research on day-to-day life, were not an issue for the pwMND allocated to the telehealth group. The investigators recommended telehealth as a cost-effective and low-burden tool for collection of outcome measures [[2]Hobson E.V. Baird W.O. Bradburn M. et al.Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study.BMJ Open. 2019; 9e028525https://doi.org/10.1136/bmjopen-2018-028525Crossref PubMed Scopus (11) Google Scholar]. In keeping with our findings, previous feasibility studies report that pwMND have a positive attitude towards videoconferencing and enjoy working with technology [[6]Helleman J. Kruitwagen E.T. van den Berg L.H. et al.The current use of telehealth in ALS care and the barriers to and facilitators of implementation: a systematic review.in: Amyotrophic Lateral Sclerosis & Frontotemporal Degeneration. 2019: 1-16https://doi.org/10.1080/21678421.2019.1706581Crossref Scopus (17) Google Scholar]. Moreover, videoconferencing was found to contribute to a sense of increased enablement amongst pwMND [[7]de Almeida J.P. Pinto A.C. Pereira J. et al.Implementation of a wireless device for real-time telemedical assistance of home-ventilated amyotrophic lateral sclerosis patients: a feasibility study.Telemedicine Journal and E-Health : The Official Journal of the American Telemedicine Association. 2010; 16 (2010/10/12): 883-888https://doi.org/10.1089/tmj.2010.0042Crossref PubMed Scopus (27) Google Scholar]. It enables local therapists [[8]Nijeweme-d’Hollosy W.O. Janssen E.P. Huis in ’t Veld RM, et al Tele-treatment of patients with amyotrophic lateral sclerosis (ALS).J. Telemed. Telecare. 2006; 12 (2006/08/04): 31-34https://doi.org/10.1258/135763306777978434Crossref PubMed Scopus (29) Google Scholar] and relatives living separately to attend the consultation. Compared to telephone consultations, videoconferencing offers several practical advantages for pwMND: We found that the ability to type free text improved the ease of communication for pwMND with bulbar impairment and that people using assistive communication devices were able to use the chat facility in conjunction with their devices. Alternatively, one might consider a self-administered version of the ALSFRS-R, which was found to be highly reliable [[9]Montes J. Levy G. Albert S. et al.Development and evaluation of a self-administered version of the ALSFRS-R.Neurology. 2006; 67: 1294-1296https://doi.org/10.1212/01.wnl.0000238505.22066.fcCrossref PubMed Scopus (43) Google Scholar]. Despite the multitude of benefits associated with videoconferencing consultations, participants in this study felt that face-to-face consultations remain an important part of their care. Others reported that pwMND did not feel comfortable discussing sensitive topics, such as end-of-life care, via videoconferencing [[8]Nijeweme-d’Hollosy W.O. Janssen E.P. Huis in ’t Veld RM, et al Tele-treatment of patients with amyotrophic lateral sclerosis (ALS).J. Telemed. Telecare. 2006; 12 (2006/08/04): 31-34https://doi.org/10.1258/135763306777978434Crossref PubMed Scopus (29) Google Scholar] or commented on the lack of “touch” [[10]Geronimo A. Wright C. Morris A. et al.Incorporation of telehealth into a multidisciplinary ALS Clinic: feasibility and acceptability.Amyotrophic Lateral Sclerosis and Frontotemporal Degeneration. 2017; 18: 555-561https://doi.org/10.1080/21678421.2017.1338298Crossref PubMed Scopus (37) Google Scholar]. In conclusion, ALSFRS-R administered through videoconferencing is reliable and has the potential to reduce burden of clinical reviews for pwMND and increase engagement and retention in clinical trials. Videoconferencing has additional advantages over telephone-based assessments by allowing pwMND with bulbar impairment to type free text. Nonetheless, face-to-face assessment remains an important component of clinical care.