Offline Evaluation Matters: Investigation of the Influence of Offline Performance of EMG-based Neural-Machine Interfaces on User Adaptation, Cognitive Load, and Physical Efforts in a Real-Time Application.

Abstract

There has been controversy about the value of offline evaluation of EMG-based neural-machine interfaces (NMIs) for their real-time application. Often, conclusions have been drawn after studying the correlation of the offline EMG decoding accuracy/error with the NMI user's real-time task performance without further considering other important human performance metrics such as adaptation rate, cognitive load, and physical effort. To fill this gap, this study aimed to investigate the relationship between the offline decoding accuracy of EMG-based NMIs and user adaptation, cognitive load, and physical effort in real-time NMI use. Twelve non-disabled subjects participated in this study. For each subject, we established three EMG decoders that yielded different offline accuracy (low, moderate, and high) in predicting continuous hand and wrist motions. The subject then used each EMG decoder to perform a virtual hand posture matching task in real time with and without a secondary task as the evaluation trials. Results showed that the high-level offline performance decoders yield the fastest adaptation rate and highest posture matching completion rate with the least muscle effort in users during online testing. A secondary task increased the cognitive load and reduced real-time virtual task competition rate for all the decoders; however, the decoder with high offline accuracy still produced the highest task completion rate. These results imply that the offline performance of EMG-based NMIs provide important insight to users' abilities to utilize them and should play an important role in research and development of novel NMI algorithms.