A novel fusion strategy for locomotion activity recognition based on multimodal signals

Abstract

Detecting locomotion activities and gait phases are critical for recognizing movement intent and assisting humans in activities of daily living. This paper introduces a novel fusion strategy that analyzes multimodal signals (electromyograms, angular rates, and footswitches) to identify four lower limb locomotion activities (walking, running, stair ascent, and descent), four gait phases (push-up, stance, step-up, and swing) and detect gait events (heel strike and toe-off). This strategy uses different networks (convolutional neural network and Bi-directional Long Short-Term Memory) and a feature fusion method (Canonical Correlation Analysis) to perform the classification tasks and gait event detection, which transcends the limitations of conventional methods that only analyze a single signal. Then, an adaptive online algorithm is proposed, which helps the model to automatically fine-tune its parameters and gradually approach the convergence state. The offline experiment results on 20 healthy subjects show an average accuracy of 99.33 % and an average Matthews correlation coefficient of 99.56. Besides, with the help of the adaptive online algorithm, lower errors are shown in gait event detection, including 18.43±7.38 ms for heel strike and 22.25±7.06 ms for toe-off. Consequently, our proposed model can provide more accurate quantitative results than mainstream methods, giving it a competitive advantage for future applications.