Cluster Kernel Reinforcement Learning-based Kalman Filter for Three-Lever Discrimination Task in Brain-Machine Interface

Abstract

Brain-Machine Interface (BMI) translates paralyzed people's neural activity into control commands of the prosthesis so that their lost motor functions could be restored. The neural activities represent brain states that change continuously over time which brings the challenge to the online decoder. Reinforcement Learning (RL) has the advantage to construct the dynamic neural-kinematic mapping during the interaction. However, existing RL decoders output discrete actions as a classification problem and cannot provide continuous estimation. Previous work has combined Kalman Filter (KF) with RL for BMI, which achieves a continuous motor state estimation. However, this method adopts a neural network structure, which might get stuck in local optimum and cannot provide an efficient online update for the neural-kinematic mapping. In this paper, we propose a Cluster Kernel Reinforcement Learning-based Kalman Filter (CKRL-based KF) to avoid the local optimum problem for online neural-kinematic updating. The neural patterns are projected into Reproducing Kernel Hilbert Space (RKHS), which builds a universal approximation to guarantee the global optimum. We compare our proposed algorithm with the existing method on rat data collected during a brain control three-lever discrimination task. Our preliminary results show that the proposed method has a higher trial accuracy with lower variance across data segments, which shows its potential to improve the performance for online BMI control. Clinical Relevance- This paper provides a more stable decoding method for adaptive and continuous neural decoding. It is promising for clinical applications in BMI.