A Reinforcement Learning Assisted Eye-Driven Computer Game Employing a Decision Tree-Based Approach and CNN Classification

Abstract

Human-Machine Interfaces employing biosignal-based inputs are hard to translate to real-life applications, in part because of the difficulty of developing generalized models to classify physiological events representing a user's actions. In the proposed framework, an Electrooculography (EOG)-based game is operated through a pipeline of decision methods. These include a user-independent classification model of eye movements using a Convolutional Neural Network (CNN), which is fed with images created from signal windows, and an Ensemble of Utility Decision Networks (EUDN), which moderates the impact of oftentimes conflicting ocular events while enabling a more natural level of control over the interface. The CNN and the EUDN replace the normally used feature-based ocular event detection methods for EOG. Finally, a Reinforcement Learning-based game actuation approach simultaneously updates multiple (State, Action) pairs for each rewarded outcome, intervenes to mitigate the consequences of wrongful game Commands, and can be used as part of a “shared-control” paradigm based on EOG. Results show a positive impact of Reinforcement Learning both in improving participants' game performance as well as in reducing some of their subjective workload indicators.