Using machine learning based on eye gaze to predict targets: An exploratory study

Abstract

Play is a crucial activity for child development. Play in children with physical disabilities may be compromised due to their physical limitations, such as having difficulties reaching and manipulating objects. Assistive technology robotic systems have been used as tools for children with disabilities to play and interact with the environment. Robots have shown a positive impact on children's independence, cognitive, and social skills. The present study is the first stage of a project to develop a telerobotic haptic system, with the goal of supporting the reaching of toys during play by children with severe physical disabilities. The end goal is to provide haptic guidance towards the toys that the children want to play with. The objective of this paper was to investigate the feasibility of predicting the selection of targets in a three-block task. This prediction was based on the Point of Gaze (POG) data of five participants while performing the task using a telerobotic haptic system. Two fixation-based algorithms, longest fixation and last fixation, and two learning algorithms, a Double Q-learning and a Multi-Layer Perceptron neural network, were implemented, tested, and compared. Results showed that the learning algorithms were better at predicting the targets than the fixation-based algorithms, with above 92% accuracy. This demonstrated that the learning algorithms can be utilized for activating haptic guidance towards the targets (toys).