Recognition of map activities using eye tracking and EEG data

Abstract

Recognizing the activities being performed on a map is crucial for adaptive map design based on user context. Despite eye tracking (ET) demonstrating potential in recognizing map activities and electroencephalography (EEG) measuring map users’ cognitive load, no studies have yet combined ET and EEG for recognition of the user’s activity on maps. Our study collected participants’ ET and EEG data during four types of map activities. After feature extraction and selection, we trained LightGBM (light Gradient-Boosting Machine) to classify these activities, and achieved 88.0% accuracy when combining ET and EEG features in the entire map usage trial, which is higher than using ET (85.9%) or EEG (53.9%) alone. Acceptable recognition accuracy could also be achieved with the early time windows (73.1% when using the first 3 seconds). Saccade features of ET were the most important for differentiating map activities, indicating selective map content for different tasks. Our findings demonstrate the feasibility and advantages of combining ET and EEG for activity recognition in map use. The results not only improve our understanding of visual patterns and cognitive processes in map use, but also enable the design of adaptive maps that can automatically adapt to the activities a map user is performing.