A machine learning approach to primacy-peak-recency effect-based satisfaction prediction

Abstract

Data retrieval systems supporting the discovery and reuse of open data are emerging as important tools in the open data ecosystem. However, user satisfaction with them is relatively low. This study proposes the primacy-peak-recency effect to investigate the cognitive mechanisms underlying data searchers’ overall satisfaction. To test the primacy-peak-recency effect, primacy-peak-recency cubes consisting of eye movement indicators at primacy, peak, and recency moments and their combinations are constructed as the theoretical model. A user experiment was conducted to collect eye movement data and satisfaction scores generated during 48 doctoral students’ interactions with data retrieval systems. An ensemble machine learning framework was then applied to analyze eye movement data to assess the theoretical model. The results indicate that the primacy-peak-recency cubes are salient predictors of data searchers’ satisfaction (the prediction accuracy=0.682 and regression R2=0.749). This finding suggests that data searchers’ complex cognitive processes at primacy, peak, and recency moments measured by uni-, bi-, and three-dimensional eye movement indicators are predictors of overall satisfaction, confirming the primacy-peak-recency effect. In addition, combinations of varying types of influential moments and multidimensional eye movement events are the best predictors of overall satisfaction. This suggests that influential moments and cognitive processes have additive effects on overall satisfaction. Combining theory-driven and data-driven approaches, this study sheds light on the potential of machine learning approaches for analyzing neuropsychological data for heuristics examination. With these insights, practical strategies to predict data searchers’ satisfaction and optimize the user-experience design of data retrieval systems are proposed.