Analysing domain-specific problem-solving processes within authentic computer-based learning and training environments by using eye-tracking: a scoping review

Abstract

Recently, many studies have been published on the use of eye-tracking to analyse complex problem-solving processes within authentic computer-based learning and training environments. This scoping review aims to provide a systematic report of the current state-of-the-art for related papers. Specifically, this work offers a scoping review of studies that analyse problem-solving processes by using eye-tracking (alongside additional process data such as log files, think aloud, facial expression recognition algorithms, or psychophysiological measures) within authentic technology-based learning and training environments for professional and vocational education and training (VET). A total of 12 studies were identified. The most commonly calculated measures in eye-tracking research are position measures, and these are almost exclusively position duration measures such as the proportion of fixation times or total dwell times. Count measures are also mostly related to the number or proportion of fixations and dwells. Movement measures are rarely computed and usually refer to saccade directions or a scan path. Also, latency and distance measures are almost never calculated. Eye-tracking data is most often analysed for group comparisons between experts vs. novices or high vs. low-performing groups by using common statistical methods such as t-test, (M)ANOVA, or non-parametric Mann–Whitney-U. Visual attention patterns in problem-solving are examined with heat map analyses, lag sequential analyses, and clustering. Recently, linear mixed-effects models have been applied to account for between and within-subjects differences. Also, post-hoc performance predictions are being developed for future integration into multimodal learning analytics. In most cases, self-reporting is used as an additional measurement for data triangulation. In addition to eye-tracking, log files and facial expression recognition algorithms are also used. Few studies use shimmer devices to detect electrodermal activity or practice concurrent thinking aloud. Overall, Haider and Frensch’s (1996, 1999) “information reduction hypothesis” is supported by many studies in the sample. High performers showed a higher visual accuracy, and visual attention was more focused on relevant areas, as seen by fewer fixation counts and higher fixation duration. Low performers showed significantly fewer fixation durations or substantially longer fixation durations and less selective visual attention. Performance is related to prior knowledge and differences in cognitive load. Eye-tracking, (in combination with other data sources) may be a valid method for further research on problem-solving processes in computer-based simulations, may help identify different patterns of problem-solving processes between performance groups, and may hold additional potential for individual learning support.