Mobile Eye Tracking Research Articles

Comparing the Perception of In-Person and Digital Monitor Viewing of Paintings

In the context of rapidly developing technologies and widespread online access, it is important to understand how our perception of images on a computer screen may vary from traditional in-person encounters. This research compared the perception of subjects in view of eight paintings, presented either on a computer monitor or as a printed reproduction. Both stationary and mobile eye-tracking technologies were used to analyze the viewing patterns of both forms of engagement. Results suggested that subjects engaging with physical works tended to exhibit more varied fixational patterns than those viewing the same works on a computer monitor. Data showed parity in the high degree of correlation between viewing times and personal preference, regardless of viewing medium. The results indicate that the modalities through which we engage with works of art matter, and that a single image can resonate across an array of media.

The visual control of locomotion when stepping onto moving surfaces: A comparison of younger and older adults

Stepping between static and moving surfaces presents a locomotor challenge associated with increased injury frequency and severity in older adults. The current study evaluates younger and older adults' behaviours when overcoming challenges sampling moving walkway and escalator environments. Twelve younger adults (18–40 years, Male = 8) and 15 older adults (60–81 years, Male = 5) were examined using an integration of optoelectronic motion capture and mobile eye-tracking. Participants were investigated approaching and stepping onto a flat conveyor belt (static or moving; with or without surface (demarcation) lines). Specifically, the four conditions were: (i) static surface without demarcation lines; (ii) static surface with demarcation lines; (iii) moving surface without demarcation lines; and (iv) moving surface with demarcation lines.A two (age group) x two (surface-condition) x two (demarcation-condition) linear mixed-model revealed no main or interaction effects (p > .05) for perturbation magnitude, indicating participants maintained successful locomotion. However, different adaptive behaviours were identified between conditions with moving and accuracy demands (e.g., moving surfaces increased step length, demarcations reduced step length). Between subject effects identified differences between age groups. Older adults utilised different behaviours, such as earlier gaze transfer from the final approach walkway step location. Overall, the current study suggests that adaptive behaviours emerge relative to the environment's specific demands and the individual's action capabilities.

Mobile Eye Tracking during Experimenting with Digital Scaffolding—Gaze Shifts between Augmented Reality and Experiment during Zinc Iodide Electrolysis Set-Up

In this exploratory eye tracking study, gaze behavior during an experiment on electrolysis of a zinc iodide solution has been examined. The experiment is designed according to inquiry-based learning and therefore well suited to students’ self-directed experimenting in science classrooms. Scaffolding by means of stepwise presented digital hints (sequential scaffolds), here by means of augmented reality (AR), supported learners during the experiment. Two succeeding AR-hints were offered. This research focuses on the set-up of the experiment by learners and the change of gaze between the AR-hints and the real experiment. This was investigated by means of mobile eye tracking. For the analysis of the gaze changes, scanpath analyses are used. Areas of Interest-based visualizations such as radial transition graphs and scarf plots are used to compare and detect similar or different gaze behaviors of participants. Results reveal that students first look at the AR-hints, try to memorize the construction and then switch to setting up the experiment. Recurrent changes of gaze between the AR-hints and the experiment hardly, or do not, occur. Furthermore, students who had already set up the experiment correctly with the first hint only use the second hint for evaluating correctness of their own set-up.

Measuring Cyclist’s Inputs, the Kinematic and Dynamic Properties of a City Bicycle, and Estimating the Road Profile via Sensor Fusion

In this paper, we present the instrumentation of a city bicycle with different sensors and devic-es in order to measure cyclists’ inputs (i.e., pedaling and steering) and the dynamical and kinematic properties of the bicycle. The instrumentation includes two tri-axial accelerometers, an Inertial Measurement Unit (IMU), GPS, a potentiometer, a laser scanner, a pedaling power meter, and speed and cadence sensors, in addition to a mobile eye tracker worn by the cyclists. After the instrumentation and adjustment of the sensors, a study was conducted in the city of Stockholm using the instrumented bicycle with the aim to evaluate cycling safety and comfort on snowy surface conditions. The outputs of this experiment will be employed further to study the interaction of cyclists with road infrastructure and other road users and their impact on cyclists’ behavior and cycling safety.

Mobile eye tracking applied as a tool for customer experience research in a crowded train station.

Train stations have increasingly become crowded, necessitating stringent requirements in the design of stations and commuter navigation through these stations. In this study, we explored the use of mobile eye tracking in combination with observation and a survey to gain knowledge on customer experience in a crowded train station. We investigated the utilization of mobile eye tracking in ascertaining customers' perception of the train station environment and analyzed the effect of a signalization prototype (visual pedestrian flow cues), which was intended for regulating pedestrian flow in a crowded underground passage. Gaze behavior, estimated crowd density, and comfort levels (an individual's comfort level in a certain situation), were measured before and after the implementation of the prototype. The results revealed that the prototype was visible in conditions of low crowd density. However, in conditions of high crowd density, the prototype was less visible, and the path choice was influenced by other commuters. Hence, herd behavior appeared to have a stronger effect than the implemented signalization prototype in conditions of high crowd density. Thus, mobile eye tracking in combination with observation and the survey successfully aided in understanding customers' perception of the train station environment on a qualitative level and supported the evaluation of the signalization prototype the crowded underground passage. However, the analysis process was laborious, which could be an obstacle for its practical use in gaining customer insights.

What we see is what we do: a practical Peripheral Vision-Based HMM framework for gaze-enhanced recognition of actions in a medical procedural task

Deep learning models have shown remarkable performances in egocentric video-based action recognition (EAR), but rely heavily on a large quantity of training data. In specific applications with only limited data available, eye movement data may provide additional valuable sensory information to achieve accurate classification performances. However, little is known about the effectiveness of gaze data as a modality for egocentric action recognition. We, therefore, propose the new Peripheral Vision-Based HMM (PVHMM) classification framework, which utilizes context-rich and object-related gaze features for the detection of human action sequences. Gaze information is quantified using two features, the object-of-interest hit and the object–gaze distance, and human action recognition is achieved by employing a hidden Markov model. The classification performance of the framework is tested and validated on a safety-critical medical device handling task sequence involving seven distinct action classes, using 43 mobile eye tracking recordings. The robustness of the approach is evaluated using the addition of Gaussian noise. Finally, the results are then compared to the performance of a VGG-16 model. The gaze-enhanced PVHMM achieves high classification performances in the investigated medical procedure task, surpassing the purely image-based classification model. Consequently, this gaze-enhanced EAR approach shows the potential for the implementation in action sequence-dependent real-world applications, such as surgical training, performance assessment, or medical procedural tasks.

Moderate Severity SARS-CoV-2 (COVID-19) Affects Ocular Vergence Indices: Eye Tracking-Based Study.

Since the start of the SARS-CoV-2 (COVID-19) pandemic, it has become clear that the brain is one of the main targets for acute and chronic damage. Although neurodegenerative changes have yet to be investigated, there is already a large body of data on damage to its fiber tracts. A mobile eye tracker is possibly one of the best tools to study such damage in a COVID hospital setting. At the same time, the available data indicate that eye tracking parameters, even in healthy volunteers, demonstrate a distinct gender-specific difference.The aim of the work is to evaluate functional and structural impairments of the fiber tracts and to find possible gender-specific dynamics of eye tracking indicators in the acute period of COVID-19 pneumonia (Delta variant) of moderate severity. A single-center non-randomized retrospective study included 84 patients in the acute period of moderate severity SARS-CoV-2 (COVID-19) pneumonia (Delta variant) (Group 1). The mean time from admission was 1.4±1.2days. M:41, F:43. According to thoracic CT, the lung involvement ranged from CT 1 to CT 2. SpO2 ranged from 95% to 99%. The mean age was 35.5±14.8years (from 18 to 60). The control group (Group 2) included 158 healthy volunteers without pathology of the vision organs and central nervous system.The eye vergence index (VRx) was determined using eye tracking as a motion correlation coefficient between the angular velocities of the left and right eyeballs and was a measure of the conjugation of horizontal and vertical eye movements.The mobile complex Eye Tracker Low-Speed 20 (BVG LLC, the Netherlands) was used. Eye tracking parameters were assessed by vertical and horizontal eye vergence (VVRx and HVRx).Statistical analysis was done using the methods of parametric and non-parametric statistics. Moderate COVID-19 pneumonia resulted in a significant decrease in both VVRx and HVRx compared to controls (0.763±0.127 and 0.856±0.043; p<0.000001; 0.729±0.018 and 0.776±0.023 p<0.000001, respectively). VVRx values were significantly higher in men (0.775±0.046 and 0.747±0.091, p=0.019, respectively), while ХVRx values were significantly higher in women (0.665±0.018 and 0.728±0.024, p<0.0000001, respectively). SARS-CoV-2 (COVID-19) of moderate severity is accompanied by a significant deterioration in eye tracking performance proving functional and structural impairments (p<0.05). VVRx was significantly higher in men, and HVRx was substantially greater in women reflecting gender-specific differences.

Eye Tracking in Dogs: Achievements and Challenges.

In this article, we review eye-tracking studies with dogs (Canis familiaris) with a threefold goal; we highlight the achievements in the field of canine perception and cognition using eye tracking, then discuss the challenges that arise in the application of a technology that has been developed in human psychophysics, and finally propose new avenues in dog eye-tracking research. For the first goal, we present studies that investigated dogs' perception of humans, mainly faces, but also hands, gaze, emotions, communicative signals, goal-directed movements, and social interactions, as well as the perception of animations representing possible and impossible physical processes and animacy cues. We then discuss the present challenges of eye tracking with dogs, like doubtful picture-object equivalence, extensive training, small sample sizes, difficult calibration, and artificial stimuli and settings. We suggest possible improvements and solutions for these problems in order to achieve better stimulus and data quality. Finally, we propose the use of dynamic stimuli, pupillometry, arrival time analyses, mobile eye tracking, and combinations with behavioral and neuroimaging methods to further advance canine research and open up new scientific fields in this highly dynamic branch of comparative cognition.

Using Natural Head Movements to Continually Calibrate EOG Signals.

Electrooculography (EOG) is the measurement of eye movements using surface electrodes adhered around the eye. EOG systems can be designed to have an unobtrusive form-factor that is ideal for eye tracking in free-living over long durations, but the relationship between voltage and gaze direction requires frequent re-calibration as the skin-electrode impedance and retinal adaptation vary over time. Here we propose a method for automatically calibrating the EOG-gaze relationship by fusing EOG signals with gyroscopic measurements of head movement whenever the vestibulo-ocular reflex (VOR) is active. The fusion is executed as recursive inference on a hidden Markov model that accounts for all rotational degrees-of-freedom and uncertainties simultaneously. This enables continual calibration using natural eye and head movements while minimizing the impact of sensor noise. No external devices like monitors or cameras are needed. On average, our method's gaze estimates deviate by 3.54° from those of an industry-standard desktop video-based eye tracker. Such discrepancy is on par with the latest mobile video eye trackers. Future work is focused on automatically detecting moments of VOR in free-living.

Using Mobile Dual Eye-Tracking to Capture Cycles of Collaboration and Cooperation in Co-located Dyads

The goal of this paper is to bring new insights to the study of social learning processes by designing measures of collaboration using high-frequency sensor data. More specifically, we are interested in understanding the interplay between moments of collaboration and cooperation, which is an understudied area of research. We collected a multimodal dataset during a collaborative learning activity typical of makerspaces: learning how to program a robot. Pairs of participants were introduced to computational thinking concepts using a block-based environment. Mobile eye-trackers, physiological wristbands, and motion sensors captured their behavior and social interactions. In this paper, we analyze the eye-tracking data to capture participants’ tendency to synchronize their visual attention. This paper provides three contributions: (1) we use an emerging methodology (mobile dual eye-tracking) to capture joint visual attention in a co-located setting and replicate findings that show how levels of joint visual attention are positively correlated with collaboration quality; (2) we qualitatively analyzed the co-occurrence of verbal activity and joint visual attention in low and high performing groups to better understand moments of collaboration and cooperation; (3) inspired by the qualitative observations and theories of collaborative learning, we designed a new quantitative measure that captures cycles of collaborative and cooperative work. Compared to simple measures of joint visual attention, we found it to increase correlation coefficients with learning and collaboration scores. We discuss those results and describe how advances in analyzing sensor data can contribute to theories of collaboration. We conclude with implications for capturing students’ interactions in co-located spaces using Multimodal Learning Analytics (MMLA).

Do executive function performance, gaze behavior, and pupil size change during incremental acute physical exercise?

Several studies have investigated the interaction between acute physical exercise and cognitive performance. However, few studies have investigated this issue during acute high-intensity exercise. In the present study, we evaluated executive functions (EFs) during incremental exercise in three different intensities [below lactate threshold (LT), at LT, and above LT], measuring EFs performance, gaze behavior, and pupil diameter. Twenty subjects were familiarized with the EFs test and participated in a graded maximal exercise test on a cycle ergometer on the first visit. On the second visit, they performed the EFs task at rest and while exercising at three different intensities using mobile eye-tracking glasses. Our results showed that the psychophysiological measures differed between the conditions. Regarding EFs performance, during exercise above LT, the subjects showed worse accuracy when compared with rest (p < .001) and below LT (p < .001). In addition, the response time (RT) at LT and above LT was shorter than in the rest condition (p < .050). Further, RT was faster (p=.002) in the above LT than in the below LT condition. In addition, the gaze behavior measures indicated that exercise, independently of the intensity, improves the number of fixations with shorter fixation durations compared to the rest condition (p < .050). Additionally, we found no significant differences in average and peak pupil diameter between conditions. In conclusion, exercise at LT improves the EFs performance while exercising above LT worsens EFs performance. However, there were no significant differences in average and peak pupil diameter between conditions.

Methodological limits on sampling visual experience with mobile eye tracking

Methodological limits on sampling visual experience with mobile eye tracking

Cognitive load, working memory capacity and driving performance: A preliminary fNIRS and eye tracking study

A common cause of road traffic incidents is driver distraction, which can occur when the driver’s attention is engaged in a concurrent secondary task. However, the relationship between cognitive demands, individual differences in working memory capacity and driving performance has received little research attention. Using a fixed-base driving simulator, the aim of this study was to use a combination of self-report, functional near infrared spectroscopy (fNIRS) and mobile eye tracking data to investigate the impact of cognitive load on drivers’ gaze behaviour and driving performance, as well as explore the relationship between working memory capacity and driving performance under increased cognitive load. Twenty-one participants with a range of driving experiences (e.g., 0–14 h per week) completed a simulated driving task in a simple environment (i.e., country highway) under single-task (driving only) and dual-task (driving + modified 2-back task) conditions. Cognitive load was assessed via fNIRS data that manifested as changes in regional oxygen saturation (rSO2) in prefrontal cortex as well as self-report data of perceived mental effort. Participants’ working memory capacity was assessed using the Operation Span Task. Findings showed that prefrontal rSO2 and perceived mental effort was significantly greater under dual-task conditions compared to the single-task condition. In the dual-task condition, participants’ gaze dwelled for longer on the road and they made fewer fixations, of longer durations. Participants were able to maintain driving performance in this condition, although this was at the expense of secondary task performance. Interestingly, driving infractions under dual-task conditions were negatively correlated with participants’ working memory capacity. The findings suggest that engaging with distracting secondary tasks while driving may increase drivers’ cognitive load and change their gaze behaviour. Driving performance can seemingly be maintained under such conditions, but this may be partly determined by the driver’s working memory capacity.

Mutable Observation Used by Television Drone Pilots: Efficiency of Aerial Filming Regarding the Quality of Completed Shots

Drones, as mobile media of the present day, increase the operational and narrative capabilities of television and accelerate the logistics of shooting. Unmanned aerial vehicles with a camera properly steered by a pilot are able, to some extent, to replace a jimmy jib/crane and a dolly; basic technical devices, used in the studios, enabling the creation of narrative systems of pictures in film and television. Television is more and more often using drone footage to report events, broadcast live, as well as create coverage and television documentaries. In many productions, the pilot of the drone simultaneously acts as the drone camera operator, which can improve the effectiveness of shooting, but also carries some risks related to flight safety. The article describes and presents in the form of processed footage the real conditional ties of a Visual Line of Sight (VLOS) flight faced by pilots filming with a drone. VLOS is a type of air operation, which requires maintaining eye contact with the drone. In many countries, a drone visibility flight is legally sanctioned as VLOS Operation. An experiment was conducted to investigate the interactions between a human and a machine in airspace steered using a controller with a touchscreen. The drone pilot was considered an integral part of the drone’s flight system control Experimental data was collected with the use of a mobile eye-tracker, video cameras, surveys, and pilot declarations. During the experiment, eight television drone pilot operators recaptured a model shot under the regime of VLOS flight at low altitudes. They all show that both advanced and beginner pilots did not look at the UAV for over half the time of shot execution. The experiment allowed establishing two coefficients related to the effectiveness of a VLOS flight aiming at filming from the drone. The results point to clear differences in screen perception styles used by drone television pilots. The coefficients were described in the form of mathematical formulas and their limit values were determined. The research also determines the limits of pilots’ perception, within which they can film with a drone. The outcomes may help to optimize the process of aerial filming with the use of a drone, carried out for television, film, and other media, as well as in a simulation of such a flight for research and training. From the perspective of media science and social communication, the presented study included a technological component that can be accessed through information science, using statistical models and variable distributions. Media scholars can study the impact of the media without having to look into the metaphorical black box. Computer science opens up this possibility.

Use of a DNN in Recording and Analysis of Operator Attention in Advanced HMI Systems

The main objective of this research was to propose a smart technology to record and analyse the attention of operators of transportation devices where human–machine interaction occurs. Four simulators were used in this study: General Aviation (GA), Remotely Piloted Aircraft System (RPAS), AS 1600, and Czajka, in which a spatio-temporal trajectory of system operator attention describing the histogram distribution of cockpit instrument observations was sought. Detection of the position of individual instruments in the video stream recorded by the eyetracker was accomplished using a pre-trained Fast R-CNN deep neural network. The training set for the network was constructed using a modified Kanade–Lucas–Tomasi (KLT) algorithm, which was applied to optimise the labelling of the cockpit instruments of each simulator. A deep neural network allows for sustained instrument tracking in situations where classical algorithms stop their work due to introduced noise. A mechanism for the flexible selection of Area Of Interest (AOI) objects that can be tracked in the recorded video stream was used to analyse the recorded attention using a mobile eyetracker. The obtained data allow for further analysis of key skills in the education of operators of such systems. The use of deep neural networks as a detector for selected instrument types has made it possible to universalise the use of this technology for observer attention analysis when applied to a different objects-sets of monitoring and control instruments.

Relations between social attention, expressed positive affect and behavioral inhibition during play.

Flexible social attention, including visually attending to social interaction partners, coupled with positive affect may facilitate adaptive social functioning. However, most research assessing social attention relies on static computer-based paradigms, overlooking the dynamics of social interactions and limiting understanding of individual differences in the deployment of naturalistic attention. The current study used mobile eye-tracking to examine relations between social attention, expressed affect, and behavioral inhibition during naturalistic play in young children. Children (N = 28, Mage = 6.12, 46.4% girls, 92.9% White) participated in a 5-min free play with a novel age- and sex-matched peer while mobile eye-tracking data were collected. Interactions were coded for social attention and expressed affect and modeled second-by-second, generating 4,399 observations. Children spent more time dwelling on toys than on peers or anywhere else in the room. Further analyses demonstrated children were almost twice as likely to gaze at their peer when simultaneously self-expressing positive affect. Additionally, children were more than twice as likely and more than three times as likely to self-express positive affect when dwelling on peer or in the presence of peer-expressed positive affect, respectively. Behavioral inhibition was not significantly related to social attention. However, children higher in behavioral inhibition were less likely to self-express positive affect in the presence of peer-expressed positive affect. The current results provide a snapshot of relations between social attention, expressed affect and individual differences during play and provide guidance for future work assessing the roles of social attention and positive affect in facilitating positive social interactions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Eye-Head-Body Coordination in the Motor-Memory Trade-off

Previous studies have demonstrated that people trade off between expending motor effort and memorization effort when completing a copying task that requires looking in different directions to gather information. When information is easy to gather with minimal movement, people choose to look more frequently and rely on less on memorization, however, they reduce motor effort and rely on memory more when looking requires larger (and more effortful) shifts of gaze. This paper investigated the eye-head-body movements that guided looking as well as participants’ trade-off between using motor effort and memory in a copying task. We add to prior work by characterizing the coordination between eyes, head, and body required to look at targets at different angles. In the task, participants copied models onto a board while wearing a mobile eye tracker that measured eye rotations and motion sensors that measured head rotations. We manipulated the angle of the model relative to the participant and the memorization difficulty of the models. Results showed that eye movements contributed more to small gaze shifts but the head and body contributed more as the angle of the target increased (requiring larger amplitude shifts of gaze). Participants chose to look frequently when models were adjacent to the workspace, but chose to rely more on memory for models that required greater shifts of gaze. Memorization difficulty affected the trade-off such that participants used more memory when the models were easy to remember. We discuss how participants’ decisions to look versus remember may depend on the contribution of eyes, head, and body to shift gaze by varying amplitudes.

Tracking visual attention during learning of complex science concepts with augmented 3D visualizations

The main purpose of the study was to explore how different augmented 3D visualizations may affect the learning of complex science concepts using the eye tracking method. An Augmented Reality (AR) learning system incorporated with different 3D visualizations was developed to assist conceptual learning in the topic of “molecular shapes.” Learners' visual attention during the AR learning was recorded using a mobile eye tracking system. Participants were 32 10th grade students. Pre- and post-tests were used to assess students' learning achievements. Learners were grouped into low and high achievers according to the posttest result. To analyze the associations between learners' visual attentions and learning achievements, correlation, t-test, correlation, and regression analyses were applied. Additionally, we conducted lag sequential analysis (LSA) to compare the visual transitions of different groups of achievers. The results showed that learners’ conceptual understanding was improved after the AR learning activity. Among the different 3D visualizations, learners attended more to the static and dynamic types in the basic learning stage and to the static and interactive types in the advanced learning stage. Attention to the 3D static model was found to associate more to the attention to the interactive type. The correlation and regression analyses found that the cognitive effort to integrate information about the static 3D model was the key predictor for the learning achievement. LSA showed that high achievers displayed a higher chance of looking back to the static 3D model from other AOIs. The visual pattern analysis suggested that a guidance to connect information delivered by the static and the animated visual representations is necessary for complex conceptual learning.

"What needs to be seen": An exploration into the visual anticipation behaviour of different skill-level football referees while observing long passes on-field.

It is well established that elite football referees possess superior anticipatory skills in specific game scenarios such as when assessing foul situations. Referees might also have better anticipatory skills in other important scenarios such as when observing a long pass. In these often-occurring situations, a referee has to use visual information to anticipate the outcome of the pass, in particular to foresee any potential infringements that might occur when players battle for ball possession. However, little is known about if and how football referees might anticipate outcomes in these scenarios. The aim of the current study was therefore to analyse the visual anticipatory behaviour of football referees when long passes occur during actual football matches. Elite (N=4) and sub-elite referees (N=12) officiated an actual football match while wearing a mobile eye-tracker to analyse their gaze behaviour when long passes occurred (N=196). The results revealed differences in the way that the elite and sub-elite referees tracked the ball and anticipated the outcome of the ball trajectories. The elite referees used a lower search rate (1.3 vs 1.8 fix/s; p<.05) and were more likely to direct their gaze towards the ball during the moment of kick (77 vs 52%; p<.05) and the early flight-phase of the pass (68 vs 45%; p<.05), and subsequently produced earlier anticipatory eye movements to the player(s) receiving the ball (at 50% vs 60% of the ball flight; p<.05). This earlier anticipation may help the elite referees to better pick-up relevant information about the receivers that could be vital in making adjudications about any potential infringement when the ball does arrive. Referee education programs can use the current study to highlight the importance of visual search behaviour and help referees to adapt a strategy that is beneficial for long-pass situations.

How do people distribute their attention while observing The Night Watch?

This study explored how people look at The Night Watch (1642), Rembrandt's masterpiece. Twenty-one participants each stood in front of the painting for 5 min, while their eyes were recorded with a mobile eye-tracker and their thoughts were verbalized with a think-aloud method. We computed a heatmap of the participants’ attentional distribution using a novel markerless mapping method. The results showed that the participants’ attention was mainly directed at the faces of the two central figures, the bright mascot girl in the painting, and detailed elements such as the apparel of the key figures. The eye-movement analysis and think-aloud data also showed that participants’ attention shifted from the faces of the key figures to other elements of the scene over the course of the 5 min. Our analyses are consistent with the theory that Rembrandt used light and texture to capture the viewer's attention. Finally, the robustness of the eye-tracking method was demonstrated by replicating the study on a smaller replica.