Analysis Of Eye-tracking Data Research Articles

DLEmotion: Deep learning-based emotion classification using visual attention location information

The correct recognition of each person's emotion can be an important step in knowing mental states and reactions. Because of the complexities of human personality, this is a difficult task and providing a method that can classify emotions with high accuracy can be useful. The purpose of doing this work is to provide a suitable method for recognizing emotions in human subjects. In other words, in this research the aim is to analyze eye tracking data using deep learning approaches and for this purpose, deep neural networks are used to classify fixations in different emotional states. The eye data of 10 women and 7 men who watched 79 emotional images are first collected. In this way, for each fixation point obtained from the subjects, a patch of 80 x 80 pixels was selected from the images and we used these patches of the obtained image as the input to the deep neural networks. The images shown to the subjects have three basic dimensions of emotion, i.e. Valence, Arousal and Dominance. Data classification was done by transfer learning and using Xception, InceptionV3 and DenseNet201 deep neural networks whose weights were trained on ImageNet datasets. We used these three neural networks in our work because they have relatively equal number of parameters and will help us in better analysis of the obtained results. In general, the results show that the points obtained from subjects' fixations location on images with three basic dimensions of emotion can be used to classify emotions. We will also see that changing the size of the patches obtained from the subject's fixation will not have much effect on the test results, which shows that deep neural networks are a powerful and accurate tool for processing people's eye fixation points.

AI-Assisted Diagnostics in Dentistry: An Eye-Tracking Study on User Behavior.

Artificial Intelligence (AI) has increasingly been integrated into dental practices, notably in radiographic imaging like Orthopantomograms (OPGs), transforming diagnostic protocols. Eye tracking technology offers a method to understand how dentists' visual attention may differ between conventional and AI-assisted diagnostics, but its integration into daily clinical practice is challenged by the cost and complexity of traditional systems. Thirty experienced practitioners and dental students participated to evaluate the effectiveness of two low-budget eye-tracking systems, including the Peye Tracker (Eye Tracking Systems LTD, Southsea, UK) and Webgazer.js (Brown University, Providence, Rhode Island) in a clinical setting to assess their utility in capturing dentists' visual engagement with OPGs. The hardware and software setup, environmental conditions, and the process for eye-tracking data collection and analysis are illustrated. The study found significant differences in eye-tracking accuracy between the two systems, with Webgazer.js showing higher accuracy compared to Peye Tracker (p<0.001). Additionally, the influence of visual aids (glasses vs. contact lenses) on the performance of eye-tracking systems revealed significant differences for both Peye Tracker (p<0.05) and Webgazer.js (p<0.05). Low-budget eye-tracking devices present challenges in achieving the desired accuracy for analyzing dentists' visual attention in clinical practice, highlighting the need for continued innovation and improvement in this technology. Key words:Artificial intelligence, Eye-tracking device, low-budget, dentistry.

Lévy Flight Model of Gaze Trajectories to Assist in ADHD Diagnoses.

The precise mathematical description of gaze patterns remains a topic of ongoing debate, impacting the practical analysis of eye-tracking data. In this context, we present evidence supporting the appropriateness of a Lévy flight description for eye-gaze trajectories, emphasizing its beneficial scale-invariant properties. Our study focuses on utilizing these properties to aid in diagnosing Attention-Deficit and Hyperactivity Disorder (ADHD) in children, in conjunction with standard cognitive tests. Using this method, we found that the distribution of the characteristic exponent of Lévy flights statistically is different in children with ADHD. Furthermore, we observed that these children deviate from a strategy that is considered optimal for searching processes, in contrast to non-ADHD children. We focused on the case where both eye-tracking data and data from a cognitive test are present and show that the study of gaze patterns in children with ADHD can help in identifying this condition. Since eye-tracking data can be gathered during cognitive tests without needing extra time-consuming specific tasks, we argue that it is in a prime position to provide assistance in the arduous task of diagnosing ADHD.

Visual Field Restriction in the Recognition of Basic Facial Expressions: A Combined Eye Tracking and Gaze Contingency Study.

Uncertainties and discrepant results in identifying crucial areas for emotional facial expression recognition may stem from the eye tracking data analysis methods used. Many studies employ parameters of analysis that predominantly prioritize the examination of the foveal vision angle, ignoring the potential influences of simultaneous parafoveal and peripheral information. To explore the possible underlying causes of these discrepancies, we investigated the role of the visual field aperture in emotional facial expression recognition with 163 volunteers randomly assigned to three groups: no visual restriction (NVR), parafoveal and foveal vision (PFFV), and foveal vision (FV). Employing eye tracking and gaze contingency, we collected visual inspection and judgment data over 30 frontal face images, equally distributed among five emotions. Raw eye tracking data underwent Eye Movements Metrics and Visualizations (EyeMMV) processing. Accordingly, the visual inspection time, number of fixations, and fixation duration increased with the visual field restriction. Nevertheless, the accuracy showed significant differences among the NVR/FV and PFFV/FV groups, despite there being no difference in NVR/PFFV. The findings underscore the impact of specific visual field areas on facial expression recognition, highlighting the importance of parafoveal vision. The results suggest that eye tracking data analysis methods should incorporate projection angles extending to at least the parafoveal level.

EyeScrollR: A software method for reproducible mapping of eye-tracking data from scrollable web pages

The Internet has become an important part of our lives and an increasing number of researchers use eye-tracking technology to examine attention and behavior in online environments. Researchers, however, face a significant challenge in mapping eye-tracking data from scrollable web pages. We describe the R package eyeScrollR for mapping eye-tracking data from scrollable content such as web pages. The package re-maps eye-tracking gaze coordinates to full-page coordinates with a deterministic algorithm based on mouse scroll data. The package includes options for handling common situations, such as sticky menus or ads that remain visible when the user scrolls. We test the package’s validity in different hardware and software settings and on different web pages and show that it is highly accurate when tested against manual coding. Compared to current methods, eyeScrollR provides a more reproducible and reliable approach for mapping eye-tracking data from scrollable web pages. With its open code and free availability, we recommend eyeScrollR as an essential tool for eye-tracking researchers, particularly those who adhere to open-science principles. The eyeScrollR package offers a valuable contribution to the field of eye-tracking research, facilitating accurate and standardized analysis of eye-tracking data in web scrolling contexts.

(No) Effects of a Self-Kindness Intervention on Self-Esteem and Visual Self-Perception: An Eye-Tracking Investigation on the Time-Course of Self-Face Viewing.

Previous research has suggested a favorable impact of self-kindness on subjective well-being. The present experiment investigated the effects of an app-assisted self-kindness intervention for increasing self-esteem and self-face gaze, and for decreasing depression. We explored self-face processing via a time-course analysis of eye-tracking data. Eighty participants (56 female, 24 male; mean age: 23.2 years) were randomly allocated to one of two intervention groups, each receiving daily instructions to enhance either self-kindness or relaxation (active control). Following a one-week intervention period, both groups reported improved self-esteem (p = .035, ηpart2 = .068) and reduced depression (p < .001, ηpart2 = .17). The duration of self-face gaze increased in both groups (p < .001, ηpart2 = .21). Self-face processing was characterized by an early automatic attention bias toward the self-face, with a subsequent reduction in self-face bias, followed in turn by an attentional self-face reapproach, and then a stable self-face bias. We thus identified a complex temporal pattern of self-face inspection, which was not specifically altered by the intervention. This research sheds light on the potential for app-assisted interventions to positively impact psychological well-being, while also highlighting the complexity of self-face processing dynamics in this context. In the future, we propose the inclusion of personalized self-kindness statements, which may amplify the benefits of these interventions.

Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programming: An analysis of eye-tracking and retrospective think-aloud data from program comprehension

Previous studies have reported mixed results regarding the relationship between students’ use of self-regulated learning (SRL) strategies and their performance in introductory programming courses. These studies were constrained by their reliance on self-report questionnaires as a means of collecting and analysing data. To address this limitation, this study aimed to employ eye-tracking and retrospective think-aloud techniques to identify differences in SRL strategy use for program comprehension tasks between high-performing students (N = 31) and low-performing students (N = 31) in an undergraduate programming course. All participants attended individual eye-tracking sessions to comprehend two Python program codes with different constructs. Their eye-tracking data and video-recalled retrospective think-aloud data were captured and recorded for analysis. The findings reveal that higher-order cognitive skills, such as elaboration and critical thinking, were mostly adopted by high-performing students, while basic cognitive and resource management strategy, such as rehearsal and help-seeking, were mostly employed by low-performing students when comprehending the program codes. This study not only demonstrates the design of combining eye-tracking and retrospective think-aloud data to explore students’ use of SRL strategies but also provides evidence to support the notion that program comprehension is a complex process that cannot be effectively addressed by employing merely rudimentary strategies, such as repetitively reading the same code segment. In the future, researchers could explore the possibility of using a webcam to monitor and assess students’ online programming processes and provide feedback based on their eye movements. They could also examine the effects of SRL strategies training on students’ motivation, engagement, and performance in various types of programming activities.

Understanding Consumers’ Visual Attention in Mobile Advertisements: An Ambulatory Eye-Tracking Study with Machine Learning Techniques

As mobile devices have become a necessity in our daily lives, mobile advertising is also prevalent. Accordingly, it is critical for practitioners to understand how consumers visually attend to mobile advertisements. One popular way of doing so is via eye-tracking methodology. However, scant eye-tracking research exists in mobile settings due to technical challenges, e.g., cumbersome data annotation. To tackle these challenges, the authors propose an object-detection machine learning (ML) algorithm—You Only Look Once (YOLO) v3—to analyze eye-tracking videos automatically. Moreover, we extend the original YOLO v3 model by developing a novel algorithm to optimize the analysis of eye-tracking data collected from mobile devices. Through a lab experiment, we investigate how two types of ad elements (i.e., textual vs. pictorial) and shopping devices (i.e., mobile vs. PC) affect consumers’ visual attention. Our findings suggest that (1) textual ad elements receive more attention than pictorial ones, and such differences are more pronounced in ads on mobile devices than those on PCs; and (2) mobile ads receive less attention than PC ads. Our findings provide managerial insights into developing effective digital advertising strategies to improve consumers’ visual attention in online and mobile advertisements.

Analysis of objective eye tracking data used for vehicle assessment

This paper describes the application of eye tracking technology in the automotive industry in the context of vehicle assessment in customer clinics. The paper presents the mentioned technology, the applied experimental methodology and selected results. In particular, the hypotheses that were expressed during the research of the topic and during the preparation of the experiment are answered. In this way, the innovative technology of eye tracking can be used to reflect the "voice of the customer" in the vehicle design.

Analyzing Cognitive Load Associated with Manual Text Classification Task Using Eye Tracking

The accuracy of machine learning-based automated text classification systems, such as spam filters and search engine results, heavily depends on the quality of manual text classification. However, the cognitive demands of manual text classification tasks, particularly when dealing with challenging or difficult-to-comprehend texts, have not been extensively explored in previous studies. This research aims to address this gap by investigating the cognitive load associated with manual text classification tasks through analyzing eye tracking data. In this study, 30 participants performed manual text classification tasks while their ocular parameters were recorded using an eye tracker. The findings of this study revealed that ocular parameters recorded through eye tracking provided valuable insights into the cognitive load experienced during manual text classification tasks. Furthermore, it was observed that complex narratives led to higher cognitive load estimation. Moreover, native English-speaking participants exhibited lower cognitive load, compared to non-native English speakers.

The visual complexity of bike maps

Abstract. More and more cities try to encourage residents to cycle more. Therefore, governments are developing comprehensive bike maps to facilitate trip planning and increase the popularity of cycling. However, research on the topic of bike maps is rare and the versatility of possible features shown on a bike map makes these visually more complex than others. To understand how base maps and the display of cycling related features affect the visual complexity of bike maps and thus their effectiveness, we used different metrics (GMLMT, Subband Entropy, Edge Density, Feature Congestion, and Distinct Object-Type Counts) on four bike maps with four different visual complexity levels. We ran an eye-tracking experiment with 35 participants solving four different everyday tasks with these four bike maps. The findings suggest that adding more detail to base maps and displaying more cycling related features on a map resulted in a visually more complex bike map. Size, shape, and colour were found to have the biggest influence on the applied metrics. The analysis of eye-tracking data revealed that the display of cycling related features can affect the time needed for successfully completing a task. To deepen the gained understanding, further research should in more detail investigate how base maps influence bike maps efficiency.

Linking information from multiple representations: an eye-tracking study

Eye tracking can provide valuable insights into how students use different representations to solve problems and can be a useful tool for measuring the integration of information from multiple representations. In this study, we measured the eye movements of 60 university students while solving two PISA items that contain graphs taken from mathematics and science assessments with the aim of studying the difference in visual attention between students who correctly and incorrectly identify graphs from a verbal description. We were particularly interested in the differences in the integration of information from different representations (text, graphs, and picture) between students who were successful or unsuccessful in solving items. The results suggest that students who solved the items correctly tend to solve the items longer than their counterparts who did not solve the items correctly. Analysis of eye tracking data suggests that students who solved science item correctly analyzed the graph for significantly longer time and had significantly longer average fixation time. This finding suggests that a careful analysis of graphs is crucial for the correct solution of PISA items used in this study. Furthermore, the results showed that students who solved the mathematics item correctly had significantly higher number of transitions between graphs and picture, which indicates a greater integration of information from two different representations. This indicates that these types of items require a lot of time and effort to complete, probably because solving them requires a lot of steps, which is cognitively demanding. We also found that the average fixation durations for different representations may vary for different items, indicating that it is not always equally difficult to extract necessary information from different types of representations. The results of this study suggest that instructors may be able to improve their teaching methods by considering the importance of individual representations (e.g., texts, graphs, and pictures) and the integration of information from multiple sources.

Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data

Eye-tracking techniques are widely used to analyze user behavior. While eye-trackers collect valuable quantitative data, the results are often described in a qualitative manner due to the lack of a model that interprets the gaze trajectories generated by routine tasks, such as reading or comparing two products. The aim of this work is to propose a new quantitative way to analyze gaze trajectories (scanpaths) using machine learning. We conducted a within-subjects study (N = 30) testing six different tasks that simulated specific user behaviors in web sites (attentional, comparing two images, reading in different contexts, and free surfing). We evaluated the scanpath results with three different classifiers (long short-term memory recurrent neural network—LSTM, random forest, and multilayer perceptron neural network—MLP) to discriminate between tasks. The results revealed that it is possible to classify and distinguish between the 6 different web behaviors proposed in this study based on the user’s scanpath. The classifier that achieved the best results was the LSTM, with a 95.7% accuracy. To the best of our knowledge, this is the first study to provide insight about MLP and LSTM classifiers to discriminate between tasks. In the discussion, we propose practical implications of the study results.

Geometric framework for statistical analysis of eye tracking heat maps, with application to a tobacco waterpipe study

Health warning labels have been found to increase awareness of the harmful effects of tobacco products. An eye tracking study was conducted to determine the optimal placement and type of a health warning label on tobacco waterpipes. Participants viewed images that contained one of (1) four waterpipes, (2) three different types of warning labels, (3) placed in three locations. Typically, statistical analysis of eye tracking data is conducted based on summary statistics such as total dwell time, duration score, and number of visits to an area of interest. However, these summary statistics fail to capture the complete variability in a participant's eye movement. Instead, we propose to estimate heat maps defined on the entire image domain using the raw two-dimensional coordinates of eye movement via kernel density estimation. For statistical analysis of heat maps, we adopt the Fisher–Rao Riemannian geometric framework, which enables computationally efficient comparisons of heat maps, statistical summarization and exploration of variability in a sample of heat maps, and metric-based hierarchical clustering. We apply this framework to eye tracking data from the tobacco waterpipe study and comment on the results in the context of the optimal placement and type of health warning labels on tobacco waterpipes.

Classification of autistic and normal children using analysis of eye-tracking data from computer games

Classification of autistic and normal children using analysis of eye-tracking data from computer games

Let's get it started: Eye Tracking in VR with the Pupil Labs Eye Tracking Add-On for the HTC Vive.

Combining eye tracking and virtual reality (VR) is a promising approach to tackle various applied research questions. As this approach is relatively new, routines are not established yet and the first steps can be full of potential pitfalls. The present paper gives a practice example to lower the boundaries for getting started. More specifically, I focus on an affordable add-on technology, the Pupil Labs eye tracking add-on for the HTC Vive. As add-on technology with all relevant source code available on GitHub, a high degree of freedom in preprocessing, visualizing, and analyzing eye tracking data in VR can be achieved. At the same time, some extra preparatory steps for the setup of hardware and software are necessary. Therefore, specifics of eye tracking in VR from unboxing, software integration, and procedures to analyzing the data and maintaining the hardware will be addressed. The Pupil Labs eye tracking add-on for the HTC Vive represents a highly transparent approach to existing alternatives. Characteristics of eye tracking in VR in contrast to other headmounded and remote eye trackers applied in the physical world will be discussed. In conclusion, the paper contributes to the idea of open science in two ways: First, by making the necessary routines transparent and therefore reproducible. Second, by stressing the benefits of using open source software.

Investigating the differences in human behavior between conventional machining and CNC machining for future workforce development: A case study

Due to the high efficiency and consistent product quality, CNC machining has gained a dominant role in the modern machining industry. However, conventional machining still has its significance for certain production settings, e.g. prototyping, and machining workforce preparation. CNC and conventional machining require particular skills and knowledge, which can be unique to specific types of machine tools or overlap to a certain extent. With increasing production volume demands and an aging workforce, a need for efficient quantification, preservation and transfer of said skillsets arises to ensure effective preparation of future professionals and the undisrupted operation of the manufacturing industry. Moreover, the observed shift towards human-centered manufacturing systems in the Industry 4.0 necessitates obtaining an in-depth understanding of human roles in machining. The following paper proposes a novel research approach based on collection and analysis of eye-tracking and video data supplemented by verbal interviews, surveys and self-assessment. A conducted case study spans the entirety of the machining process, from part evaluation, cutting strategy determination, machining operations, to process re-evaluation and optimization. The results show that far greater variability in cutting strategy in terms of operation order, number of operations and used production tooling between consecutive production runs can be observed for conventional machining, with little variation in those terms noted for CNC-based production. Overall, the collected data has allowed to gain an insight into the machinist's decision-making processes and the rationale for observed cutting strategy changes, allowing for potential future application of the proposed research method in improving the machining training and potentially aiding process design by applying the outcomes of studies performed with the use of presented research method to expert systems and future CAM/CAE software solutions.

Car-Following Behavior of Human-Driven Vehicles in Mixed-Flow Traffic: A Driving Simulator Study

Connected and autonomous vehicles (CAVs) and human-driven vehicles (HDVs) will inevitably coexist on roads in the future, creating mixed-flow traffic. The heterogeneous car-following (CF) behavior of HDVs can degrade the control performance of CAVs and introduce inefficiencies in CAV operations. To address these challenges, it is necessary to comprehend HDV CF behavior in mixed-flow traffic. This driving simulator-based study investigates HDV CF behavior in mixed-flow traffic under three different CAV control settings (string-stable, string-unstable, and HDV-like). The effects of traffic congestion level and demographic characteristics on CF behavior are also considered. Statistical analysis and CF model calibration, based on trajectory data collected from 72 participants in driving simulator experiments, are performed to examine the impacts of these factors on string stability, traffic efficiency, and safety. Then, online parameter estimation is conducted to illustrate the time-varying desired time headway, and sensitivity to spacing and speed variations (i.e., CF behavior evolution). Additionally, analysis of post-experiment interview results and eye-tracking data show that the string-stable CAV control setting is preferred by most HDV drivers, but can trigger driver distraction. The results also provide insights for CF behavior prediction and the optimal mixed platoon formation to enhance CAV benefits for traffic flow.

Investigating Financial Statements Recognition Process by Non-professional Users using eye tracking data analysis

Investigating Financial Statements Recognition Process by Non-professional Users using eye tracking data analysis

The Effects of Content Ephemerality on Information Processing

Many marketing communications, from verbal conversations to messaging and content sharing via apps such as Snapchat, limit the number of times people can view content. How do such restrictions affect consumers’ information processing? Building on the proposition that people strategically allocate cognitive resources, the authors hypothesize that consumers of content that cannot be viewed repeatedly consider the risk of failing to process it sufficiently and, consequently, allocate more cognitive resources to its processing (e.g., by increasing viewing time). The authors test this hypothesis in ten preregistered online studies (total N = 17,620), an exploratory analysis of eye-tracking data, and a field study on Facebook's advertising platform. Across the studies, they find that making content ephemeral elevates consumers’ perceived risk of missing information; consequently, it increases attention allocation, prolongs voluntary viewing time, and magnifies focus on relevant information. These effects have important downstream consequences, including improved content comprehension and recall, enhanced positive attitudes, and increased efficiency of sponsored content placement on social media. Taken together, the findings indicate that marketers can communicate information more effectively by restricting consumers from viewing it again.