Eye Tracking Sensors Research Articles

The Effects of Competition on Exercise Intensity and the User Experience of Exercise during Virtual Reality Bicycling for Young Adults

Background: Regular moderate–vigorous intensity exercise is recommended for adults as it can improve longevity and reduce health risks associated with a sedentary lifestyle. However, there are barriers to achieving intense exercise that may be addressed using virtual reality (VR) as a tool to promote exercise intensity and adherence, particularly through visual feedback and competition. The purpose of this work is to compare visual feedback and competition within fully immersive VR to enhance exercise intensity and user experience of exercise for young adults; and to describe and compare visual attention during each of the conditions. Methods: Young adults (21–34 years old) bicycled in three 5 min VR conditions (visual feedback, self-competition, and competition against others). Exercise intensity (cycling cadence and % of maximum heart rate) and visual attention (derived from a wearable eye tracking sensor) were measured continuously. User experience was measured by an intrinsic motivation questionnaire, perceived effort, and participant preference. A repeated-measures ANOVA with paired t-test post hoc tests was conducted to detect differences between conditions. Results: Participants exercised at a higher intensity and had higher intrinsic motivation in the two competitive conditions compared to visual feedback. Further, participants preferred the competitive conditions and only reached a vigorous exercise intensity during self-competition. Visual exploration was higher in visual feedback compared to self-competition. Conclusions: For young adults bicycling in VR, competition promoted higher exercise intensity and motivation compared to visual feedback.

New Paradigm of Identifiable General-response Cognitive Diagnostic Models: Beyond Categorical Data.

Cognitive diagnostic models (CDMs) are a popular family of discrete latent variable models that model students' mastery or deficiency of multiple fine-grained skills. CDMs have been most widely used to model categorical item response data such as binary or polytomous responses. With advances in technology and the emergence of varying test formats in modern educational assessments, new response types, including continuous responses such as response times, and count-valued responses from tests with repetitive tasks or eye-tracking sensors, have also become available. Variants of CDMs have been proposed recently for modeling such responses. However, whether these extended CDMs are identifiable and estimable is entirely unknown. We propose a very general cognitive diagnostic modeling framework for arbitrary types of multivariate responses with minimal assumptions, and establish identifiability in this general setting. Surprisingly, we prove that our general-response CDMs are identifiable under -matrix-based conditions similar to those for traditional categorical-response CDMs. Our conclusions set up a new paradigm of identifiable general-response CDMs. We propose an EM algorithm to efficiently estimate a broad class of exponential family-based general-response CDMs. We conduct simulation studies under various response types. The simulation results not only corroborate our identifiability theory, but also demonstrate the superior empirical performance of our estimation algorithms. We illustrate our methodology by applying it to a TIMSS 2019 response time dataset.

Enjoyment and Appreciation of Political Advertisements: How voters’ Issue Involvement and Congruence with the Sponsor Influence Their Responses and Decisions

This study investigates how voters derive pleasure and meaning from political advertisements by conducting a 3 (sponsorship) × 2 (issue) factorial experiment. It examines the impact of the interaction of issue involvement and congruence between sponsorship and voters’ party affiliation on enjoyment and appreciation of the advertisement. Voters’ real-time emotions (determined by their facial expression) and their attention to the screen (captured via eye-tracking sensor) were measured. Voters’ responses to advertisement were examined in association with their subsequent candidate preference. Advertisements by congruent sponsors are enjoyed and appreciated more than those by incongruent sponsors. Negative and mixed emotions and time spent on the screen are more pronounced among those with higher issue involvement when viewing congruent advertisements. Only enjoyment, appreciation, and expression of positive emotions during advertisement exposure are positively associated with voting decision.

Eye tracking and audio sensors to evaluate surgeon's non-technical skills: An empirical study

Non-Technical Skills (NTS) of medical teams are currently measured using subjective and resource-intensive ratings given by experts. This study explores if objective NTS assessment approaches with eye-tracking and audio sensors can measure teamwork and communication skills in surgery. Eight surgeons participated in a simulated two-phase surgical scenario developed to assess their NTS. Sensor-based audio, eye tracking and video data were collected and analyzed along with rating from the NOTSS scale. Different levels of communication were detected by the sensor data during the two phases of the simulated surgery. Sensor data detected leadership qualities among surgeons based on speech metrics, and eye tracking offered additional evidence about gaze patterns related to NTS. This objective approach to NTS measurement captured differences in communication in greater detail as opposed to a single collective rating obtained using current assessment tools.

Face Recognition Characteristics in Patients with Age-Related Macular Degeneration Determined Using a Virtual Reality Headset with Eye Tracking.

Face recognition is one of the most serious disabilities of patients with age-related macular degeneration (AMD). Our purpose was to study face recognition using a novel method incorporating virtual reality (VR) and eye tracking. Eighteen patients with AMD (seven male; median age 83 years; 89% with bilateral advanced AMD) and nineteen healthy controls (five male; median age 68 years) underwent the face recognition test IC FACES (Synthesius, Ljubljna, Slovenia) on a VR headset with built-in eye tracking sensors. Analysis included recognition accuracy, recognition time and fixation patterns. Additionally, a screening test for dementia and imaging with fundus autofluorescence and optical coherence tomography was performed. AMD patients had significantly lower face recognition accuracy (42% vs. 92%; p < 0.001) and longer recognition time (median 4.0 vs. 2.0 s; p < 0.001) in comparison to controls. Both parameters were significantly worse in patients with lower visual acuity. In both groups, eye-tracking data revealed the two classical characteristics of the face recognition process, i.e., fixations clustering mainly in the nose-eyes-mouth triangle and starting observation in the nasal area. The study demonstrates usability of a VR headset with eye tracking for studying visual perception in real-world situations which could be applicable in the design of clinical studies.

Eye-tracking Analysis: College Website Visual Impact on Emotional Responses Reflected on Subconscious Preferences

This study examined students’ behaviour on the college website and the content of information they were able to obtain. With the eye-tracking sensor, this study aims to investigate the university websites’ effectiveness, satisfaction, and efficiency and collect data regarding users' visual impacts. The research was carried out using mobile phone neuromarketing tools of eye-tracking, facial coding, and supplementary short memory post-survey. The study was focused on two web pages, the homepage, and the CARE page. The analysis results from both web pages were then compared and further discussed. The results suggest that participants mostly elicited sadness (29.55%), neutrality (33.19%), and puzzlement (13.60%) while browsing the homepage, regardless of the areas of interest (AOI). They also elicited slight disgust (4.33%), fear (3.51%), joy (5.21%), and surprise (29.55%). The heat map for the CARE page reveals that the top of the CARE page was a point of attraction for participants. The study found that participants' negative feelings were more intense than good ones concerning homepage scrolling. Also, their pleasant mood intensity increased moderately when they looked at regions with only photos in a subdued color scheme or where brighter colors were used to emphasize essential textual information such as upcoming events and student blogs. This reveals that the website's complexity further affects the cognitive load. Therefore, making it more accessible will be beneficial to students. According to the student's responses, change such as the page's design, color, and text could be implemented.

Being in-sync: A multimodal framework on the emotional and cognitive synchronization of collaborative learners

Collaborative learners share an experience when focusing on a task together and coevally influence each other’s emotions and motivations. Continuous emotional synchronization relates to how learners co-regulate their cognitive resources, especially regarding their joint attention and transactive discourse. “Being in-sync” then refers to multiple emotional and cognitive group states and processes, raising the question: to what extent and when is being in-sync beneficial and when is it not? In this article, we propose a framework of multi-modal learning analytics addressing synchronization of collaborative learners across emotional and cognitive dimensions and different modalities. To exemplify this framework and approach the question of how emotions and cognitions intertwine in collaborative learning, we present contrasting cases of learners in a tabletop environment that have or have not been instructed to coordinate their gaze. Qualitative analysis of multimodal data incorporating eye-tracking and electrodermal sensors shows that gaze instruction facilitated being emotionally, cognitively, and behaviorally “in-sync” during the peer collaboration. Identifying and analyzing moments of shared emotional shifts shows how learners are establishing shared understanding regarding both the learning task as well as the relationship among them when they are emotionally “in-sync.”

Multimodal motivation modelling and computing towards motivationally intelligent E-learning systems

Motivation to engage in learning is essential for learning performance. Learners’ motivation is traditionally assessed using self-reported data, which is time-consuming, subjective, and interruptive to their learning process. To address this issue, this paper proposes a novel framework for multimodal assessment of learners’ motivation in e-learning environments with the ultimate purpose of supporting intelligent e-learning systems to facilitate dynamic, context-aware, and personalized services or interventions, thus sustaining learners’ motivation for learning engagement. We investigated the performance of the machine learning classifier and the most and least accurately predicted motivational factors. We also assessed the contribution of different electroencephalogram (EEG) and eye gaze features to motivation assessment. The applicability of the framework was evaluated in an empirical study in which we combined eye tracking and EEG sensors to produce a multimodal dataset. The dataset was then processed and used to develop a machine learning classifier for motivation assessment by predicting the levels of a range of motivational factors, which represented the multiple dimensions of motivation. We also proposed a novel approach to feature selection combining data-driven and knowledge-driven methods to train the machine learning classifier for motivation assessment, which has been proved effective in our empirical study at selecting predictors from a large number of extracted features from EEG and eye tracking data. Our study has revealed valuable insights for the role played by brain activities and eye movements on predicting the levels of different motivational factors. Initial results using logistic regression classifier have achieved significant predictive power for all the motivational factors studied, with accuracy of between 68.1% and 92.8%. The present work has demonstrated the applicability of the proposed framework for multimodal motivation assessment which will inspire future research towards motivationally intelligent e-learning systems.

A Highly Integrated Ambient Light Robust Eye-Tracking Sensor for Retinal Projection AR Glasses Based on Laser Feedback Interferometry

Robust and highly integrated eye-tracking is a key technology to improve resolution of near-eye-display technologies for augmented reality (AR) glasses such as focus-free retinal projection as it enables display enhancements like foveated rendering. Furthermore, eye-tracking sensors enables novel ways to interact with user interfaces of AR glasses, improving thus the user experience compared to other wearables. In this work, we present a novel approach to track the user's eye by scanned laser feedback interferometry sensing. The main advantages over modern video-oculography (VOG) systems are the seamless integration of the eye-tracking sensor and the excellent robustness to ambient light with significantly lower power consumption. We further present an algorithm to track the bright pupil signal captured by our sensor with a significantly lower computational effort compared to VOG systems. We evaluate a prototype to prove the high robustness against ambient light and achieve a gaze accuracy of 1.62\,$^\circ$, which is comparable to other state-of-the-art scanned laser eye-tracking sensors. The outstanding robustness and high integrability of the proposed sensor will pave the way for everyday eye-tracking in consumer AR glasses.

Quality of experience of 360 video \u2013 subjective and eye-tracking assessment of encoding and freezing distortions

The research domain on the Quality of Experience (QoE) of 2D video streaming has been well established. However, a new video format is emerging and gaining popularity and availability: VR 360-degree video. The processing and transmission of 360-degree videos brings along new challenges such as large bandwidth requirements and the occurrence of different distortions. The viewing experience is also substantially different from 2D video, it offers more interactive freedom on the viewing angle but can also be more demanding and cause cybersickness. The first goal of this article is to complement earlier research by Tran, et al. (2017) [39] testing the effects of quality degradation, freezing, and content on the QoE of 360-videos. The second goal is to test the contribution of visual attention as an influence factor in the QoE assessment. Data was gathered through subjective tests where participants watched degraded versions of 360-videos through a Head-Mounted Display with integrated eye-tracking sensors. After each video they answered questions regarding their quality perception, experience, perceptual load, and cybersickness. Our results showed that the participants rated the overall QoE rather low, and the ratings decreased with added degradations and freezing events. Cyber sickness was found not to be an issue. The effects of the manipulations on visual attention were minimal. Attention was mainly directed by content, but also by surprising elements. The addition of eye-tracking metrics did not further explain individual differences in subjective ratings. Nevertheless, it was found that looking at moving objects increased the negative effect of freezing events and made participants less sensitive to quality distortions. More research is needed to conclude whether visual attention is an influence factor on the QoE in 360-video.

Eye Know You Too: Toward Viable End-to-End Eye Movement Biometrics for User Authentication

Eye movement biometrics (EMB) is a relatively recent behavioral biometric modality that may have the potential to become the primary authentication method in virtual- and augmented-reality (VR/AR) devices due to their emerging use of eye-tracking sensors to enable foveated rendering techniques. However, existing EMB models have yet to demonstrate levels of performance that would be acceptable for real-world use. The present study proposes an improved methodology for EMB with the goal of satisfying the FIDO Biometrics Requirements&#x2019; recommendation of 5% false rejection rate at 1-in-10,000 false acceptance rate. A DenseNet-based convolutional neural network is proposed that is memory-efficient, relatively quick to train, and has only &#x007E;123K learnable parameters. The model is trained over an array of different eye-tracking tasks to improve the generalizability of learned features. Authentication performance is evaluated on a held-out set of up to 59 individuals across different eye-tracking tasks, test-retest intervals, and with increasing amounts of data available for enrollment and authentication. The impact of degraded sampling rates and spatial precision on authentication performance is also briefly explored to set the stage for future research targeting modern VR/AR devices. The proposed technique not only outperforms the previous state of the art but is also the first to approach a level of authentication performance that would be acceptable for real-world use.

Feasibility Study to Identify Brain Activity and Eye-Tracking Features for Assessing Hazard Recognition Using Consumer-Grade Wearables in an Immersive Virtual Environment

Hazard recognition is vital to achieving effective safety management. Unmanaged or unrecognized hazards on construction sites can lead to unexpected accidents. Recent research has identified cognitive failures among workers as being a principal factor associated with poor hazard recognition levels. Therefore, understanding cognitive correlates of when individuals recognize hazards versus when they fail to recognize hazards will be useful in combating poor hazard recognition. Such efforts are now possible with recent advances in electroencephalograph (EEG) and eye-tracking technologies. This paper presents a feasibility study that combines EEG and eye tracking in an immersive virtual environment (IVE) to predict when safety hazards will be successfully recognized during hazard recognition efforts using machine learning techniques. Workers wear a virtual reality (VR) head-mounted device (HMD) that is equipped with an eye-tracking sensor. Together with an EEG sensor, brain activities and eye movements are recorded as the workers navigate a simulated virtual construction site and recognize safety hazards. Through an experiment and a feature extraction and selection process, 13 best features out of 306 features from EEG and eye tracking were selected to train a machine learning model. The results show that EEG and eye tracking together can be leveraged to predict when individuals will recognize safety hazards. The developed IVE can be potentially used to first identify hazard types that are correlated with higher arousal and valence. Also, the developed IVE can be potentially used to evaluate the correlation among arousal, valence, and hazard recognition.

Endogenous Eye Blinking Rate to Support Human–Automation Interaction for E-Learning Multimedia Content Specification

As intelligent systems demand for human–automation interaction increases, the need for learners’ cognitive traits adaptation in adaptive educational hypermedia systems (AEHS) has dramatically increased. AEHS utilize learners’ cognitive processes to attain fair human–automation interaction for their adaptive processes. However, obtaining accurate cognitive trait for the AEHS adaptation process has been a challenge due to the fact that it is difficult to determine what extent such traits can comprehend system functionalities. Hence, this study has explored correlation among learners’ pupil size dilation, learners’ reading time and endogenous blinking rate when using AEHS so as to enable cognitive load estimation in support of AEHS adaptive process. An eye-tracking sensor was used and the study found correlation among learners’ pupil size dilation, reading time and learners’ endogenous blinking rate. Thus, the results show that endogenous blinking rate, pupil size and reading time are not only AEHS reliable parameters for cognitive load measurement but can also support human–automation interaction at large.

The Study of Walking, Walkability and Wellbeing in Immersive Virtual Environments

Recent approaches in the research on walkable environments and wellbeing go beyond correlational analysis to consider the specific characteristics of individuals and their interaction with the immediate environment. Accordingly, a need has been accentuated for new human-centered methods to improve our understanding of the mechanisms underlying environmental effects on walking and consequently on wellbeing. Immersive virtual environments (IVEs) were suggested as a potential method that can advance this type of research as they offer a unique combination between controlled experimental environments that allow drawing causal conclusions and a high level of environmental realism that supports ecological validity. The current study pilot tested a walking simulator with additional sensor technologies, including biosensors, eye tracking and gait sensors. Results found IVEs to facilitate extremely high tempo-spatial-resolution measurement of physical walking parameters (e.g., speed, number of gaits) along with walking experience and wellbeing (e.g., electrodermal activity, heartrate). This level of resolution is useful in linking specific environmental stimuli to the psychophysiological and behavioral reactions, which cannot be obtained in real-world and self-report research designs. A set of guidelines for implementing IVE technology for research is suggested in order to standardize its use and allow new researchers to engage with this emerging field of research.

A Novel Camera-Free Eye Tracking Sensor for Augmented Reality Based on Laser Scanning

Next generation AR glasses require a highly integrated, high-resolution near-eye display technique such as focus-free retinal projection to enhance usability. Combined with low-power eye-tracking, such glasses enable better user experience and performance. We propose a novel and robust low-power eye-tracking sensor for integration into retinal projection systems. In our approach, a MEMS micro mirror scans an IR laser beam over the eye region and the scattered light is received by a photodiode. The advantages of our approach over typical VOG systems are its high integration capability and low-power consumption, which qualify our approach for next generation AR glasses. Here, we present a mathematical framework to estimate the achievable gaze angle resolution of our approach. We further show the viability of the proposed eye-tracking sensor based on a laboratory setup and discuss power consumption and gaze angle resolution compared to typical eye-tracking techniques.

SteadEye-Head-Improving MARG-Sensor Based Head Orientation Measurements Through Eye Tracking Data.

This paper presents the use of eye tracking data in Magnetic AngularRate Gravity (MARG)-sensor based head orientation estimation. The approach presented here can be deployed in any motion measurement that includes MARG and eye tracking sensors (e.g., rehabilitation robotics or medical diagnostics). The challenge in these mostly indoor applications is the presence of magnetic field disturbances at the location of the MARG-sensor. In this work, eye tracking data (visual fixations) are used to enable zero orientation change updates in the MARG-sensor data fusion chain. The approach is based on a MARG-sensor data fusion filter, an online visual fixation detection algorithm as well as a dynamic angular rate threshold estimation for low latency and adaptive head motion noise parameterization. In this work we use an adaptation of Madgwicks gradient descent filter for MARG-sensor data fusion, but the approach could be used with any other data fusion process. The presented approach does not rely on additional stationary or local environmental references and is therefore self-contained. The proposed system is benchmarked against a Qualisys motion capture system, a gold standard in human motion analysis, showing improved heading accuracy for the MARG-sensor data fusion up to a factor of 0.5 while magnetic disturbance is present.

Bioinformatics-Based Adaptive System towards Real-Time Dynamic E-learning Content Personalization

Adaptive Educational Hypermedia Systems (AEHS) play a crucial role in supporting adaptive learning and immensely outperform learner-control based systems. AEHS’ page indexing and hyperspace rely mostly on navigation supports which provide the learners with a user-friendly interactive learning environment. Such AEHS features provide the systems with a unique ability to adapt learners’ preferences. However, obtaining timely and accurate information for their adaptive decision-making process is still a challenge due to the dynamic understanding of individual learner. This causes a spontaneous changing of learners’ learning styles that makes hard for system developers to integrate learning objects with learning styles on real-time basis. Thus, in previous research studies, multiple levels navigation supports have been applied to solve this problem. However, this approach destroys their learning motivation because of imposing time and work overload on learners. To address such a challenge, this study proposes a bioinformatics-based adaptive navigation support that was initiated by the alternation of learners’ motivation states on a real-time basis. EyeTracking sensor and adaptive time-locked Learning Objects (LOs) were used. Hence, learners’ pupil size dilation and reading and reaction time were used for the adaption process and evaluation. The results show that the proposed approach improved the AEHS adaptive process and increased learners’ performance up to 78%.

Evaluation of an Eye Tracking Sensor to Detect Cannabis Impairment

Evaluation of an Eye Tracking Sensor to Detect Cannabis Impairment

Intelligent decision-making of online shopping behavior based on internet of things

The development of big data and Internet of things (IoT) have brought big changes to e-commerce. Different kinds of information sources have improved the consumers’ online shopping performance and make it possible to realize the business intelligence. Grip force and eye-tracking sensors are applied to consumers' online reviews search behavior by relating them to the research approaches in IoT. To begin with, public cognition of human contact degrees of recycled water reuses with grip force test was measured. According to the human contact degrees, 9 recycled water reuses presented by the experiment are classified into 4 categories. Based on the conclusion drawn from grip force test, purified recycled water and fresh vegetable irrigated with recycled water are regarded as the drinking for high-level human contact degree and the irrigation of food crops for low-level human contact degree respectively. Several pictures are designed for eye-tracking test by simulating an on-line shopping web page on Taobao (the most popular online shopping platform in China). By comparing the fixation time participants spent on the areas of interest (AOIs), we justify that consumers' online reviews search behavior is substantially affected by human contact degrees of recycled products. It was found that consumers rely on safety perception reviews when buying high contact goods.

Experimental characterisation of eye-tracking sensors for adaptive human-machine systems

Adaptive Human-Machine Interfaces and Interactions (HMI2) are closed-loop cyber-physical systems comprising a network of sensors measuring human, environmental and mission parameters, in conjunction with suitable software for adapting the HMI2 (command, control and display functions) in response to these real-time measurements. Cognitive HMI2 are a particular subclass of these systems, which support dynamic HMI2 adaptations based on the user’s cognitive state. These states are estimated in real-time using various neuro-physiological parameters from gaze, cardiorespiratory and brain signals, which are processed by an Adaptive Neuro-Fuzzy Inference System (ANFIS). However, the accuracy and precision of neuro-physiological measurements are affected by a variety of environmental factors and therefore need to be accurately characterised prior to operational use. This paper describes the characterisation activities performed on two types of eye tracking devices used in the Aerospace Intelligent and Autonomous Systems (AIAS) laboratory of RMIT University to support the development of cognitive human-machine systems. The uncertainty associated with the ANFIS outputs is quantified by propagating the uncertainties in the input data (determined experimentally) through the inference engine. This process is of growing relevance because similar machine learning techniques are now being developed for an increasing number of applications including aerospace, transport, biomedical and defence cyber-physical systems.