Eye Tracking And Electroencephalography Research Articles

Effects of cleft lip on visual scanning and neural processing of infant faces.

Infant faces readily capture adult attention and elicit enhanced neural processing, likely due to their importance evolutionarily in facilitating bonds with caregivers. Facial malformations have been shown to impact early infant-caregiver interactions negatively. However, it remains unclear how such facial malformations may impact early visual processing. The current study used a combination of eye tracking and electroencephalography (EEG) to investigate adults' early visual processing of infant faces with cleft lip/palate as compared to normal infant faces, as well as the impact cleft palate has on perceived cuteness. The results demonstrated a significant decrease in early visual attention to the eye region for infants with cleft palate, while increased visual attention is registered on the mouth region. Increased neural processing of the cleft palate was evident at the N170 and LPP, suggesting differences in configural processing and affective responses to the faces. Infants with cleft palate were also rated significantly less cute than their healthy counterparts (mean difference = .73, p < .001). These results suggest that infants' faces with cleft lip/palate are processed differently at early visual perception. These processing differences may contribute to several important aspects of development (e.g., joint attention) and may play a vital role in the previously observed difficulties in mother-infant interactions.

Sex differences in eye movements and neural oscillations during mental rotation in virtual reality

Virtual reality (VR) has been a promising tool for developing visuospatial tasks. Among visuospatial tasks, mental rotation tasks are widely used in the assessment of visuospatial ability. Males have a distinct advantage in mental rotation ability compared to females, yet it is generally produced by investigations based on two-dimensional (2D) images on a computer screen. Sex differences in mental rotation tasks with three-dimensional (3D) objects in VR were not fully investigated. It is unclear whether the male's advantages in 2D mental rotation tasks are weakened in 3D tasks. The aim of this study was to provide new insights into the understanding of sex differences in mental rotation tasks presented in VR. Here, we developed a VR mental rotation task (VR-MRT) using 3D objects presented by a head-mounted display (HMD) and used VR-based eye tracking and electroencephalography (EEG) to examine eye movements and neural oscillations for males and females. Our results showed that females preferred a piecemeal strategy compared to males, suggesting a significant sex difference in visual strategy. More importantly, we found no significant sex differences in alpha-band and beta-band oscillations related to rotation processes of VR-MRT. These findings indicated that sex differences in the VR-MRT were mainly attributed to the selection of visual strategy rather than the rotation processes. The study helps to comprehensively understand the dominant factors contributing to the sex differences in the VR-MRT.

Analyzing motivating functions of consumer behavior: Evidence from attention and neural responses to choices and consumption.

Academia and business have shown an increased interest in using neurophysiological methods, such as eye-tracking and electroencephalography (EEG), to assess consumer motivation. The current research contributes to this literature by verifying whether these methods can predict the effects of antecedent events as motivating functions of attention, neural responses, choice, and consumption. Antecedent motivational factors are discussed, with a specific focus on deprivation as such a situational factor. Thirty-two participants were randomly assigned to the experimental and control conditions. Water deprivation of 11-12 h was used as an establishing operation to increase the reinforcing effectiveness of water. We designed three experimental sessions to capture the complexity of the relationship between antecedents and consumer behavior. Experimental manipulations in session 1 established the effectiveness of water for the experimental group and abolished it for the control group. Results from session 2 show that participants in the experimental group had significantly higher average fixation duration for the image of water. Their frontal asymmetry did not provide significant evidence of greater left frontal activation toward the water image. Session 3 demonstrated that choice and consumption behavior of the relevant reinforcer was significantly higher for participants in the experimental group. These early findings highlight the potential application of a multi-method approach using neurophysiological tools in consumer research, which provides a comprehensive picture of the functional relationship between motivating events, behavior (attention, neural responses, choice, and consumption), and consequences.

A Trend on Autism Spectrum Disorder Research: Eye Tracking-EEG Correlative Analytics

The greatest challenge in autism spectrum disorder (ASD) detection and treatment arises from the elusive etiological origins and heterogeneity of ASD. ET-EEG correlative analytics refers to correlating simultaneously recorded eye tracking (ET) and electroencephalography (EEG) data or combining them to yield diagnostic biomarkers. This approach was recently applied in ASD research as EEG and video-based ET are suitable for children and do not interfere with each other. It allows researchers to associate neural correlates with gaze patterns in the same cognitive task. Besides, correlative analytics has shed light on the inconsistent findings derived from the unimodal approach, reveals the developmental trajectory of ASD, and provides biomarkers to assist ASD diagnosis or intervention. This review focuses on eight articles applying ET-EEG correlative analytics and synthesized the reported correlation between ET and EEG patterns. Studies concerning such patterns that acquired either ET or EEG data were also reviewed to provide comparisons. In recent years, correlative analytics has also been developed in other research fields, such as visual information processing and EEG artifact correction. Incorporating methodologies used in these research works into ASD studies would help identify cognitive alteration related to a specific visual pattern in naturalistic settings. The opportunities and limitations of this approach in ASD research are discussed at last.

Exploration of the Virtual Reality Teleportation Methods Using Hand-Tracking, Eye-Tracking, and EEG

Although recent developments in VR tracking technology have enabled VR to become more portable and convenient, VR must overcome limited traversable space to be used for more general purposes. Since future VR HMDs aim to embed bio-sensors, such as eye-tracking and electroencephalography (EEG), the present study investigates how bio-sensors could be integrated into VR locomotion and how usability could be improved. In this paper, teleportation methods were developed by combining eye-tracking, EEG, and hand-tracking for location targeting and teleport triggering processes. In a static teleportation task, we compared the efficiency, accuracy, and usability of each method as well as the interactions between different combinations of methods. The experimental results verified that these locomotion methods were overall suitable for hands-free VR contents, and revealed relative superiority of eye-tracking for location targeting and hand-tracking for teleport triggering. Importantly, we identified specific combinations of locomotion methods appropriate for context by showing the interactions between experimental factors, and components that require improvement for future application. The results of this study contribute to development and understanding of novel VR locomotion methods utilizing bio-signal data, for more efficient and natural VR locomotion experience.

Real-time processes in the development of action planning

Across species and ages, planning multi-step actions is a hallmark of intelligence and critical for survival. Traditionally, researchers adopt a "top-down" approach to action planning by focusing on the ability to create an internal representation of the world that guides the next step in a multi-step action. However, a top-down approach does not inform on underlying mechanisms, so researchers can only speculate about how and why improvements in planning occur. The current study takes a "bottom-up" approach by testing developmental changes in the real-time, moment-to-moment interplay among perceptual, neural, and motor components of action planning using simultaneous video, motion-tracking, head-mounted eye tracking, and electroencephalography (EEG). Preschoolers (n= 32) and adults (n= 22) grasped a hammer with their dominant hand to pound a peg when the hammer handle pointed in different directions. When the handle pointed toward their non-dominant hand, younger children ("nonadaptive planners") used a habitual overhand grip that interfered with wielding the hammer, whereas adults and older children ("adaptive planners") used an adaptive underhand grip. Adaptive and nonadaptive children differed in when and where they directed their gaze to obtain visual information, neural activation of the motor system before reaching, and straightness oftheir reach trajectories. Nonadaptive children immediately used a habitual overhand grip before gathering visual information, leaving insufficient time to form a plan before acting. Our novel bottom-up approach transcends mere speculation by providing converging evidence that the development of action planning depends on a real-time "tug of war" between habits and information gathering and processing.

Impacts of Color Coding on Programming Learning in Multimedia Learning: Moving Toward a Multimodal Methodology.

In the present study, we tested the effectiveness of color coding on the programming learning of students who were learning from video lectures. Effectiveness was measured using multimodal physiological measures, combining eye tracking and electroencephalography (EEG). Using a between-subjects design, 42 university students were randomly assigned to two video lecture conditions (color-coded vs. grayscale). The participants’ eye tracking and EEG signals were recorded while watching the assigned video, and their learning performance was subsequently assessed. The results showed that the color-coded design was more beneficial than the grayscale design, as indicated by smaller pupil diameter, shorter fixation duration, higher EEG theta and alpha band power, lower EEG cognitive load, and better learning performance. The present findings have practical implications for designing slide-based programming learning video lectures; slides should highlight the format of the program code using color coding.

신경과학적 방법론을 적용한 교과교육 연구의 동향 탐색 - 문법교육적 시사점을 중심으로 -

This study reviewed curriculum education studies that applied neuroscientific methods to conduct preliminary work introducing neuroscience into grammar education. As a result of the scoping review, 161 studies were selected and reviewed in terms of their research methods and topics. Since the 2000s, the number of studies applying neuroscientific methods has been increasing quantitatively, mainly using eye tracking (ET) and electroencephalography (EEG), while there are few studies that have used event-related potentials (ERPs), functional magnetic resonance imaging (fMRI), and magnetoencephalography (MEG). Given that ET is mainly used in the field of Korean language education, more diverse methods need to be attempted in the future. A total of two main research topics are discovered: “understanding phenomena,” which focus on explaining certain pedagogical situations and “development and verification,” which focus on verifying or developing specific educational tools. As for the former, studies related to “understanding learning activities” were mainly conducted, and regarding the latter, studies related to “verification of treatment effects” were frequently found. Finally, by integrating the research topics and methods, possible research questions applying neuroscience were explored. The activation of grammatical thinking can be explored by fMRI, and the language processing patterns of learners and speakers can be captured using ERP. It will be possible to obtain clues on the characteristics of "grammatically gifted" using EEG, and empirical evidence on the reading process of grammar materials can be captured by ETs. This study is meaningful in that it analyzed the trends of curriculum education research applying neuroscientific methods, and it showed the advantages of applying neuroscience by suggesting research problems that can be performed in the field of grammar education.

Memory Deficit in Patients With Temporal Lobe Epilepsy: Evidence From Eye Tracking Technology.

Objective: To explore quantitative measurements of the visual attention and neuroelectrophysiological relevance of memory deficits in temporal lobe epilepsy (TLE) by eye tracking and electroencephalography (EEG).Methods: Thirty-four TLE patients and twenty-eight healthy controls were invited to complete neurobehavioral assessments, cognitive oculomotor tasks, and 24-h video EEG (VEEG) recordings using an automated computer-based memory assessment platform with an eye tracker. Visit counts, visit time, and time of first fixation on areas of interest (AOIs) were recorded and analyzed in combination with interictal epileptic discharge (IED) characteristics from the bilateral temporal lobes.Results: The TLE patients had significantly worse Wechsler Digit Span scores [F(1, 58) = 7.49, p = 0.008]. In the Short-Term Memory Game with eye tracking, TLE patients took a longer time to find the memorized items [F(1, 57) = 17.30, p < 0.001]. They had longer first fixation [F(1, 57) = 4.06, p = 0.049] and more visit counts [F(1, 57) = 7.58, p = 0.008] on the target during the recall. Furthermore, the performance of the patients in the Digit Span task was negatively correlated with the total number of IEDs [r(28) = −0.463, p = 0.013] and the number of spikes per sleep cycle [r(28) = −0.420, p = 0.026].Conclusion: Eye tracking appears to be a quantitative, objective measure of memory evaluation, demonstrating memory retrieval deficits but preserved visual attention in TLE patients. Nocturnal temporal lobe IEDs are closely associated with memory performance, which might be the electrophysiological mechanism for memory impairment in TLE.

Using Biometric Tools to Investigate How Students Cognitively Process the Structure of Prostaglandin H 2 Synthase

Protein representations such as those found in the Protein Data Bank repository are important for any practicing biochemist to be able to use, interpret, and manipulate; skills commonly referred to as visual literacy in biochemistry. As such, any student enrolled in a biochemistry course should be trained in these skills. However, little is known as to how students navigate these 3-dimensional representations and how to best teach these skills to biochemistry students, especially using virtual 3D modeling software. An additional barrier to learning these skills are that they are associated with the students’ spatial ability. The proposed study uses biometric tools such as eye tracking and electroencephalography (EEG) to investigate how biochemistry students cognitively process and use modeling software UCSF Chimera to explore the structure of Prostaglandin H2 Synthase. Nine biochemistry students were sampled at a metropolitan Midwestern campus in fall 2018 while completing the activity. During the activity, the students’ cognitive processing was captured using EEG while gaze patterns and areas of fixation were captured by Tobii Glasses 2. Additionally, each student completed a series of spatial ability assessments (Hidden Figures Test and Purdue Visualizations of Rotations Test) to capture their level of ability to think visually. This presentation will answer the following research questions: 1) Which combination of electrode channels suggests the most appropriate measurement of cognitive load? 2) How does the cognitive load measurement change when students analyze a quaternary structure versus a tertiary structure of the same protein? and 3) How can we optimize the cognitive load of a 3D virtual modeling activity? The answers to these questions will be used to improve the activity for replication in following years in hopes of decreasing the impact of extraneous factors on cognition and providing the mental space needed for learning the skills associated with visual literacy in biochemistry.

Application of Neuroscience in the Area of Sustainability: Mapping the Territory

Due to the relevance of sustainability, several methods of data collection are being used to verify its impact on different organizational and social aspects. Despite its importance, in the area of Sustainability, no literature review research has been found that described as some tools derived from neuroscience, more specifically eye tracking and electroencephalography (EEG), which are among the most used tools. Thus, this paper aims to present, through the exploratory study, research that applied eye tracking and EEG tools to evaluate aspects of sustainability in consumption. This article can be used as a reference basis for studies regarding this theme, thus contributing to research advances in this area. Marketing professionals can also use it to learn about the brain and visual mechanisms that are related to sustainable consumption and sustainable decision-making, enabling them to develop their products and communication more effectively. Additionally, future research in the area is suggested.

Definition: Implicit memory

Developmental accounts of autism spectrum disorder (ASD) state that infants and children with ASD are spontaneously less attracted by and less proficient in processing social stimuli such as faces. This is hypothesized to partly underlie social communication difficulties in ASD. While in some studies a reduced preference for social stimuli has been shown in individuals with ASD, effect sizes are moderate and vary across studies, stimuli, and designs. Eye tracking, often the methodology of choice to study social preference, conveys information about overt orienting processes but conceals covert attention, possibly resulting in an underestimation of the effects. In this study, we recorded eye tracking and electroencephalography (EEG) during fast periodic visual stimulation to address this issue. We tested 21 boys with ASD (8-12 years old) and 21 typically developing (TD) control boys, matched for age and IQ. Streams of variable images of faces were presented at 6 Hz alongside images of houses presented at 7.5 Hz or vice versa, while children were engaged in an orthogonal task. While frequency-tagged neural responses were larger in response to faces than simultaneously presented houses in both groups, this effect was much larger in TD boys than in boys with ASD. This group difference in saliency of social versus non-social processing is significant after 5 sec of stimulus presentation and holds throughout the entire trial. Although there was no interaction between group and stimulus category for simultaneously recorded eye-tracking data, eye tracking and EEG measures were strongly correlated. We conclude that frequency-tagging EEG, allowing monitoring of both overt and covert processes, provides a fast, objective and reliable measure of decreased preference for social information in ASD.

Combined frequency-tagging EEG and eye tracking reveal reduced social bias in boys with autism spectrum disorder

Developmental accounts of autism spectrum disorder (ASD) state that infants and children with ASD are spontaneously less attracted by and less proficient in processing social stimuli such as faces. This is hypothesized to partly underlie social communication difficulties in ASD. While in some studies a reduced preference for social stimuli has been shown in individuals with ASD, effect sizes are moderate and vary across studies, stimuli, and designs. Eye tracking, often the methodology of choice to study social preference, conveys information about overt orienting processes but conceals covert attention, possibly resulting in an underestimation of the effects. In this study, we recorded eye tracking and electroencephalography (EEG) during fast periodic visual stimulation to address this issue. We tested 21 boys with ASD (8-12 years old) and 21 typically developing (TD) control boys, matched for age and IQ. Streams of variable images of faces were presented at 6 Hz alongside images of houses presented at 7.5 Hz or vice versa, while children were engaged in an orthogonal task. While frequency-tagged neural responses were larger in response to faces than simultaneously presented houses in both groups, this effect was much larger in TD boys than in boys with ASD. This group difference in saliency of social versus non-social processing is significant after 5 sec of stimulus presentation and holds throughout the entire trial. Although there was no interaction between group and stimulus category for simultaneously recorded eye-tracking data, eye tracking and EEG measures were strongly correlated. We conclude that frequency-tagging EEG, allowing monitoring of both overt and covert processes, provides a fast, objective and reliable measure of decreased preference for social information in ASD.

A Consumer Neuroscience Study of Conscious and Subconscious Destination Preference

In studying consumer behaviors, the inclusion of neuroscience tools and methods is improving our understanding of preference formation and choice. But such responses are mostly related to the consumption of goods and services that meet an immediate need. Tourism represents a consumer behavior that is related to a more complex decision-making process, involving a stronger relationship with a future self, and choices typically being of a higher level of involvement and of a transformational type. The aim of this study was to test whether direct emotional and cognitive responses to travel destination would be indicative of subsequent stated destination preference. Participants were shown images and videos from multiple travel destinations while being monitored using eye-tracking and electroencephalography (EEG) brain monitoring. The EEG responses to each image and video were further calculated into neurometric scores of emotional (frontal asymmetry and arousal) and cognitive load metrics. Our results show that arousal and cognitive load were significantly related to subsequent stated travel preferences, accounting for about 20% of the variation in preference. Still, results also suggested that subconscious emotional and cognitive responses are not identical to subjective travel preference, suggesting that other mechanisms may be at play in forming conscious, stated preference. This study both supports the idea that destination preferences can be studied using consumer neuroscience and brings further insights into the mechanisms at stake during such choices.

Neural Mechanisms of Overt Attention Shifts to Emotional Faces

Emotional faces draw attention and eye-movements towards them. However, the neural mechanisms of attention have mainly been investigated during fixation, which is uncommon in everyday life where people move their eyes to shift attention to faces. Therefore, the current study combined eye-tracking and Electroencephalography (EEG) to measure neural mechanisms of overt attention shifts to faces with happy, neutral and angry expressions, allowing participants to move their eyes freely towards the stimuli. Saccade latencies towards peripheral faces did not differ depending on expression and early neural response (P1) amplitudes and latencies were unaffected. However, the later occurring Early Posterior Negativity (EPN) was significantly larger for emotional than for neutral faces. This response appears after saccades towards the faces. Therefore, emotion modulations only occurred after an overt shift of gaze towards the stimulus had already been completed. Visual saliency rather than emotional content may therefore drive early saccades, while later top-down processes reflect emotion processing.

Joint analysis of simultaneous EEG and eye tracking data for video images

PurposeTechnological innovation has made it possible to review how a film cues particular reactions on the part of the viewers. The purpose of this paper is to capture and interpret visual perception and attention by the simultaneous use of eye tracking and electroencephalography (EEG) technologies.Design/methodology/approachThe authors have developed a method for joint analysis of EEG and eye tracking. To achieve this goal, an algorithm was implemented to capture and interpret visual perception and attention by the simultaneous use of eye tracking and EEG technologies. All parameters have been measured as a function of the relationship between the tested signals, which, in turn, allowed for a more accurate validation of hypotheses by appropriately selected calculations.FindingsThe results of this study revealed a coherence between EEG and eye tracking that are of particular relevance for human perception.Practical implicationsThis paper endeavors both to capture and interpret visual perception and attention by the simultaneous use of eye tracking and EEG technologies. Eye tracking provides a powerful real-time measure of viewer region of interest. EEG technologies provides data regarding the viewer’s emotional states while watching the movie.Originality/valueThe approach in this paper is distinct from similar studies because it takes into account the integration of the eye tracking and EEG technologies. This paper provides a method for determining a fully functional video introspection system.

Processamento de interrogativas-QU em Português Brasileiro: evidências de eyetracking e EEG

Este artigo trata do processamento de sentenças com interrogativas-QU de sujeito e de objeto em português brasileiro1 (PB). Especificamente, investigou-se a Assimetria Sujeito-Objeto nesse tipo de sentença, partindo-se de resultados anteriores com a técnica de leitura automonitorada (OLIVEIRA; MAIA, 2010; OLIVEIRA, 2013). Foram utilizadas as técnicas de rastreamento ocular (eyetracking) e eletroencefalografia (EEG) como ferramentas de diagnóstico do processamento linguístico em tempo real. O objetivo era buscar evidências sobre o modus operandi do parser durante o processamento das funções de sujeito e de objeto em sentenças interrogativas-QU no PB e verificar se há discrepâncias entre as técnicas utilizadas. Os resultados revelam diferenças interessantes, permitindo discernir os níveis de análise representacional, algorítmico e implementacional, na direção da clássica proposta de Marr (1982).

Explicit and Implicit Factors That Determine Private Labels’ Possible Purchase: Eyetracking and EEG Research

Abstract Objective: This paper investigates the explicit and implicit factors affecting private-label (PL) products’ possible purchase decision for different retailers. Design: The study uses eyetracking and electroencephalography (EEG) to explore the differences in eye movement and brain activity for PL products. This article examines how approach motivation, measured by total fixation duration and by EEG asymmetry over the frontal hemisphere of the brain, predicts PL purchase decision. Findings: This study investigates implicit variables that can influence consumers’ willingness to PL purchase. The relatively greater left frontal activation (i.e., higher approach motivation) during the predecision period predicted an affirmative purchase decision in some cases. The eyetracking study did not reveal differences between women’s and men’s esthetics sensitivity toward the presented PL products. EEG research proved that consumers were not influenced by the PL product price. Originality/value: Literature lacks credible information on young buyers’ behavior in the context of PL products. This paper elaborates on PL perception, revealing the neural origins of the associated psychological processes.

Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation

Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation

Cognitive Alignment with Performance Targeted Training Intervention Model

We propose that the use of two simple behavioral measures, in conjunction with neurophysiological measures, can be used to create a training intervention that has the potential to provide: (1) real-time notification as to when a training intervention is needed and (2) real-time information as to the type of training intervention that should be employed. The Cognitive Alignment with Performance Targeted Training Intervention Model (CAPTTIM) determines if a trainee’s cognitive state is aligned or misaligned with actual performance. When misalignment occurs, it indicates that a training intervention is needed. Neurophysiological markers, as captured by eyetracking and electroencephalography (EEG), can assist in determining why misalignment between cognitive state and performance occurred, leading to more effective and targeted training intervention. Because all measures are captured continuously in real time, this model has the potential to increase training efficiency and effectiveness in a variety of training domains. The model is illustrated with two case studies.