Eye Gaze Movement Research Articles

Evaluation of Optical Tracking to Distinguish Surgeon Experience During Endoscopic Stone Surgery.

Introduction and Objectives: Optical tracking (OT) has shown potential in assessing surgical skill but has yet to be evaluated for endoscopic urologic surgery. We sought to evaluate the potential for OT to distinguish expert and trainee surgeons during flexible ureteroscopy (fURS) for kidney stone treatment in both simulated and live surgical settings. Methods: We performed OT analysis of six surgeons performing stone localization during fURS in two settings. In the first setting, surgeons were tasked with fiducial localization in three separate kidney phantoms during fURS. In the second setting, surgeons performed stone localization via fURS in five separate patients. Surgeons were categorized as "expert" (n = 3, endourologist, average case volume of >100 fURS per year) or trainee (n = 3, trainee, <100 fURS per year). OT metrics were recorded for both settings using the Microsoft HoloLens 2© as the surgeons viewed the surgical monitor during fURS. Standard OT metrics of experts and trainees were compared, and included: area of eye gaze movement, gaze distance traveled, number of gaze fixation points, percentage of gaze fixation dwell time, and number of saccades. Results: In the simulated setting, the average time for stone localization was greater for trainees compared to experts (318 seconds vs 52 seconds, p < 0.01). Additionally, the mean area of eye gaze movements with area of interest (AOI) was greater for trainees compared to experts (1430 cm2 vs 1060 cm2, p < 0.01). The total gaze distance traveled was also greater for trainees compared to experts (1480 cm vs 730 cm, p < 0.01). In the live surgical setting, average time for stone localization was similar for trainees and experts (74 seconds vs 51 seconds, p = 0.05). The total area of eye gaze movements in AOI was greater for trainees compared to experts (700 cm2 vs 30 cm2, p < 0.01). Additionally, the total gaze distance traveled was greater for trainees compared to experts (14,000 cm vs 680 cm, p < 0.01). This suggests more varied and less point specific concentration on the surgical screen by trainees. There was no difference in percentage of gaze fixation dwell time and number of saccades between experts and trainees in either setting. Conclusions: OT analysis can objectively distinguish surgical experience between experts and trainee surgeons performing fURS in both simulated and live surgical settings. These findings may play a role in future surgical training and skills assessment.

An action-observation method for studying social perception: a mini-review.

An important aspect of any social interaction involves inferring other people's mental states, intentions, and their likely next actions, by way of facial expression, body posture, eye gaze, and limb movements. An actor's production of actions during social interactions and the observer's perception of these actions are thus closely linked. In this review, we outline an action-observation methodology, which not only allows for separate analyses of production and perception, but also promotes the study of the dynamic interaction between these two sides of every social exchange. We review two lines of research that have benefited from its application. The first line focuses on individuals performing tasks alone and the observation of their actions by other individuals in order to make inferences about their attentional states. The second line of study focused on pairs of individuals performing collaborative tasks in naturalistic settings and the observation of these performances by other individuals. We offer several suggestions for how this methodology can be extended to improve on the limitations of the present studies, as well as some suggestions of how to use this methodology to venture into new territory. Our aim is to inspire future research applications of this methodology in order to advance our understanding of social action production and perception.

Physiological and performance metrics during a cardiopulmonary real-time feedback simulation to estimate cognitive load

Multitasking is crucial for First Responders (FRs) in emergency scenarios, enabling them to prioritize and treat victims efficiently. However, research on multitasking and its impact on rescue operations are limited. This study explores the relationship between multitasking, working memory, and the performance of chest compressions during cardiopulmonary resuscitation (CPR). In this experiment, eighteen first-year residents participated in a CPR maneuver using a real-time feedback simulator to learn chest compressions. Different additional secondary tasks were developed and accomplished concurrently with the chest compressions. Heart rate, respiration rate, galvanic skin response, body temperature, eye gaze movements and chest compression performance data were collected. The findings of this study indicated that multitasking impacted chest compression quality for all secondary tasks, showing significance (p-value < 0.05) for the frequency of the chest compressions which worsened in all cases. Additionally, vital signs such as heart rate, respiration rate, and eye gaze speed were also affected during multitasking. Nevertheless, this change on vital signs was different depending on the type of secondary task accomplished. Therefore, as a conclusion, performing multiple tasks during chest compressions affects performance. Understanding cognitive load and its impact on vital signs can aid in training FRs to handle complex scenarios efficiently.

The Gaze Dialogue Model: Nonverbal Communication in HHI and HRI.

When humans interact with each other, eye gaze movements have to support motor control as well as communication. On the one hand, we need to fixate the task goal to retrieve visual information required for safe and precise action-execution. On the other hand, gaze movements fulfil the purpose of communication, both for reading the intention of our interaction partners, as well as to signal our action intentions to others. We study this Gaze Dialogue between two participants working on a collaborative task involving two types of actions: 1) individual action and 2) action-in-interaction. We recorded the eye-gaze data of both participants during the interaction sessions in order to build a computational model, the Gaze Dialogue, encoding the interplay of the eye movements during the dyadic interaction. The model also captures the correlation between the different gaze fixation points and the nature of the action. This knowledge is used to infer the type of action performed by an individual. We validated the model against the recorded eye-gaze behavior of one subject, taking the eye-gaze behavior of the other subject as the input. Finally, we used the model to design a humanoid robot controller that provides interpersonal gaze coordination in human-robot interaction scenarios. During the interaction, the robot is able to: 1) adequately infer the human action from gaze cues; 2) adjust its gaze fixation according to the human eye-gaze behavior; and 3) signal nonverbal cues that correlate with the robot's own action intentions.

Embodied-Visual Practices during Conversational Repair: Scoping Review

ABSTRACT Repair organization provides a powerful mechanism for handling problems of mutual understanding in natural conversation. Our study reviews past research on repair initiation and resolution practices involving embodied-visual practices, including eye gaze, facial expressions, hand gestures, and head and body movements. We charted details of theoretical background, methodology, and main findings for 31 studies. Embodiment was studied equally in connection with self- (SIR) and other-initiated repair (OIR); gaze and gestures were the most commonly studied embodied-visual practices. In OIR sequences, research has focused on upper body and gaze. In SIR sequences, more gestures were examined, but this may be specific to word search, which was predominant in SIR studies. In addition, embodied-visual practices can form gestalts independently from talk, and can occur early in the repair sequence, even pre-trouble source. Data were in Argentine Sign Language, Cha’palaa, Danish, English, Finnish, French, German, Italian, Japanese, Korean, Mandarin, Turkish, and Yélî Dnye.

Leveraging Computer Vision to Establish a Correlation Between Eye-Gaze Estimation and Saccades

The purpose of this project was to develop a model that can detect pupils on a subject’s face and draw the graph of both pupils to determine if a subject has saccades. Current methods cannot accurately detect contours when the face is presented at an angle. To resolve this issue, subjects recorded a video while performing the Head Impulse Test (HIT). Next, a computer vision library, Opencv, extracted frames from the recorded video and detected the facial key point for the nose. The Multi-task Cascaded Convolutional Neural Network (MTCNN) extracted the face from the frame and generated contours for the eyes using a segmentation library. The largest contours on the mask were divided into two parts and their extreme bounding box points were identified using dilation, erosion, and blur. Our model rendered a video of both contours applied to the patient performing the HIT test. Also, the model generated two graphs, comparing each eye’s gaze with the pose estimation. There were two main outputs: the graph may resemble a y = -x line if a patient does not have saccades or the graph may be more distorted if the patient has saccades due to a sudden shift in eye-gaze movement. This study demonstrates a precise method of eye-gaze estimation and the detection of saccades. Further experiments will help to validate the notion that saccades are connected to neurological disorders. Future experiments may include more data during the training process and to quantitatively determine the accuracy of our model.

Exploring the relationship of working memory to the temporal distribution of pausing and revision behaviors during L2 writing

AbstractThis study examined the extent to which L2 writers with varied working memory display differential pausing and revision behaviors at different periods during writing. The participants were 30 advanced Chinese L2 users of English, who wrote an argumentative essay. While composing, participants’ keystrokes and eye-gaze movements were recorded to capture their pausing, revision, and eye-gaze behaviors. The working memory battery included tests of phonological and visual short-term memory and executive functions. We divided the writing process into five equal periods. The results revealed that participants’ pausing and revision patterns were consistent with previous findings that planning, linguistic encoding, and monitoring processes dominate the initial, middle, and later composing periods, respectively. Various working memory components had differential effects on pausing depending on period, largely reflecting the predictions of Kellogg’s (1996, 2001) model. However, we identified no differences in the temporal distribution of revision behaviors contingent on working memory.

An exploratory study on eye-gaze patterns of experts and novices of science inference graph items

&lt;abstract&gt;&lt;p&gt;Graphs are highly prevalent as a form of quantitative data in various science, technology, engineering and mathematics fields. Thus, graphical literacy is especially important in understanding today's world and being scientifically literate. However, students often face difficulties in graph interpretation and differ substantially in their graphical literacy. While many teachers are aware of students' difficulties in answering graph items, there is limited knowledge about how students go about attempting graph items. In this exploratory study, we investigated the eye-gaze patterns of experts and novices in graph interpretation of five science inference-based multiple-choice items requiring no prior content knowledge to problem-solve. Experts refer to science university faculty members who are currently teaching science content courses to undergraduate students. Novices refer to university undergraduates majoring in one of the science subjects. Participants' eye-gaze movements were recorded using the Dikablis eye-tracker, and their eye-gaze patterns and total glance time (s) were subsequently analyzed using the software D-Lab 3.0. Experts focused more on the question stem, whereas novices focused more on the graph. Additionally, experts tend to focus on contextual and graph data features initially, before moving to cues such as options. Conversely, novices demonstrated more sporadic search patterns. The findings contribute to the literature that compares how experts and novices' problem-solve graph items that are inference-based. An interesting future study on the eye gaze patterns and accuracy of answers is suggested from a finding. This study also provides a set of heuristics to be adopted in the teaching and learning of graph interpretation. The findings of this study have implications for teachers in the way they scaffold students' approach to answering graphical items. Additionally, students can employ heuristics to answer graphical items more effectively.&lt;/p&gt;&lt;/abstract&gt;

Gaze and Environmental Context-Guided Deep Neural Network and Sequential Decision Fusion for Grasp Intention Recognition.

Grasp intention recognition plays a crucial role in controlling assistive robots to aid older people and individuals with limited mobility in restoring arm and hand function. Among the various modalities used for intention recognition, the eye-gaze movement has emerged as a promising approach due to its simplicity, intuitiveness, and effectiveness. Existing gaze-based approaches insufficiently integrate gaze data with environmental context and underuse temporal information, leading to inadequate intention recognition performance. The objective of this study is to eliminate the proposed deficiency and establish a gaze-based framework for object detection and its associated intention recognition. A novel gaze-based grasp intention recognition and sequential decision fusion framework (GIRSDF) is proposed. The GIRSDF comprises three main components: gaze attention map generation, the Gaze-YOLO grasp intention recognition model, and sequential decision fusion models (HMM, LSTM, and GRU). To evaluate the performance of GIRSDF, a dataset named Invisible containing data from healthy individuals and hemiplegic patients is established. GIRSDF is validated by trial-based and subject-based experiments on Invisible and outperforms the previous gaze-based grasp intention recognition methods. In terms of running efficiency, the proposed framework can run at a frequency of about 22 Hz, which ensures real-time grasp intention recognition. This study is expected to inspire additional gaze-related grasp intention recognition works.

Estimation of Confidence in the Dialogue based on Eye Gaze and Head Movement Information

In human-robot interaction, human mental states in dialogue have attracted attention to human-friendly robots that support educational use. Although estimating mental states using speech and visual information has been conducted, it is still challenging to estimate mental states more precisely in the educational scene. In this paper, we proposed a method to estimate human mental state based on participants’ eye gaze and head movement information. Estimated participants’ confidence levels in their answers to the miscellaneous knowledge question as a human mental state. The participants’ non-verbal information, such as eye gaze and head movements during dialog with a robot, were collected in our experiment using an eye-tracking device. Then we collect participants’ confidence levels and analyze the relationship between human mental state and non-verbal information. Furthermore, we also applied a machine learning technique to estimate participants’ confidence levels from extracted features of gaze and head movement information. As a result, the performance of a machine learning technique using gaze and head movements information achieved over 80 % accuracy in estimating confidence levels. Our research provides insight into developing a human-friendly robot considering human mental states in the dialogue.

EyeT4Empathy: Dataset of foraging for visual information, gaze typing and empathy assessment

We present a dataset of eye-movement recordings collected from 60 participants, along with their empathy levels, towards people with movement impairments. During each round of gaze recording, participants were divided into two groups, each one completing one task. One group performed a task of free exploration of structureless images, and a second group performed a task consisting of gaze typing, i.e. writing sentences using eye-gaze movements on a card board. The eye-tracking data recorded from both tasks is stored in two datasets, which, besides gaze position, also include pupil diameter measurements. The empathy levels of participants towards non-verbal movement-impaired people were assessed twice through a questionnaire, before and after each task. The questionnaire is composed of forty questions, extending a established questionnaire of cognitive and affective empathy. Finally, our dataset presents an opportunity for analysing and evaluating, among other, the statistical features of eye-gaze trajectories in free-viewing as well as how empathy is reflected in eye features.

Investigating age differences in the influence of joint attention on working memory.

Previous research has demonstrated that older adults make limited use of social cues as compared to younger adults. This has been investigated by testing the influence of gaze cues on attentional processes, with findings showing significantly smaller gaze-cuing effects for older than younger adults. Here, we aimed to investigate whether this would also result in age-related differences in the influence of gaze cues on working memory (WM). We therefore tested the effects of gaze cues from realistic human avatars on working memory across two experiments using dynamic head turns and more subtle eye gaze movements. We compared working memory for items looked at by the cue (congruent), looked away from by the cue (incongruent), and items shown when the cue looked down (neutral). Results demonstrated that for both older and younger adults, gaze cues influenced working memory processes, though there were some important differences related to the nature of the cue. When the cue made a dynamic head turn both younger and older adults showed an equivalent effect of gaze on attention. However, when only the eyes moved, while both the younger and older adults showed an effect of gaze on working memory, there appeared to be a difference in how the participants interpreted the neutral cue, with the older adults appearing to interpret the neutral cues in a similar way to the congruent cues. Overall, we provide important evidence that sharing attention benefits cognition across the life span. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Two Cultural Processing Asymmetries Drive Spatial Attention.

Cultural routines, such as reading and writing direction (script direction), channel attention orientation. Depending on one's native language habit, attention is biased from left-to-right (LR) or from right-to-left (RL). Here, we further document this bias, as it interacts with the spatial directionality that grounds time concepts. We used a spatial cueing task to test whether script direction and the grounding of time in Portuguese (LR, Exp. 1) and Arabic (RL, Exp. 2) shape visuomotor performance in target discrimination. Temporal words (e.g., tomorrow, yesterday) were presented as cues in two modalities: visual (Exp. 1-2) and auditory (Exp. 1). Gaze movement (Exp. 1) and speed of discrimination decisions (Exp. 1-2) of targets presented to the left or right sides of the screen were assessed. As predicted, the interaction between target location and time concepts was significant across both modalities and linguistic communities. Additionally, LR participants detected the target on the right side of the screen faster after a future word than the target on the left side of the screen after a past word cue. In contrast, RL participants detected the target on the left side of the screen faster when the cue word was a future word than the target on the right side of the screen cued by a past word. In both modalities, the initial eye-gaze movement (Exp. 1) was responsive to the cue's time referent, further confirming that time orients attention. An additional bias was observed for the first fixation onset, which landed earlier on the target set that matched habitualized spatial routines. We conclude that scanning regularities are shaped by writing habits and bodily grounded categorical features.

Spatiotemporal Coupling of Hand and Eye Movements When Using a Myoelectric Prosthetic Hand.

Upper limb prosthesis users have disruptions in hand-eye coordination, with increased fixations towards the hand and less visual allocation for feedforward planning. The purpose of this study was to explore whether improved motor planning, as reflected by eye gaze behaviour, was associated with more efficient hand movement patterns. Able-bodied participants wore a simulated prosthesis while performing a functional object movement task. Motion and eye tracking data were collected to quantify the eye gaze and hand movement during object interaction. The results of this study demonstrated that the latency of the eye to precede the hand at pick-up was correlated with measures of hand function, including hand variability, movement units, and grasp time, but not reach time. During transport and release, longer latency to disengage gaze from the grasped object and look ahead towards the target was correlated to hand kinematics of hand variability, distance travelled, and transport time. In addition, the latency of the eye to disengage the drop-off location was correlated to release time. Together these may point to control issues with opening and closing the prosthetic hand. Overall, increased feedforward fixations towards the target and reduced feedback fixations towards the hand were related to improved measures of hand function. Hence, coordination between eye and hand movements when using a myoelectric prosthesis may prove to be a useful metric to assess motor planning.

Visual Art Design of Digital Works Guided by Big Data

With the rapid development of digital technology, the development speed of digital media is also relatively fast. Digital media technology has a great impact on people’s lifestyles and aesthetic concepts, and it also has a greater impact on visual art, creative thinking communication methods, and expression methods. In this study, the quality enhancement of digital images has been intensively studied based on the guidance of big data of eye-movement gaze points. A large amount of visual data are collected from public social resources, and the optimization research of image sensory quality is carried out in-depth using the acquired big data. Next, the region of interest (ROI) is obtained by combining the data with a two-dimensional Gaussian distribution model-fitting method, and the obtained data clustered and improved based on the K-means clustering algorithm to obtain ROI fixation points. Finally, discontinuities in the choice of sharpness in graphics and video playback are pointed out, and the final fixation data analysis is utilized. Results show that targeted optimization is very effective in improving the quality of digital images and saving space, enabling users to enjoy higher-quality visual digital images. The proposed method can be used to improve the dynamic resolution of images and videos.

Eye gaze controlled adaptive virtual keyboard for users with SSMI

BACKGROUND: Users with Severe Speech and Motor Impairment (SSMI) often use a communication chart through their eye gaze or limited hand movement and care takers interpret their communication intent. There is already significant research conducted to automate this communication through electronic means. Developing electronic user interface and interaction techniques for users with SSMI poses significant challenges as research on their ocular parameters found that such users suffer from Nystagmus and Strabismus limiting number of elements in a computer screen. This paper presents an optimized eye gaze controlled virtual keyboard for English language with an adaptive dwell time feature for users with SSMI. OBJECTIVE: Present an optimized eye gaze controlled English virtual keyboard that follows both static and dynamic adaptation process. The virtual keyboard can automatically adapt to reduce eye gaze movement distance and dwell time for selection and help users with SSMI type better without any intervention of an assistant. METHODS: Before designing the virtual keyboard, we undertook a pilot study to optimize screen region which would be most comfortable for SSMI users to operate. We then proposed an optimized two-level English virtual keyboard layout through Genetic algorithm using static adaptation process; followed by dynamic adaptation process which tracks users’ interaction and reduces dwell time based on a Markov model-based algorithm. Further, we integrated the virtual keyboard for a web-based interactive dashboard that visualizes real-time Covid data. RESULTS: Using our proposed virtual keyboard layout for English language, the average task completion time for users with SSMI was 39.44 seconds in adaptive condition and 29.52 seconds in non-adaptive condition. Overall typing speed was 16.9 lpm (letters per minute) for able-bodied users and 6.6 lpm for users with SSMI without using any word completion or prediction features. A case study with an elderly participant with SSMI found a typing speed of 2.70 wpm (words per minute) and 14.88 lpm (letters per minute) after 6 months of practice. CONCLUSIONS: With the proposed layout for English virtual keyboard, the adaptive system increased typing speed statistically significantly for able bodied users than a non-adaptive version while for 6 users with SSMI, task completion time reduced by 8.8% in adaptive version than nonadaptive one. Additionally, the proposed layout was successfully integrated to a web-based interactive visualization dashboard thereby making it accessible for users with SSMI.

The Effect of Simultaneously Presented Words and Auditory Tones on Visuomotor Performance.

The experiment reported here used a variation of the spatial cueing task to examine the effects of unimodal and bimodal attention-orienting primes on target identification latencies and eye gaze movements. The primes were a nonspatial auditory tone and words known to drive attention consistent with the dominant writing and reading direction, as well as introducing a semantic, temporal bias (past-future) on the horizontal dimension. As expected, past-related (visual) word primes gave rise to shorter response latencies on the left hemifield and future-related words on the right. This congruency effect was differentiated by an asymmetric performance on the right space following future words and driven by the left-to-right trajectory of scanning habits that facilitated search times and eye gaze movements to lateralized targets. Auditory tone prime alone acted as an alarm signal, boosting visual search and reducing response latencies. Bimodal priming, i.e., temporal visual words paired with the auditory tone, impaired performance by delaying visual attention and response times relative to the unimodal visual word condition. We conclude that bimodal primes were no more effective in capturing participants' spatial attention than the unimodal auditory and visual primes. Their contribution to the literature on multisensory integration is discussed.

Design and Evaluation of Attention Guidance Through Eye Gazing of “NAMIDA” Driving Agent

The driving agents considered thus far have aimed at navigating the driver’s attention while driving, for example, using interactions through linguistic conversations. Therefore, in this study, to investigate such a role in automatic driving from the perspective of nonverbal communication focusing on physicality (e.g., head movements and eye gaze), we constructed a driving agent called NAMIDA, along with its physical properties, as a research platform to investigate the role of nonverbal communication. We conducted a cognitive experiment on attention guidance, focusing on “gaze direction,” i.e., the movement of the eyes of NAMIDA. As a result, we confirmed that the attention of the participants is attracted by such eye-gaze movements of “NAMIDA,” which become a “cue” to exploring the surroundings.

The Search Patterns of Abdominal Imaging Subspecialists for Abdominal Computed Tomography: Toward a Foundational Pattern for New Radiology Residents.

Objectives:The routine search patterns used by subspecialty abdominal imaging experts to inspect the image volumes of abdominal/pelvic computed tomography (CT) have not been well characterized or rendered in practical or teachable terms. The goal of this study is to describe the search patterns used by experienced subspecialty imagers when reading a normal abdominal CT at a modern picture archiving and communication system workstation, and utilize this information to propose guidelines for residents as they learn to interpret CT during training.Material and Methods:Twenty-two academic subspecialists enacted their routine search pattern on a normal contrast-enhanced abdominal/pelvic CT study under standardized display parameters. Readers were told that the scan was normal and then asked to verbalize where their gaze centered and moved through the axial, coronal, and sagittal image stacks, demonstrating eye position with a cursor as needed. A peer coded the reported eye gaze movements and scrilling behavior. Spearman correlation coefficients were calculated between years of professional experience and the numbers of passes through the lung bases, liver, kidneys, and bowel.Results:All readers followed an initial organ-by-organ approach. Larger organs were examined by drilling, while smaller organs by oscillation or scanning. Search elements were classified as drilling, scanning, oscillation, and scrilling (scan drilling); these categories were parsed as necessary. The greatest variability was found in the examination the body wall and bowel/mesentery. Two modes of scrilling were described, and these classified as roaming and zigzagging. The years of experience of the readers did not correlated to number of passes made through the lung bases, liver, kidneys, or bowel.Conclusion:Subspecialty abdominal radiologists negotiate through the image stacks of an abdominal CT study in broadly similar ways. Collation of the approaches suggests a foundational search pattern for new trainees.

Directing User’s Eye Gaze Movement in an Interactive VR Animation

Directing User’s Eye Gaze Movement in an Interactive VR Animation