Gaze Transition Research Articles

Action capability constrains visuo-motor complexity during planning and performance in on-sight climbing.

The capability to adapt to changing conditions is crucial for safe and successful performance in physical activities and sports. According to the affordance‐based control perspective, individuals act in such a way as to take into account the limits of their capability to act. However, it is not clear how strength interacts with skill in shaping performer‐environment interactions. We, therefore, determined whether fingertip strength influences patterns of gaze and climbing behavior on new routes of ever‐increasing difficulty. We expected that comparatively weaker climbers would show less complex behavior because of an inability to perceive and act. Stronger climbers would show more complex visuo‐motor behavior because more opportunities for action remain, even as route difficulty increases. For very strong climbers the route would not be challenging enough, and less complex patterns suffice. Twenty climbers, ranging from lower grade to elite level participated. Maximum fingertip strength was obtained. Participants previewed and then climbed two separate 3 m long traverses, gradually decreasing in edge depth. Gaze and hip positions were collected for subsequent computation of gaze transition entropy (during preview) and hip displacement entropy (during climbing). Data revealed statistically significant curvilinear relationships between both fingertip strength and gaze transition entropy, and fingertip strength, and hip displacement entropy. Visuo‐motor complexity is scaled by how close the individual must act relative to boundaries of what the environment affords and does not afford for action given the individual constraints. Future research should examine in greater detail relationships between action capabilities and functional movement variability.

Viewing garden scenes: Interaction between Gaze Behavior and Physiological Responses.

Previous research has shown that exposure to Japanese gardens reduces physiological measures of stress, e.g. heart rate, in both healthy subjects and dementia patients. However, the correlation between subjects’ physiological responses and their visual behavior while viewing the garden has not yet been investigated. To address this, we developed a system to collect simultaneous measurements of eye gaze and three physiological indicators of autonomic nervous system activity: electrocardiogram, blood volume pulse, and galvanic skin response. We recorded healthy subjects’ physiological/behavioral responses when they viewed two environments (an empty courtyard and a Japanese garden) in two ways (directly or as a projected 2D photograph). Similar to past work, we found that differences in subject’s physiological responses to the two environments when viewed directly, but not as a photograph. We also found differences in their behavioral responses. We quantified subject’s behavioral responses using several gaze metrics commonly considered to be measures of engagement of focus: average fixation duration, saccade amplitude, spatial entropy and gaze transition entropy. We found decrease in gaze transition entropy, the only metric that accounts for both the spatial and temporal properties of gaze, to have a weak positive correlation with decrease in heart rate. This suggests a relationship between engagement/focus and relaxation. Finally, we found gender differences: females’ gaze patterns were more spatially distributed and had higher transition entropy than males.

Extraction of parameters for lane change intention based on driver’s gaze transfer characteristics

In naturalistic driving environment, lane change is a necessary, frequent operation, and causes traffic accidents and fatalities. Current lane-change driver assistance system uses turn signals as start time; however, drivers show lane-change intention before turning on the signals. To identify lane-change intention, a driving experiment was designed and conducted, during which eye movement characteristics were recorded from 16 drivers. Time period data from 5 s before changing lanes and 5 s in lane-keeping period were analysed. Saccade speed, eye–head coordination mode, and other parameters were confirmed to be significantly different between two periods. Based on area division and Markov theory, attributes related to gaze transition were obtained, including one- and two-step gaze transition probabilities, main gaze transition paths, and regional gaze probabilities. In summary, all features extracted from eye movement characteristics, gaze transition paths, and gaze areas demonstrate that visual attention is more dispersed during the lane-change intention period. These parameters also lay the foundation for lane-change intention recognition systems and make lane change safer.

Gaze entropy measures detect alcohol-induced driver impairment

Driving under the influence of alcohol is an ongoing cause of road traffic accidents. The biphasic nature of alcohol effects on subjective experience appears to contribute to the prevalence of drink-driving, as people perceive the declining phase of the BAC curve as recovery from intoxication and are more willing to drive despite significant impairments in objectively measured functions. The present study investigates whether alcohol-induced changes in gaze behaviour can be detected during engagement in a simulated driving task. In a repeated-measures and placebo-controlled design, this study examines the biphasic influence of moderate alcohol intake (0.6 g/kg) on measures of gaze behaviour and simulated driving performance. Twenty-two healthy young adults completed three driving sessions (baseline, ascending and descending) under two conditions (placebo, alcohol) while their eye movements were simultaneously recorded. The results revealed that gaze behaviour as measured by gaze transition entropy (GTE) and stationary gaze entropy (SGE) and driving performance measured by the standard deviation of lateral position (SDLP) of the vehicle, were significantly affected by alcohol across the ascending and descending sessions. The alcohol-induced reduction in GTE with an increase in SGE is discussed as alcohol’s impact on top-down modulation of gaze resulting in more dispersed and erratic pattern of visual scanning. The observed changes in gaze behaviour also mediated the influence of alcohol upon driving performance. These results have significant implications for the development of driver monitoring systems that can detect alcohol-induced impairment.

Gaze Information Channel in Cognitive Comprehension of Poster Reading.

Today, eye trackers are extensively used in studying human cognition. However, it is hard to analyze and interpret eye movement data from the cognitive comprehension perspective of poster reading. To find quantitative links between eye movements and cognitive comprehension, we tracked observers’ eye movement for reading scientific poster publications. We model in this paper eye tracking fixation sequences between content-dependent Areas of Interests (AOIs) as a Markov chain. Furthermore, we use the fact that a Markov chain is a special case of information or communication channel. Then, the gaze transition can be modeled as a discrete information channel, the gaze information channel. Next, some traditional eye tracking metrics, together with the gaze entropy and mutual information of the gaze information channel are calculated to quantify cognitive comprehension for every participant. The analysis of the results demonstrate that the gaze entropy and mutual information from individual gaze information channel are related to participants’ individual differences. This is the first study that eye tracking technology has been used to assess the cognitive comprehension of poster reading. The present work provides insights into human cognitive comprehension by using the novel gaze information channel methodology.

Gaze Entropy Measures Reveal Alcohol-Induced Visual Scanning Impairment During Ascending and Descending Phases of Intoxication

Visuospatial awareness is critical for everyday activities like driving. Higher order processes, such as visuospatial working memory (VSWM) and top-down modulation of gaze control, enable goal-driven visual scanning. Although available evidence suggests that alcohol differentially affects VSWM during the ascending and descending phases of the blood alcohol concentration (BAC) curve, it remains unclear whether such impact extends to a systemic disruption of visual scanning behavior. In a placebo-controlled and repeated-measures design, the present study investigated the influence of moderate alcohol intake (0.6 g/kg) on VSWM and gaze behavior by using gaze entropy measures to quantify visual scanning efficiency. Thirty-eight (18 female, 20 male) healthy participants completed a VSWM task across three consecutive sessions (baseline, ascending, descending) while their eye movements were simultaneously recorded. Performance in VSWM was affected during the descending session, where response accuracy declined significantly. Stationary gaze entropy (SGE) and gaze transition entropy (GTE) measures significantly reduced during both ascending and descending sessions. Fixation rate and duration were also affected by alcohol, but only during the ascending session. Reduction in SGE suggests a less explorative distribution of gaze, whereas low GTE is indicative of reduced visual scanning efficiency. These findings reflect the detrimental effects of alcohol on top-down control of gaze behavior, which may limit visual scanning efficiency. Deficits in visual scanning efficiency across the blood alcohol concentration curve may reduce visuospatial awareness and contribute to unintentional injuries during periods of intoxication.

On the Interaction between Gaze Behavior and Physiological Responses when Viewing Garden Scenes.

Recent research has shown that design principles inherent in a Japanese style garden can reduce measures of stress in both healthy and dementia patients. However, it was not clear how subjects' visual interaction with the scene affected their physiological responses. To address that, we developed a novel non-invasive system to collect synchronized measurements of eye gaze and physiological indicators of sympathetic neural activity: the electrocardiogram, the blood volume pulse and the galvanic skin response, as subjects view a garden environment. We characterized the visual engagement of subjects using the average fixation duration, the saccade amplitude and the gaze transition entropy. We find a statistically significant positive correlation between gaze transition entropy and mean heart rate change. Our results suggest that the visual engagement of subjects with their environment may influence their physiological responses to it: more engagement may lead to more relaxation. Our results also highlight the importance of taking into account the detailed spatio-temporal characteristics of the gaze trajectory.

Behavioral Analysis of Kinetic Telepresence for Small Symmetric Group-to-Group Meetings

Nonverbal behavior analysis revealed the effect of MMSpace , a kinetic telepresence developed for social telepresence, on small symmetric group-to-group conversations. MMSpace consists of kinetic avatars, equipped with flat projection screen panels as faces, that can change their pose and position automatically to mirror the remote user's head motions. The advantage is the realistic kinetic expression of human head movements, which form gestures like nodding and indicate the focus of visual attention, through the use of four degree-of-freedom low-latency precision actuators. Another feature is the support of eye contact among remote participants, which is made possible by the avatar's kinetic pose changes and by adaptive camera selection for orienting the user's face toward the remote addressee. Its limitation is its room-scale infrastructure and restricted participant positions. Targeting a symmetric 2 $\times$ 2 setting, participants’ nonverbal behaviors, including gaze directions and head gestures, were compared among three conditions, MMSpace with/without physical motions and face-to-face settings. There was a significant difference between the conditions in terms of the duration of glance/mutual glances, total gaze transition time, amount of head gesturing, and co-occurrences of head gestures in the remote participants. The results indicate that the avatar's physical motion can elicit longer (mutual) glances with a shorter total transition time and more (co-)occurrences of head gestures, and it makes MMSpace -based conversations closer, in terms of these nonverbal statistics, to face-to-face ones compared with those of a static version of MMSpace without physical motion.

Watch to Edit: Video Retargeting using Gaze

AbstractWe present a novel approach to optimally retarget videos for varied displays with differing aspect ratios by preserving salient scene content discovered via eye tracking. Our algorithm performs editing with cut, pan and zoom operations by optimizing the path of a cropping window within the original video while seeking to (i) preserve salient regions, and (ii) adhere to the principles of cinematography. Our approach is (a) content agnostic as the same methodology is employed to re‐edit a wide‐angle video recording or a close‐up movie sequence captured with a static or moving camera, and (b) independent of video length and can in principle re‐edit an entire movie in one shot.Our algorithm consists of two steps. The first step employs gaze transition cues to detect time stamps where new cuts are to be introduced in the original video via dynamic programming. A subsequent step optimizes the cropping window path (to create pan and zoom effects), while accounting for the original and new cuts. The cropping window path is designed to include maximum gaze information, and is composed of piecewise constant, linear and parabolic segments. It is obtained via L(1) regularized convex optimization which ensures a smooth viewing experience. We test our approach on a wide variety of videos and demonstrate significant improvement over the state‐of‐the‐art, both in terms of computational complexity and qualitative aspects. A study performed with 16 users confirms that our approach results in a superior viewing experience as compared to gaze driven re‐editing [JSSH15] and letterboxing methods, especially for wide‐angle static camera recordings.

Directing Gaze on a Scene Before Reaching for an Object: Changes Over the First Year of Life

We used eye tracking to investigate where infants and adults directed their gaze on a scene right before reaching. Infants aged 5, 7, 9, and 11 months old and adults looked at a human hand holding an object out of reach for 5 s, then the hand moved the object toward the participant for reaching. We analyzed which part of the scene (the object, the hand, or elsewhere) infants and adults attended the most during those 5 s before reaching. Findings revealed that adults’ visual fixations were majorly focused on the object to reach. Young infants’ looking patterns were more widely distributed between the hand holding the object, the object, and other nonrelevant areas on the scene. Despite distributed looking on the scene, infants increased their amount of time looking at the object between 5 and 11 months. Nine- and 11-month-olds showed overall accumulated looking durations comparable to adults’ for most of the objects; however, 9-month-olds differed in their rate of gaze transition between scene areas. From the age of 5 months old, infants are able to sustain their gaze to the pertinent scene area when the scene contains a central object on which they will later be able to act.

Classification of Driver's Neutral and Cognitive Distraction States Based on Peripheral Vehicle Behavior in Driver's Gaze Transition

To support safe driving, numerous methods of detecting distractions using measurements of a driver's gaze have been proposed. These methods empirically focused on certain driving contexts and analyzed gaze behavior under particular peripheral vehicle conditions; therefore, numerous driving situations were not considered. To address this problem with hypothesis-testing approaches, we turn the problem around and propose a data-mining approach that analyzes peripheral vehicle behavior during gaze transitions of drivers in order to compare their neutral driving state with a cognitive distraction state. This change in thinking is the first contribution of this paper. The analysis results show that under the neutral condition, drivers generally turned their gaze to peripheral vehicles to be focused on; however, they did not do this consistently under the distracted condition. As the second contribution, we propose a simple classifier to discriminate between the cognitive distraction and neutral states by analyzing the peripheral vehicle behavior. The proposed classifier can manage various situations and provide high classification accuracy by focusing on gaze transitions from the front view toward other directions.

Prediction of Who Will Be the Next Speaker and When Using Gaze Behavior in Multiparty Meetings

In multiparty meetings, participants need to predict the end of the speaker’s utterance and who will start speaking next, as well as consider a strategy for good timing to speak next. Gaze behavior plays an important role in smooth turn-changing. This article proposes a prediction model that features three processing steps to predict (I) whether turn-changing or turn-keeping will occur, (II) who will be the next speaker in turn-changing, and (III) the timing of the start of the next speaker’s utterance. For the feature values of the model, we focused on gaze transition patterns and the timing structure of eye contact between a speaker and a listener near the end of the speaker’s utterance. Gaze transition patterns provide information about the order in which gaze behavior changes. The timing structure of eye contact is defined as who looks at whom and who looks away first, the speaker or listener, when eye contact between the speaker and a listener occurs. We collected corpus data of multiparty meetings, using the data to demonstrate relationships between gaze transition patterns and timing structure and situations (I), (II), and (III). The results of our analyses indicate that the gaze transition pattern of the speaker and listener and the timing structure of eye contact have a strong association with turn-changing, the next speaker in turn-changing, and the start time of the next utterance. On the basis of the results, we constructed prediction models using the gaze transition patterns and timing structure. The gaze transition patterns were found to be useful in predicting turn-changing, the next speaker in turn-changing, and the start time of the next utterance. Contrary to expectations, we did not find that the timing structure is useful for predicting the next speaker and the start time. This study opens up new possibilities for predicting the next speaker and the timing of the next utterance using gaze transition patterns in multiparty meetings.

Gaze transitions when learning with multimedia

Eye tracking methodology is used to examine the influence of interactive multimedia on the allocation of visual attention and its dynamics during learning. We hypothesized that an interactive simulation promotes more organized switching of attention between different elements of multimedia learning material, e.g., textual description and pictorial visualization. Participants studied a description of an algorithm accompanied either by an interactive simulation, self-paced animation, or static illustration. Using a novel framework for entropy-based comparison of gaze transition matrices, results showed that the interactive simulation elicited more careful visual investigation of the learning material as well as reading of the problem description through to its completion.

Discriminating classes of sequential and qualitative colour schemes

ABSTRACTWe present an analysis of the influence of colour distance on the user experience with choropleth and chorochromatic maps. We systematically evaluated five sequential and five qualitative colour schemes in a two-stage user experiment. At first, we conducted an online study to obtain performance metrics accuracy and response time from a large, heterogeneous population. Following this, in a controlled lab study with eye-tracking, we re-examined the findings from the online study for a subset of experimental stimuli and further assessed the user experience through an analysis of their visual behaviour. In this process, along with accuracy and response time, eye-tracking metrics fixation frequency, fixation duration and scanpath speed as well as a gaze transition analysis were utilized. In both experiments, participants were asked to compare two areas with controlled colour distances between them, and decide whether these areas were of the same colour. Results suggest that we are able to discriminate narrower colour distances than commonly used in practice, however, not as narrow as suggested in previous literature.

Neural Activity To Viewed Dynamic Gaze Is Affected By Social Decision

Event Abstract Back to Event Neural Activity To Viewed Dynamic Gaze Is Affected By Social Decision Aina Puce1*, Marianne Latinus2, Scott Love2, Alejandra Rossi1, Francisco Parada1, Lisa Huang1, Laurence Conty3, Karin James1 and Nathalie George4 1 Indiana University, Psychological and Brain Sciences, United States 2 CNRS-Aix Marseille Université, Institut de Neurosciences de la Timone, France 3 3CNRS, UMR 7225, CRICM, France 4 3CNRS, UMR 7225, CRICM, France Puce et al. (2000) reported larger N170s to gaze aversion in a passive viewing task, while Conty et al. (2007) described larger N170s to gaze change toward the subject in a social judgment task. We aimed to reconcile these studies by recording ERPs in two tasks in the same subjects. In a Social Task, subjects judged if a gaze transition moved Away or Toward them. In a Non-social Task, a gaze transition was judged as moving to the Left or Right. Continuous 256 channel EEG was recorded from 22 healthy adults viewing dynamic gaze stimuli in 6 conditions: (1) Direct to Extreme gaze [Dir-Ext] (2) Extreme to Direct gaze [Ext-Dir] (3) Intermediate to Extreme [Int-Ext] (4) Intermediate to Direct gaze [Int-Dir] (5) Direct gaze to Intermediate [Dir-Int] (6) Extreme to Intermediate [Ext-Int] Conditions (1) and (2) were from Puce et al. (2000), whereas (3) and (4) were a subset from Conty et al. (2007). Conditions (5) and (6) were not used previously. N170 peak latencies and amplitudes were calculated from two occipitotemporal 9-electrode clusters. Repeated measures ANOVAs were run for behavior and N170 latency and amplitude. Behavior: Subjects were faster in the non-social task, and more accurate for gaze aversions (non-social task). In the social task, faster RTs occurred for gaze transitions made toward subjects. ERP findings: In the non-social task, N170s were significantly larger for gaze aversions relative to gaze changes toward subjects. In the social task, this difference disappeared in the right hemisphere. The current study reconciled differences of two earlier studies. The brain's response to a gaze change depends on the decision being made. When not in a 'social' mode, the brain selectively responds (with a larger N170) to another's averted gaze. Operating in 'social' mode leads to larger N170s to gaze transitions toward the subject, suggesting that social context increases salience of direct gaze consistent with our behavioral data. Keywords: Gaze Direction, social attention, eyes, ERPs, N170 Conference: XII International Conference on Cognitive Neuroscience (ICON-XII), Brisbane, Queensland, Australia, 27 Jul - 31 Jul, 2014. Presentation Type: Oral Presentation Topic: Emotional and Social Processes Citation: Puce A, Latinus M, Love S, Rossi A, Parada F, Huang L, Conty L, James K and George N (2015). Neural Activity To Viewed Dynamic Gaze Is Affected By Social Decision. Conference Abstract: XII International Conference on Cognitive Neuroscience (ICON-XII). doi: 10.3389/conf.fnhum.2015.217.00051 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 19 Feb 2015; Published Online: 24 Apr 2015. * Correspondence: Dr. Aina Puce, Indiana University, Psychological and Brain Sciences, Bloomington, United States, ainapuce@gmail.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Aina Puce Marianne Latinus Scott Love Alejandra Rossi Francisco Parada Lisa Huang Laurence Conty Karin James Nathalie George Google Aina Puce Marianne Latinus Scott Love Alejandra Rossi Francisco Parada Lisa Huang Laurence Conty Karin James Nathalie George Google Scholar Aina Puce Marianne Latinus Scott Love Alejandra Rossi Francisco Parada Lisa Huang Laurence Conty Karin James Nathalie George PubMed Aina Puce Marianne Latinus Scott Love Alejandra Rossi Francisco Parada Lisa Huang Laurence Conty Karin James Nathalie George Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Gaze awareness in conversational agents

In face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation, and they change their conversational strategy if the listener is not fully engaged. With the goal of building a conversational agent that can adaptively control conversations, in this study we analyze listener gaze behaviors and develop a method for estimating whether a listener is engaged in the conversation on the basis of these behaviors. First, we conduct a Wizard-of-Oz study to collect information on a user's gaze behaviors. We then investigate how conversational disengagement, as annotated by human judges, correlates with gaze transition, mutual gaze (eye contact) occurrence, gaze duration, and eye movement distance. On the basis of the results of these analyses, we identify useful information for estimating a user's disengagement and establish an engagement estimation method using a decision tree technique. The results of these analyses show that a model using the features of gaze transition, mutual gaze occurrence, gaze duration, and eye movement distance provides the best performance and can estimate the user's conversational engagement accurately. The estimation model is then implemented as a real-time disengagement judgment mechanism and incorporated into a multimodal dialog manager in an animated conversational agent. This agent is designed to estimate the user's conversational engagement and generate probing questions when the user is distracted from the conversation. Finally, we evaluate the engagement-sensitive agent and find that asking probing questions at the proper times has the expected effects on the user's verbal/nonverbal behaviors during communication with the agent. We also find that our agent system improves the user's impression of the agent in terms of its engagement awareness, behavior appropriateness, conversation smoothness, favorability, and intelligence.

A model of head movement contribution for gaze transitions

Head posture has been associated with work-related neck symptoms and discomfort, but its relationship with visual tasks has received much less attention. Head movement amplitude is normally a fraction of the angular distance to a visual target, as gaze transition is usually achieved through the combination of both head and eye movement. In this study, the proportion of head orientation vs. target orientation, named head movement contribution ratio (HMCR), was quantified and modelled as a function of target location. Head movements were measured on subjects orienting and maintaining gaze for 2 s at randomly presented visual targets distributed along an arc placed horizontally or vertically. Bootstrap regression models showed that the horizontal HMCR was approximately 69% of target azimuth. The vertical HMCR was bilinear and corresponded to 52% and 8% and of target elevation for targets above and below the horizontal plane, respectively. The data also demonstrated that head orientation is affected by the kinematic coupling between horizontal and vertical components of head movement. Statement of Relevance: Awkward head and neck posture is a risk factor for work-related musculoskeletal disorders. This study investigated the influence of visual target location on head orientation over a large range of target eccentricity, as an attempt to predict the head and neck posture required for visual target detection and identification.