Visual Information Uptake Research Articles

Sensorimotor prediction is used to direct gaze toward task-relevant locations in a goal-directed throwing task.

Previous research has shown that action effects of self-generated movements are internally predicted before outcome feedback becomes available. To test whether these sensorimotor predictions are used to facilitate visual information uptake for feedback processing, we measured eye movements during the execution of a goal-directed throwing task. Participants could fully observe the effects of their throwing actions (ball trajectory and either hitting or missing a target) in most of the trials. In a portion of the trials, the ball trajectory was not visible, and participants only received static information about the outcome. We observed a large proportion of predictive saccades, shifting gaze toward the goal region before the ball arrived and outcome feedback became available. Fixation locations after predictive saccades systematically covaried with future ball positions in trials with continuous ball flight information, but notably also in trials with static outcome feedback and only efferent and proprioceptive information about the movement that could be used for predictions. Fixation durations at the chosen positions after feedback onset were modulated by action outcome (longer durations for misses than for hits) and outcome uncertainty (longer durations for narrow vs. clear outcomes). Combining both effects, durations were longest for narrow errors and shortest for clear hits, indicating that the chosen locations offer informational value for feedback processing. Thus, humans are able to use sensorimotor predictions to direct their gaze toward task-relevant feedback locations. Outcome-dependent saccade latency differences (miss vs. hit) indicate that also predictive valuation processes are involved in planning predictive saccades.NEW & NOTEWORTHY We elucidate the potential benefits of sensorimotor predictions, focusing on how the system actually uses this information to optimize feedback processing in goal-directed actions. Sensorimotor information is used to predict spatial parameters of movement outcomes, guiding predictive saccades toward future action effects. Saccade latencies and fixation durations are modulated by outcome quality, indicating that predictive valuation processes are considered and that the locations chosen are of high informational value for feedback processing.

Automotive Holographic Head-Up Displays.

Driver's access to information about navigation and vehicle data through in-car displays and personal devices distract the driver from safe vehicle management. The discrepancy between road safety and infotainment must be addressed to develop safely operated modern vehicles. Head-up displays (HUDs) aim to introduce a seamless uptake of visual information for the driver while securely operating a vehicle. HUDs projected on the windshield provide the driver with visual navigation and vehicle data within the comfort of the driver's personal eye box through a customizable extended display space. Windshield HUDs do not require the driver to shift the gaze away from the road to attain road information. This article presents a review of technological advances and future perspectives in holographic HUDs by analyzing the optoelectronics devices and the user experience of the driver. The review elucidates holographic displays and full augmented reality in 3D with depth perception when projecting the visual information on the road within the driver's gaze. Design factors, functionality, and the integration of personalized machine learning technologies into holographic HUDs are discussed. Application examples of the display technologies regarding road safety and security are presented. An outlook is provided to reflect on display trends and autonomous driving.

Speaking for seeing: Sentence structure guides visual event apprehension

Human experience and communication are centred on events, and event apprehension is a rapid process that draws on the visual perception and immediate categorization of event roles (“who does what to whom”). We demonstrate a role for syntactic structure in visual information uptake for event apprehension. An event structure foregrounding either the agent or patient was activated during speaking, transiently modulating the apprehension of subsequently viewed unrelated events. Speakers of Dutch described pictures with actives and passives (agent and patient foregrounding, respectively). First fixations on pictures of unrelated events that were briefly presented (for 300 ms) next were influenced by the active or passive structure of the previously produced sentence. Going beyond the study of how single words cue object perception, we show that sentence structure guides the viewpoint taken during rapid event apprehension.

Construal in language: A visual-world approach to the effects of linguistic alternations on event perception and conception

Abstract The theoretical notion of ‘construal’ captures the idea that the way in which we describe a scene reflects our conceptualization of it. Relying on the concept of ception – which conjoins conception and perception – we operationalized construal and employed a Visual World Paradigm to establish which aspects of linguistic scene description modulate visual scene perception, thereby affecting event conception. By analysing viewing behaviour after alternating ways of describing location (prepositions), agentivity (active/passive voice) and transfer (NP/PP datives), we found that the linguistic construal of a scene affects its spontaneous visual perception in two ways: either by determining the order in which the components of a scene are accessed or by modulating the distribution of attention over the components, making them more or less salient than they naturally are. We also found evidence for the existence of a cline in the construal effect with stronger expressive differences, such as the prepositional manipulation, inducing more prominent changes in visual perception than the dative manipulation. We discuss the claims language can lay to affecting visual information uptake and hence conceptualization of a static scene in the light of these results.

Predictable events elicit less visual and temporal information uptake in an oddball paradigm.

In the visual oddball paradigm, surprising inputs can seem expanded in time relative to unsurprising repeated events. A horizontal input embedded in a train of successive vertical inputs can, for instance, seem relatively protracted in time, even if all inputs are presented for an identical duration. It is unclear if this effect results from surprising events becoming apparently protracted, or from repeated events becoming apparently contracted in time. To disambiguate, we used a non-relative duration reproduction task, in which several standards preceded a test stimulus that had to be reproduced. We manipulated the predictability of test content over successive presentations. Overall, our data suggest that predictable stimuli induce a contraction of apparent duration (Experiments 1, 3, and 4). We also examine sensitivity to test content, and find that predictable stimuli elicit less uptake of visual information (Experiments 2 and 3). We discuss these findings in relation to the predictive coding framework.

A smart peek: Processing of rapid visual displays is disturbed in newly diagnosed, cognitively intact MS patients and refers to cognitive performance and disease progression in late stages

BackgroundMS can reduce the speed of information processing (IPS) leading to a variable pattern of cognitive impairment. To better understand this deficit, a separate evaluation of the sensory, cognitive, and motor speed component is required. Tests using rapid visual displays allow for assessment of separate components of information uptake. We utilized such a test to compare deficit profiles at the earlier and later stage of MS and their relation to cognitive ability and disease progression. MethodTwo groups were evaluated: “Early MS” comprised N = 24 patients with disease durations <2 years; “late MS” N = 45 with disease durations >12 years. Rapid visual displays of letters were utilized to derive individual profiles of visual information uptake according to the ‘theory of visual attention’ (TVA). The resulting data was then compared with measures of disability, fatigue, depression, IPS, visual-spatial ability, verbal and visual memory. ResultsIn the EMS group, where cognitive impairment was the exception, three of the four main parameters of visual information uptake were already modified, i.e. processing rate C, storage capacity K, and iconic memory μ. In LMS an additional elevation of the fourth parameter, i.e., the perceptual threshold t0 was evident. Threshold values were related to most clinical and cognitive measures. ConclusionsAn early deficit pattern of visual information uptake can be detected at a stage, when performance in tests of IPS is still well-preserved. At later disease stages, a single parameter reflecting the threshold of conscious visual perception may provide a valid estimate of cognitive performance and disease progression.

Phasic alertness cues modulate visual processing speed in healthy aging.

Warning signals temporarily increase the rate of visual information in younger participants and thus optimize perception in critical situations. It is unclear whether such important preparatory processes are preserved in healthy aging. We parametrically assessed the effects of auditory alertness cues on visual processing speed and their time course using a whole report paradigm based on the computational Theory of Visual Attention. We replicated prior findings of significant alerting benefits in younger adults. In conditions with short cue-target onset asynchronies, this effect was baseline-dependent. As younger participants with high baseline speed did not show a profit, an inverted U-shaped function of phasic alerting and visual processing speed was implied. Older adults also showed a significant cue-induced benefit. Bayesian analyses indicated that the cueing benefit on visual processing speed was comparably strong across age groups. Our results indicate that in aging individuals, comparable to younger ones, perception is active and increased expectancy of the appearance of a relevant stimulus can increase the rate of visual information uptake.

Motor-cognitive dual-task performance: effects of a concurrent motor task on distinct components of visual processing capacity

Dual tasking, or the simultaneous execution of two continuous tasks, is frequently associated with a performance decline that can be explained within a capacity sharing framework. In this study, we assessed the effects of a concurrent motor task on the efficiency of visual information uptake based on the ‘theory of visual attention’ (TVA). TVA provides parameter estimates reflecting distinct components of visual processing capacity: perceptual threshold, visual processing speed, and visual short-term memory (VSTM) storage capacity. Moreover, goodness-of-fit values and bootstrapping estimates were derived to test whether the TVA-model is validly applicable also under dual task conditions, and whether the robustness of parameter estimates is comparable in single- and dual-task conditions. 24 subjects of middle to higher age performed a continuous tapping task, and a visual processing task (whole report of briefly presented letter arrays) under both single- and dual-task conditions. Results suggest a decline of both visual processing capacity and VSTM storage capacity under dual-task conditions, while the perceptual threshold remained unaffected by a concurrent motor task. In addition, goodness-of-fit values and bootstrapping estimates support the notion that participants processed the visual task in a qualitatively comparable, although quantitatively less efficient way under dual-task conditions. The results support a capacity sharing account of motor-cognitive dual tasking and suggest that even performing a relatively simple motor task relies on central attentional capacity that is necessary for efficient visual information uptake.

Parameter-Based Evaluation of Attentional Impairments in Schizophrenia and Their Modulation by Prefrontal Transcranial Direct Current Stimulation.

Attentional dysfunctions constitute core cognitive symptoms in schizophrenia, but the precise underlying neurocognitive mechanisms remain to be elucidated. In this randomized, double-blind, sham-controlled study, we applied, for the first time, a theoretically grounded modeling approach based on Bundesen's Theory of Visual Attention (TVA) to (i) identify specific visual attentional parameters affected in schizophrenia and (ii) assess, as a proof of concept, the potential of single-dose anodal transcranial direct current stimulation (tDCS; 20 min, 2 mA) to the left dorsolateral prefrontal cortex to modulate these attentional parameters. To that end, attentional parameters were measured before (baseline), immediately after, and 24 h after the tDCS intervention in 20 schizophrenia patients and 20 healthy controls. At baseline, analyses revealed significantly reduced visual processing speed and visual short-term memory storage capacity in schizophrenia. A significant stimulation condition × time point interaction in the schizophrenia patient group indicated improved processing speed at the follow-up session only in the sham condition (a practice effect), whereas performance remained stable across the three time points in patients receiving verum stimulation. In healthy controls, anodal tDCS did not result in a significant change in attentional performance. With regard to question (i) above, these findings are indicative of a processing speed and short-term memory deficit as primary sources of attentional deficits in schizophrenia. With regard to question (ii), the efficacy of single-dose anodal tDCS for improving (speed aspects of visual) cognition, it appears that prefrontal tDCS (at the settings used in the present study), rather than ameliorating the processing speed deficit in schizophrenia, actually may interfere with practice-dependent improvements in the rate of visual information uptake. Such potentially unexpected effects of tDCS ought to be taken into consideration when discussing its applicability in psychiatric populations. The study was registered at http://apps.who.int/trialsearch/Trial2.aspx?TrialID=DRKS00011665.

Lateralized electrical brain activity reveals covert attention allocation during speaking

Speakers usually begin to speak while only part of the utterance has been planned. Earlier work has shown that speech planning processes are reflected in speakers’ eye movements as they describe visually presented objects. However, to-be-named objects can be processed to some extent before they have been fixated upon, presumably because attention can be allocated to objects covertly, without moving the eyes. The present study investigated whether EEG could track speakers’ covert attention allocation as they produced short utterances to describe pairs of objects (e.g., “dog and chair”). The processing difficulty of each object was varied by presenting it in upright orientation (easy) or in upside down orientation (difficult). Background squares flickered at different frequencies in order to elicit steady-state visual evoked potentials (SSVEPs). The N2pc component, associated with the focusing of attention on an item, was detectable not only prior to speech onset, but also during speaking. The time course of the N2pc showed that attention shifted to each object in the order of mention prior to speech onset. Furthermore, greater processing difficulty increased the time speakers spent attending to each object. This demonstrates that the N2pc can track covert attention allocation in a naming task. In addition, an effect of processing difficulty at around 200–350ms after stimulus onset revealed early attention allocation to the second to-be-named object. The flickering backgrounds elicited SSVEPs, but SSVEP amplitude was not influenced by processing difficulty. These results help complete the picture of the coordination of visual information uptake and motor output during speaking.

The Interaction Between Vision and Eye Movements †.

The existence of a central fovea, the small retinal region with high analytical performance, is arguably the most prominent design feature of the primate visual system. This centralization comes along with the corresponding capability to move the eyes to reposition the fovea continuously. Past research on visual perception was mainly concerned with foveal vision while the observers kept their eyes stationary. Research on the role of eye movements in visual perception emphasized their negative aspects, for example, the active suppression of vision before and during the execution of saccades. But is the only benefit of our precise eye movement system to provide high acuity of the small foveal region, at the cost of retinal blur during their execution? In this review, I will compare human visual perception with and without saccadic and smooth pursuit eye movements to emphasize different aspects and functions of eye movements. I will show that the interaction between eye movements and visual perception is optimized for the active sampling of information across the visual field and for the calibration of different parts of the visual field. The movements of our eyes and visual information uptake are intricately intertwined. The two processes interact to enable an optimal perception of the world, one that we cannot fully grasp by doing experiments where observers are fixating a small spot on a display.

Which technology to investigate visual perception in sport: Video vs. virtual reality

Visual information uptake is a fundamental element of sports involving interceptive tasks. Several methodologies, like video and methods based on virtual environments, are currently employed to analyze visual perception during sport situations. Both techniques have advantages and drawbacks. The goal of this study is to determine which of these technologies may be preferentially used to analyze visual information uptake during a sport situation. To this aim, we compared a handball goalkeeper’s performance using two standardized methodologies: video clip and virtual environment. We examined this performance for two response tasks: an uncoupled task (goalkeepers show where the ball ends) and a coupled task (goalkeepers try to intercept the virtual ball). Variables investigated in this study were percentage of correct zones, percentage of correct responses, radial error and response time. The results showed that handball goalkeepers were more effective, more accurate and started to intercept earlier when facing a virtual handball thrower than when facing the video clip. These findings suggested that the analysis of visual information uptake for handball goalkeepers was better performed by using a ‘virtual reality’-based methodology. Technical and methodological aspects of these findings are discussed further.

Binocular Coordination of the Eyes during Reading

Saccadic eye movements and fixations are the behavioral means by which we visually sample text during reading. Human oculomotor control is governed by a complex neurophysiological system involving the brain stem, superior colliculus, and several cortical areas. A very widely held belief among researchers investigating primate vision is that the oculomotor system serves to orient the visual axes of both eyes to fixate the same target point in space. It is argued that such precise positioning of the eyes is necessary to place images on corresponding retinal locations, such that on each fixation a single, nondiplopic, visual representation is perceived. Vision works actively through a continual sampling process involving saccades and fixations. Here we report that during normal reading, the eyes do not always fixate the same letter within a word. We also demonstrate that saccadic targeting is yoked and based on a unified cyclopean percept of a whole word since it is unaffected if different word parts are delivered exclusively to each eye via a dichoptic presentation technique. These two findings together suggest that the visual signal from each eye is fused at a very early stage in the visual pathway, even when the fixation disparity is greater than one character (0.29 deg), and that saccade metrics for each eye are computed on the basis of that fused signal.

Automated Corneal-Reflection Eye Tracking in Infancy: Methodological Developments and Applications to Cognition.

Since the mid-1800s, experimental psychologists have been using eye movements and gaze direction to make inferences about perception and cognition in adults (Müller, 1826, cited in Boring, 1942). In the past 175 years, these oculomotor measures have been refined (see Kowler, 1990) and used to address similar questions in infants (see Aslin, 1985, 1987; Branson, 1982; Haith, 1980; Maurer, 1975). The general rationale for relying on these visual behaviors is that where one is looking is closely tied to what one is seeing. This is not to deny the fact that we can detect visual stimuli in the peripheral visual field, but rather that there is a bias to attend to and process information primarily when it is located in the central portion of the retina. Thus, although the direction of gaze is not perfectly correlated with the uptake of visual information (e.g., as in a blank stare or a covert shift of attention), there is a strong presumption that the direction of gaze can provide important information about visual stimuli even in newborn infants (Haith, 1966; Salapatek, 1968; Salapatek & Kessen, 1966).

Eye movements when reading disappearing text: is there a gap effect in reading?

Readers’ eye movements were monitored when they read either normal sentences or sentences with masked or disappearing text (in which the fixated word disappeared or was masked after 60 ms). The goals of the research were to investigate (1) whether a gap effect occurred in reading and (2) the influence of linguistic and visual factors on oculomotor control. The results of a number of global analyses of eye movements under disappearing text conditions clearly demonstrated that there is no gap effect in reading. However, comparative analyses across a number of local measures in the experiments indicated that cognitive/lexical processes, as well as the continual uptake of visual information, influence eye movement control during reading. A persistent visual object throughout fixation caused refixations and even when a fixated word had disappeared (or been masked), there were significant effects of word frequency and word length.

Reading disappearing text: cognitive control of eye movements.

Participants read sentences containing high- or low-frequency target words under normal reading conditions or disappearing-text conditions (in which the word that was fixated disappeared after 60 ms). Even though the fixated word had disappeared after 60 ms, there was still a robust frequency effect wherein readers fixated longer on low-frequency words than on high-frequency words. Thus, the results are consistent with cognitive-control models of eye movement control and inconsistent with visual/oculomotor-control models. Although the uptake of visual information is clearly important for reading, it is the cognitive processes associated with understanding the fixated words that drive the eyes through the text.

The eye-movements of the mantis shrimp Odontodactylus scyllarus (Crustacea: Stomatopoda)

Odontodactylus scyllarus makes discrete spontaneous eye-movements at a maximum rate of 3/s. These movements are unpredictable in direction and timing, and there is no detectable co-ordination between the two eyes. The eye-movements were measured with a computer-aided video method, and from 208 of these the following picture of a typical movement emerges. It has roughly equal horizontal and vertical components of 7–8°, taking the eye-stalk axis about 12° around a great circle, and also a rotational component of about 8°. The 3 components can occur independently of each other and are thus separately driven by the brain (Fig. 6). The average duration is 300 ms, and average velocity is 40° s (Fig. 5). Most movements are made in a direction approximately at right angles to the orientation of the specialised central band. It is shown that the slow speed of the eye-movements is compatible with scanning, that is, the uptake of visual information during the movement rather than its exclusion as in conventional saccades.

Predictive strategies in eye-head co-ordination during text processing.

During text processing skilled touch typists often scan copy and display in a repetitive way. They read a text string from the copy and consecutively check the typed characters on the display. This behavioural pattern permits analysis of eye-head co-ordination with respect to two different modes of visual information uptake, i.e. reading and checking. The results obtained showed that the eye-head co-ordination was predictive mainly in the sense that the head started to move a certain amount of time before the eye saccade was initiated. This effect has been analysed here in order to obtain criteria for the evaluation of cognitive difficulties during the performance of visual search tasks.