Task-relevant Visual Information Research Articles

Suppression on the basis of template for rejection is reactive: Evidence from human electrophysiology.

According to most theories of attention, the selection of task-relevant visual information can be enhanced by holding them in visual working memory (VWM). However, there has been a long-standing debate concerning whether similar optimization can also be achieved for task-irrelevant information, known as a "template for rejection". The present study aimed to explore this issue by examining the consequence of cue distractors before visual search tasks. For this endeavor, we manipulated the display heterogeneity by using two distractor conditions, salient and non-salient, to explore the extent to which holding the distractor color in VWM might affect attentional selection. We measured the reaction times of participants while their EEG activity was recorded. The results showed that WM-matched distractors did not improve reaction times but rather slowed them down in both tasks. Event-related potential (ERP) results showed that the display heterogeneity had no modulatory effect on the degree of distractor suppression. Even in the salient distractor condition, the WM-matched distractor received no greater suppression. Furthermore, the WM-matched distractor but not the neutral distractor elicited an N2pc before the PD in salient distractor conditions. This suggests that the template for rejection operates reactively since suppression occurs after extra attentional processes to the distractor. Moreover, the presence of WM-matched distractors led to a reduction of P3b, indicating a competition between target processing and WM-matched distractor rejection. Our findings provide insights into the mechanisms underlying the optimization of attentional selection, and have implications for future studies aimed at understanding the role of VWM in cognition.

Investigating the Effects of Avatarization and Interaction Techniques on Near-field Mixed Reality Interactions with Physical Components.

Mixed reality (MR) interactions feature users interacting with a combination of virtual and physical components. Inspired by research investigating aspects associated with near-field interactions in augmented and virtual reality (AR & VR), we investigated how avatarization, the physicality of the interacting components, and the interaction technique used to manipulate a virtual object affected performance and perceptions of user experience in a mixed reality fundamentals of laparoscopic peg-transfer task wherein users had to transfer a virtual ring from one peg to another for a number of trials. We employed a 3 (Physicality of pegs) X 3 (Augmented Avatar Representation) X 2 (Interaction Technique) multi-factorial design, manipulating the physicality of the pegs as a between-subjects factor, the type of augmented self-avatar representation, and the type of interaction technique used for object-manipulation as within-subjects factors. Results indicated that users were significantly more accurate when the pegs were virtual rather than physical because of the increased salience of the task-relevant visual information. From an avatar perspective, providing users with a reach envelope-extending representation, though useful, was found to worsen performance, while co-located avatarization significantly improved performance. Choosing an interaction technique to manipulate objects depends on whether accuracy or efficiency is a priority. Finally, the relationship between the avatar representation and interaction technique dictates just how usable mixed reality interactions are deemed to be.

Are auditory cues special? Evidence from cross-modal distractor-induced blindness

A target that shares features with preceding distractor stimuli is less likely to be detected due to a distractor-driven activation of a negative attentional set. This transient impairment in perceiving the target (distractor-induced blindness/deafness) can be found within vision and audition. Recently, the phenomenon was observed in a cross-modal setting involving an auditory target and additional task-relevant visual information (cross-modal distractor-induced deafness). In the current study, consisting of three behavioral experiments, a visual target, indicated by an auditory cue, had to be detected despite the presence of visual distractors. Multiple distractors consistently led to reduced target detection if cue and target appeared in close temporal proximity, confirming cross-modal distractor-induced blindness. However, the effect on target detection was reduced compared to the effect of cross-modal distractor-induced deafness previously observed for reversed modalities. The physical features defining cue and target could not account for the diminished distractor effect in the current cross-modal task. Instead, this finding may be attributed to the auditory cue acting as an especially efficient release signal of the distractor-induced inhibition. Additionally, a multisensory enhancement of visual target detection by the concurrent auditory signal might have contributed to the reduced distractor effect.

Visual attention prediction improves performance of autonomous drone racing agents.

Humans race drones faster than neural networks trained for end-to-end autonomous flight. This may be related to the ability of human pilots to select task-relevant visual information effectively. This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural networks' performance for the challenging task of vision-based autonomous drone racing. We hypothesize that gaze-based attention prediction can be an efficient mechanism for visual information selection and decision making in a simulator-based drone racing task. We test this hypothesis using eye gaze and flight trajectory data from 18 human drone pilots to train a visual attention prediction model. We then use this visual attention prediction model to train an end-to-end controller for vision-based autonomous drone racing using imitation learning. We compare the drone racing performance of the attention-prediction controller to those using raw image inputs and image-based abstractions (i.e., feature tracks). Comparing success rates for completing a challenging race track by autonomous flight, our results show that the attention-prediction based controller (88% success rate) outperforms the RGB-image (61% success rate) and feature-tracks (55% success rate) controller baselines. Furthermore, visual attention-prediction and feature-track based models showed better generalization performance than image-based models when evaluated on hold-out reference trajectories. Our results demonstrate that human visual attention prediction improves the performance of autonomous vision-based drone racing agents and provides an essential step towards vision-based, fast, and agile autonomous flight that eventually can reach and even exceed human performances.

Gaze Behavior During Navigation and Visual Search of an Open-World Virtual Environment.

Eye tracking has been an essential tool within the vision science community for many years. However, the majority of studies involving eye-tracking technology employ a relatively passive approach through the use of static imagery, prescribed motion, or video stimuli. This is in contrast to our everyday interaction with the natural world where we navigate our environment while actively seeking and using task-relevant visual information. For this reason, an increasing number of vision researchers are employing virtual environment platforms, which offer interactive, realistic visual environments while maintaining a substantial level of experimental control. Here, we recorded eye movement behavior while subjects freely navigated through a rich, open-world virtual environment. Within this environment, subjects completed a visual search task where they were asked to find and count occurrence of specific targets among numerous distractor items. We assigned each participant into one of four target conditions: Humvees, motorcycles, aircraft, or furniture. Our results show a statistically significant relationship between gaze behavior and target objects across Target Conditions with increased visual attention toward assigned targets. Specifically, we see an increase in the number of fixations and an increase in dwell time on target relative to distractor objects. In addition, we included a divided attention task to investigate how search changed with the addition of a secondary task. With increased cognitive load, subjects slowed their speed, decreased gaze on objects, and increased the number of objects scanned in the environment. Overall, our results confirm previous findings and support that complex virtual environments can be used for active visual search experimentation, maintaining a high level of precision in the quantification of gaze information and visual attention. This study contributes to our understanding of how individuals search for information in a naturalistic (open-world) virtual environment. Likewise, our paradigm provides an intriguing look into the heterogeneity of individual behaviors when completing an un-timed visual search task while actively navigating.

Task-relevant perceptual features can define categories in visual memory too.

Although Konkle, Brady, Alvarez, and Oliva (2010, Journal of Experimental Psychology: General, 139(3), 558) claim that visual long-term memory (VLTM) is organized on underlying conceptual, not perceptual, information, visual memory results from visual search tasks are not well explained by this theory. We hypothesized that when viewing an object, any task-relevant visual information is critical to the organizational structure of VLTM. In two experiments, we examined the organization of VLTM by measuring the amount of retroactive interference created by objects possessing different combinations of task-relevant features. Based on task instructions, only the conceptual category was task relevant or both the conceptual category and a perceptual object feature were task relevant. Findings indicated that when made task relevant, perceptual object feature information, along with conceptual category information, could affect memory organization for objects in VLTM. However, when perceptual object feature information was task irrelevant, it did not contribute to memory organization; instead, memory defaulted to being organized around conceptual category information. These findings support the theory that a task-defined organizational structure is created in VLTM based on the relevance of particular object features and information.

Earlier saccades to task-relevant targets irrespective of relative gain between peripheral and foveal information.

Saccades bring objects of interest onto the fovea for high-acuity processing. Saccades to rewarded targets show shorter latencies that correlate negatively with expected motivational value. Shorter latencies are also observed when the saccade target is relevant for a perceptual discrimination task. Here we tested whether saccade preparation is equally influenced by informational value as it is by motivational value. We defined informational value as the probability that information is task-relevant times the ratio between postsaccadic foveal and presaccadic peripheral discriminability. Using a gaze-contingent display, we independently manipulated peripheral and foveal discriminability of the saccade target. Latencies of saccades with perceptual task were reduced by 36 ms in general, but they were not modulated by the information saccades provide (Experiments 1 and 2). However, latencies showed a clear negative linear correlation with the probability that the target is task-relevant (Experiment 3). We replicated that the facilitation by a perceptual task is spatially specific and not due to generally heightened arousal (Experiment 4). Finally, the facilitation only emerged when the perceptual task is in the visual but not in the auditory modality (Experiment 5). Taken together, these results suggest that saccade latencies are not equally modulated by informational value as by motivational value. The facilitation by a perceptual task only arises when task-relevant visual information is foveated, irrespective of whether the foveation is useful or not.

Moving to Capture Children's Attention: Developing a Methodology for Measuring Visuomotor Attention.

Attention underpins many activities integral to a child’s development. However, methodological limitations currently make large-scale assessment of children’s attentional skill impractical, costly and lacking in ecological validity. Consequently we developed a measure of ‘Visual Motor Attention’ (VMA)—a construct defined as the ability to sustain and adapt visuomotor behaviour in response to task-relevant visual information. In a series of experiments, we evaluated the capability of our method to measure attentional processes and their contributions in guiding visuomotor behaviour. Experiment 1 established the method’s core features (ability to track stimuli moving on a tablet-computer screen with a hand-held stylus) and demonstrated its sensitivity to principled manipulations in adults’ attentional load. Experiment 2 standardised a format suitable for use with children and showed construct validity by capturing developmental changes in executive attention processes. Experiment 3 tested the hypothesis that children with and without coordination difficulties would show qualitatively different response patterns, finding an interaction between the cognitive and motor factors underpinning responses. Experiment 4 identified associations between VMA performance and existing standardised attention assessments and thereby confirmed convergent validity. These results establish a novel approach to measuring childhood attention that can produce meaningful functional assessments that capture how attention operates in an ecologically valid context (i.e. attention's specific contribution to visuomanual action).

An exploratory analysis of variations in quiet eye duration within and between levels of expertise

The ability to pick-up task-relevant visual information during movement control is crucial in successful sport performance. Quiet eye (QE), the final fixation prior to final movement onset, has been shown to be characteristic of the visual search strategies exhibited by skilled athletes in self-paced aiming tasks. Longer QE durations were previously associated with skill and successful performance outcomes. In this study, gaze behaviour data of six expert (E) and six novice (N) 10-pin bowlers were measured using a mobile eye tracker as they completed 20 trials of two single-pin conditions each (Easy: 1-pin; Hard: 10-pin). Expert bowlers exhibited significantly longer QE durations in both conditions as compared to their less-skilled counterparts. However, QE duration was not found to be significantly different as a function of accuracy nor task condition. Further detailed analysis revealed considerable variance in QE characteristics between individuals, warranting the need to explore individualised interventions centred on the development of perceptual-motor control during self-paced aiming tasks. Moreover, this study raised an important methodological issue relating to the analysis of trials with the absence of QE.

Retinotopic patterns of background connectivity between V1 and fronto-parietal cortex are modulated by task demands.

Attention facilitates the processing of task-relevant visual information and suppresses interference from task-irrelevant information. Modulations of neural activity in visual cortex depend on attention, and likely result from signals originating in fronto-parietal and cingulo-opercular regions of cortex. Here, we tested the hypothesis that attentional facilitation of visual processing is accomplished in part by changes in how brain networks involved in attentional control interact with sectors of V1 that represent different retinal eccentricities. We measured the strength of background connectivity between fronto-parietal and cingulo-opercular regions with different eccentricity sectors in V1 using functional MRI data that were collected while participants performed tasks involving attention to either a centrally presented visual stimulus or a simultaneously presented auditory stimulus. We found that when the visual stimulus was attended, background connectivity between V1 and the left frontal eye fields (FEF), left intraparietal sulcus (IPS), and right IPS varied strongly across different eccentricity sectors in V1 so that foveal sectors were more strongly connected than peripheral sectors. This retinotopic gradient was weaker when the visual stimulus was ignored, indicating that it was driven by attentional effects. Greater task-driven differences between foveal and peripheral sectors in background connectivity to these regions were associated with better performance on the visual task and faster response times on correct trials. These findings are consistent with the notion that attention drives the configuration of task-specific functional pathways that enable the prioritized processing of task-relevant visual information, and show that the prioritization of visual information by attentional processes may be encoded in the retinotopic gradient of connectivty between V1 and fronto-parietal regions.

Searching for targets in visual working memory: investigating a dimensional feature bundle (DFB) model.

The human visual working memory (WM) system enables us to store a limited amount of task-relevant visual information temporally in mind. One actively debated issue in cognitive neuroscience centers around the question of how this WM information is maintained. The currently dominant views advocated by prominent WM models hold that the units of memory are configured either as independent feature representations, integrated bound objects, or a combination of both. Here, we approached this issue by measuring lateralized brain electrical activity during a retro-cue paradigm, in order to track people's ability to access WM representations as a function of the dimensional relation between WM items and task settings. Both factors were revealed to selectively influence WM access: whereas cross relative to intradimensional WM targets gave rise to enhanced contralateral delay activity (CDA) amplitudes, localization relative to identification task demands yielded speeded CDA and manual response times. As these dimension-based findings are not reconcilable with contemporary feature- and/or object-based accounts, an alternative view that is based on the hierarchical feature-bundle model is proposed. We argue that WM units may consist of three hierarchically structured levels of representations, with an intermediate dimensionally organized level that mediates between top-level object and lower-level feature representations.

Age-related differences in processing visual device and task characteristics when using technical devices

With aging visual feedback becomes increasingly relevant in action control. Consequently, visual device and task characteristics should more and more affect tool use. Focussing on late working age, the present study aims to investigate age-related differences in processing task irrelevant (display size) and task relevant visual information (task difficulty). Young and middle-aged participants (20–35 and 36–64 years of age, respectively) sat in front of a touch screen with differently sized active touch areas (4″ to 12″) and performed pointing tasks with differing task difficulties (1.8–5 bits). Both display size and age affected pointing performance, but the two variables did not interact and aiming duration moderated both effects. Furthermore, task difficulty affected the pointing durations of middle-aged adults moreso than those of young adults. Again, aiming duration accounted for the variance in the data. The onset of an age-related decline in aiming duration can be clearly located in middle adulthood. Thus, the fine psychomotor ability “aiming” is a moderator and predictor for age-related differences in pointing tasks. The results support a user-specific design for small technical devices with touch interfaces.

Predicting human visuomotor behaviour in a driving task

The sequential deployment of gaze to regions of interest is an integral part of human visual function. Owing to its central importance, decades of research have focused on predicting gaze locations, but there has been relatively little formal attempt to predict the temporal aspects of gaze deployment in natural multi-tasking situations. We approach this problem by decomposing complex visual behaviour into individual task modules that require independent sources of visual information for control, in order to model human gaze deployment on different task-relevant objects. We introduce a softmax barrier model for gaze selection that uses two key elements: a priority parameter that represents task importance per module, and noise estimates that allow modules to represent uncertainty about the state of task-relevant visual information. Comparisons with human gaze data gathered in a virtual driving environment show that the model closely approximates human performance.

Neural Representation of Targets and Distractors during Object Individuation and Identification

In many everyday activities, we need to attend and encode multiple target objects among distractor objects. For example, when driving a car on a busy street, we need to simultaneously attend objects such as traffic signs, pedestrians, and other cars, while ignoring colorful and flashing objects in display windows. To explain how multiple visual objects are selected and encoded in visual STM and in perception in general, the neural object file theory argues that, whereas object selection and individuation is supported by inferior intraparietal sulcus (IPS), the encoding of detailed object features that enables object identification is mediated by superior IPS and higher visual areas such as the lateral occipital complex (LOC). Nevertheless, because task-irrelevant distractor objects were never present in previous studies, it is unclear how distractor objects would impact neural responses related to target object individuation and identification. To address this question, in two fMRI experiments, we asked participants to encode target object shapes among distractor object shapes, with targets and distractors shown in different spatial locations and in different colors. We found that distractor-related neural processing only occurred at low, but not at high, target encoding load and impacted both target individuation in inferior IPS and target identification in superior IPS and LOC. However, such distractor-related neural processing was short-lived, as it was only present during the visual STM encoding but not the delay period. Moreover, with spatial cuing of target locations in advance, distractor processing was attenuated during target encoding in superior IPS. These results are consistent with the load theory of visual information processing. They also show that, whereas inferior IPS and LOC were automatically engaged in distractor processing under low task load, with the help of precuing, superior IPS was able to only encode the task-relevant visual information.

Looking for Discriminating Is Different from Looking for Looking’s Sake

Recent studies provide evidence for task-specific influences on saccadic eye movements. For instance, saccades exhibit higher peak velocity when the task requires coordinating eye and hand movements. The current study shows that the need to process task-relevant visual information at the saccade endpoint can be, in itself, sufficient to cause such effects. In this study, participants performed a visual discrimination task which required a saccade for successful completion. We compared the characteristics of these task-related saccades to those of classical target-elicited saccades, which required participants to fixate a visual target without performing a discrimination task. The results show that task-related saccades are faster and initiated earlier than target-elicited saccades. Differences between both saccade types are also noted in their saccade reaction time distributions and their main sequences, i.e., the relationship between saccade velocity, duration, and amplitude.

Shoot or don't shoot? Why police officers are more inclined to shoot when they are anxious.

We investigated the effect of anxiety on police officers' shooting decisions. Thirty-six police officers participated and executed a low- and high-anxiety video-based test that required them to shoot or not shoot at rapidly appearing suspects that either had a gun and "shot," or had no gun and "surrendered." Anxiety was manipulated by turning on (high anxiety) or turning off (low anxiety) a so-called "shootback canon" that could fire small plastic bullets at the participants. When performing under anxiety, police officers showed a response bias toward shooting, implying that they accidentally shot more often at suspects that surrendered. Furthermore, shot accuracy was lower under anxiety and officers responded faster when suspects had a gun. Finally, because gaze behavior appeared to be unaffected by anxiety, it is concluded that when they were anxious, officers were more inclined to respond on the basis of threat-related inferences and expectations rather than objective, task-relevant visual information.

The effect of social context on the use of visual information

Social context modulates action kinematics. Less is known about whether social context also affects the use of task relevant visual information. We tested this hypothesis by examining whether the instruction to play table tennis competitively or cooperatively affected the kind of visual cues necessary for successful table tennis performance. In two experiments, participants played table tennis in a dark room with only the ball, net, and table visible. Visual information about both players’ actions was manipulated by means of self-glowing markers. We recorded the number of successful passes for each player individually. The results showed that participants’ performance increased when their own body was rendered visible in both the cooperative and the competitive condition. However, social context modulated the importance of different sources of visual information about the other player. In the cooperative condition, seeing the other player’s racket had the largest effects on performance increase, whereas in the competitive condition, seeing the other player’s body resulted in the largest performance increase. These results suggest that social context selectively modulates the use of visual information about others’ actions in social interactions.

The role of vision in detecting and correcting fingertip force errors during object lifting

Vision provides many reliable cues about the likely weight of an object, allowing individuals to predict how heavy it will be. The forces used to lift an object for the first time reflect these predictions. This, however, leads to inevitable errors during lifts of objects that weigh unexpected amounts. Fortunately, these errors are rarely made twice in a row-lifters have the impressive ability to detect and correct large or small misapplications of fingertip forces, even while experiencing weight illusions. Although it has been assumed that we detect and correct these errors exclusively with our sense of touch, recent evidence has demonstrated a role for vision in this fingertip force scaling. Here, we demonstrate that even when stimulus set size, delay, and modality are controlled for, individuals are unable to skillfully scale their grip and load force rates over repeated lifts without vision. However, eliminating only the task-relevant visual information, while maintaining the rest of the visual world, shifts participants back into the normal, skilled mode of control. These findings clarify the role of visual information in the ostensibly haptic task of lifting objects, suggesting individuals use priors under conditions where uncertainty is high.

Visual information interacts with neuromuscular factors in the coordination of bimanual isometric force

The role of perceptual-motor processes in the coordination and control of movement is a long standing issue. Nevertheless, there is no coherence on theoretical perspectives with their being frameworks that emphasize perceptual, motor and perceptual-motor processes in coordination and control. The purpose of this study was to examine the interactive effects of visual information and factors of neuromuscular organization (force level, force direction, and homologous muscle pairs) on coordination patterns in bimanual isometric force production. In Experiment 1, the participants were required to abduct two index fingers isometrically and produce simultaneous forces such that their sum matched the constant force target specified at two force levels (10 and 40% of maximum voluntary contraction (MVC)). Visual information of the force outputs was either present or absent between conditions. The results showed that the coordination patterns interact with visual feedback in that the two finger forces exhibit negative correlation with vision and positive correlation without vision, with stronger correlation in each case found at higher force levels. In Experiment 2, the force direction and muscles involved in the task were different between the hands. In comparison with Experiment 1, the negative correlation was stronger with vision at 40% MVC (but equal at 10% MVC), and positive correlation was weaker without vision at 10% MVC (but equal at 40% MVC). The findings provide further evidence that the coordination patterns in bimanual isometric force production are specified by the interaction of task-relevant visual information and force level and, to a lesser degree by force direction and the muscles involved in the task. The capacity to exploit information mediates coordination and control, and the effective utilization of information is dependent on the specific action.

P.1.f.010 Event-related potentials and secondary task performance during simulated driving performance

Unexpected changes in task-irrelevant auditory stimuli are capable to distract processing of task-relevant visual information. This effect is accompanied by the elicitation of event-related potential (ERP) components associated with attentional orientation, i.e. P3a and reorienting negativity (RON). In the present study we varied the demands of a visual task in order to test whether the RON component – as an index of attentional reorientation after distraction – is confined to a semantic task requiring working memory. In two ERP experiments we applied an auditory-visual distraction paradigm in which subjects were instructed to discriminate visual stimuli preceded by a task-irrelevant sound, this being either a standard tone (600 Hz, 88%) or a deviant tone (660 Hz, 12%). The visual stimuli were numbers which had to be judged on basis of a semantic (odd or even) or physical feature (either size or colour). As expected, deviance related ERP components namely the mismatch negativity (MMN), P3a, and RON were elicited. Importantly, the RON was affected by the variation of the task: within the semantic task an early RON and within the physical task a late RON was obtained. These results suggest that the RON component reflects two functionally distinct processes of attentional allocation after distraction: refocusing on task-relevant information on the working memory level, and general reorientation of attention, e.g. preparation for the upcoming task.