Visual Conjunction Search Research Articles

Deficits in visual search for conjunctions of motion and form after parietal damage but with spared hMT+/V5

It has been argued that area hMT+/V5 in humans acts as a motion filter, enabling targets defined by a conjunction of motion and form to be efficiently selected. We present data indicating that (a) damage to parietal cortex leads to a selective problem in processing motion–form conjunctions, and (b) that the presence of a structurally and functional intact hMT+/V5 is not sufficient for efficient search for motion–form conjunctions. We suggest that, in addition to motion-processing areas (e.g., hMT+/V5), the posterior parietal cortex is necessary for efficient search with motion–form conjunctions, so that damage to either brain region may bring about deficits in search. We discuss the results in terms of the involvement of the posterior parietal cortex in the top-down guidance of search or in the binding of motion and form information.

Eye movements in active visual search: A computable phenomenological model

We present a computational model and corresponding computer simulations that mimic phenomenologically the eye movement trajectories observed in a conjunctive visual search task. The element of randomness is captured in the model through a Monte Carlo selection of a particular eye movement based on its probability, which depends on three factors, adjusted to match to the observed saccade amplitude distribution, forward bias in consecutive saccades, and return rates. Memory is assumed to operate through tagging of objects already recognized as nontargets, which, in turn, requires their processing within the attentional area of conspicuity (AC). That AC is adjusted so that computer simulations optimally reproduce the distribution of the number of saccades, the failure rate for capturing the target, and the return rate to previously inspected locations. For their viability, computer simulations critically depend on memory's being long-ranged. In turn, the simulations confirm the formation of circulating or spiraling patterns in the observed eye trajectories. We also relate consistently the average number of saccades per trial to the saccade amplitude distribution by modeling analytically the combined roles of the AC in attention and memory. The full Supplemental Appendix A for this article may be downloaded from http://app.psychonomic-journals.org/content/supplemental.

Eye movement trajectories in active visual search: Contributions of attention, memory, and scene boundaries to pattern formation

We relate the roles of attention, memory, and spatial constraints to pattern formation in eye movement trajectories previously measured in a conjunctive visual search task. Autocorrelations and power spectra of saccade direction cosines confirm a bias to progress forwardly, while turning at the display boundaries, plus a long-range memory component for the search path. Analyses of certain measures of circulation and imbalance in the eye trajectories, and their relations with the display area correspondingly subtended, bear signatures of spiraling or circulating patterns. We interpret their prevalence as mainly due to the interactions between three basic psychoneural mechanisms (conspicuity area, forward bias, long-range memory) and two task-specific geometric-spatial constraints on the eye trajectories (central start and display confinement). Conversely, computer simulations of random walks in which all psychoneural mechanisms are eliminated, while geometric-spatial constraints are maintained, show no prevalence of circulating patterns by those measures. We did find certain peculiarities of some individual participants in their pattern selections, but they appear too casual and incidental to suggest more systematic or complex search strategies in our randomized displays of uninformative stimuli.

Contribution of frontal cortex to the spatial representation of number

Neuropsychological and neurophysiological studies have suggested that prefrontal cortex may be involved in non-verbal number processing, when relevant for current behavioural goals. More precisely, it has been suggested that an intact right inferior frontal gyrus (rIFG) in humans may be necessary to the use of a spatial representation of numbers, also known as mental number line. In a popular model of spatial functions (e.g., Corbetta et al., 2008), rIFG is part of a right-lateralised ventral fronto-parietal network that conveys signals to a dorsal network supporting attentional orienting in contralateral space. Within the dorsal network, the frontal eye fields (FEF) are known to contribute to visual scene analysis and visual conjunction search tasks when eye movement commands are not required. In the present study, we hypothesised they might also be involved in exploring a conceptual space, such as the mental number line. We examined the proposed functions of the human rIFG and right Frontal Eye Field (rFEF) by interfering with their normal functioning with repetitive Transcranial Magnetic Stimulation (rTMS) while participants performed numerical tasks. The results suggest that, when number magnitude is relevant to the task, rIFG supports orienting to the entire mental number line while rFEF are crucial for contralateral orienting (that is towards small numbers).

Assessment of spatial neglect using computerised feature and conjunction visual search tasks

The purpose of this study was to assess the diagnostic sensitivity of tasks employing feature and conjunction visual searches in stroke patients with unilateral spatial neglect (USN). Seventy-two stroke patients (right/left hemispheric damage with/without USN) and 39 healthy controls participated in the study. Hit rate and reaction time measures of feature and conjunction searches were tested using a newly developed computerised programme for the assessment of visual spatial attention (VISSTA). In addition, subjects received a set of diagnostic paper-and-pencil tests, and were also assessed for the impact of neglect on activities of daily living. Results indicated that the computerised test clearly differentiated between stroke patients and healthy controls, and between the different patient groups. USN patients showed significant contralesional disadvantage in both feature and conjunction visual search tasks. It is proposed that computerised assessment of visual search capacity is a useful and sensitive adjunct to standard paper-and-pencil tests of USN, with the advantage of testing responses based on attention shifts under a time constraint. The learning effects that limit the usefulness of paper-and-pencil tests in longitudinal studies are less likely to affect a computerised test, making it more suitable for monitoring treatment-induced or natural recovery by way of repeated testing.

Object-based biasing for attentional control of gaze: a comparison of biologically plausible mechanisms

INTRODUCTION In the visual system, attending to important objects in the visual field relies on the transfer of top-down, object-based task information to the spatially organised areas of cortex. How this occurs and the method by which this information can influence the dorsal stream and redirect gaze are not well understood. Current models of the ventral stream mostly focus on the feed-forward mechanisms involved and current feedback models do not seem to address the issue of object-space binding in a comprehensive and plausible manner. METHODS We investigated these questions using the following modeling framework. A bidirectional, ventral stream object recognition hierarchy up to anterior inferior temporal cortex (AIT) from primary visual cortex (V1) and a model of dorsal stream to frontal eye fields (FEF) with our previously developed oculomotor system [1]. Selection is performed in both the object-based mapping of AIT [2] and the spatial mapping of FEF [3] by basal ganglia loops [4]. Modeling of the ventral stream consists of a hierarchy of increasingly spatially invariant cortical areas linked by both feed-forward excitatory and feedback connections. Within each receptive field, there is a competition to represent the strongest and thus most likely representation for that region, which can be biased by the feedback from higher visual areas. Three models of feedback attention mechanism were tested: additive feedback, shunting (multiplicative) feedback and a shunt of feedback by feed-forward. The model was tested using a simple visual world (colored flags) that nevertheless challenged all the main competencies being investigated. Performance was measured by (i) eliciting saccadic behavior in simulated visual search with different numbers of distractors, and (ii) target segmentation in cluttered scenes within a fixed time window. RESULTS In the target segmentation task, the additive feedback model consistently fails to bind the AIT representation of the object to the correct location on the visual field. The shunting model was able to segment 58% of scenes while the gating model was most successful (83%) (Figure 1). We then took the most successful (gating) model and challenged it with a conjunction visual search task. Here, by simulating models trained and naive to the target stimulus, we showed that subsequent learning of a combined representation of an untrained target stimulus can explain the experimentally observed decrease in the slope of reaction time against number of distractors for that target (Figure 2) [5].

Human dorsolateral prefrontal cortex is involved in visual search for conjunctions but not features: A theta TMS study

Functional neuroimaging studies have shown that the detection of a target defined by more than one feature (for example, a conjunction of colour and orientation) amongst distractors is associated with the activation of a network of brain areas. Dorsolateral prefrontal cortex (DLPFC), along with areas such as the frontal eye fields (FEF) and posterior parietal cortex (PPC), is a component of this network. While transcranial magnetic stimulation (TMS) had shown that both FEF and PPC are necessary for, and not just correlated with, successful conjunction search, this is not the case for DLPFC. To test the hypothesis that this area is also necessary for efficient conjunction search, TMS was applied over DLPFC and the effects on conjunction and feature (in this case colour) search performance compared with those when TMS was delivered over area MT/V5 and a vertex control stimulation condition. DLPFC TMS impaired performance on the conjunction search task but was without effect on feature search, similar to findings when TMS is delivered over PPC or FEF. Vertex TMS had no effects whereas MT/V5 TMS significantly improved performance with a time course that may indicate that this was due to modulation of V4 activity. These findings illustrate that, like FEF and PPC, DLPFC is necessary for fully effective conjunction visual search performance.

The timing of the involvement of the frontal eye fields and posterior parietal cortex in visual search

The frontal eye fields (FEFs) and posterior parietal cortex (PPC) are important for target detection in conjunction visual search but the relative timings of their contribution have not been compared directly. We addressed this using temporally specific double pulse transcranial magnetic stimulation delivered at different times over FEFs and PPC during performance of a visual search task. Disruption of performance was earlier (0/40 ms) with FEF stimulation than with PPC stimulation (120/160 ms), revealing a clear and substantial temporal dissociation of the involvement of these two areas in conjunction visual search. We discuss these timings with reference to the respective roles of FEF and PPC in the modulation of extrastriate visual areas and selection of responses.

Is posterior parietal cortical involvement in visual search contingent upon frontal eye field activity? A dual site TMS study

The decrease in reaction time when target location in successive trials in visual search is an established effect. Using transcranial magnetic stimulation (TMS) we have previously shown that repetition of target location in a colour/orientation conjunction visual search task is also commensurate with a decrease in involvement of right posterior parietal cortex (rPPC) an area known to be involved in the processing of this task. The decrease in reaction time seems to be mediated by a concomitant lack of engagement of rPPC.

Visual selective attention among persons with schizophrenia: The distractor ratio effect

The current study investigated whether impaired visual attention among patients with schizophrenia can be accounted for by poor perceptual organization and impaired search selectivity. Twenty-three patients with schizophrenia and 22 healthy control participants completed a conjunctive visual search task where the relative frequency of the two types of distractors was manipulated. It has been shown that, when the total number of items in a display is fixed, search performance depends on the relative frequency of the types of distractors (i.e., as the ratio becomes more discrepant search time decreases). This modulation of search efficiency reflects participants' ability to group items by their perceptual similarity and then search only the smaller group of items that share a feature with the target. Results show that patients modulate their response time normally as a function of the distractor ratio – that is, they benefit from the presence of a smaller distractor subset in the display. This suggests that patients with schizophrenia, group items according to their perceptual similarity and flexibly deploy their attention to the smaller subset of distractors on each trial. These results demonstrate that search selectivity as a function of the relative frequency of distractors is unimpaired among patients with schizophrenia.

Spatial and Temporal Dynamics of Attentional Guidance during Inefficient Visual Search

Spotting a prey or a predator is crucial in the natural environment and relies on the ability to extract quickly pertinent visual information. The experimental counterpart of this behavior is visual search (VS) where subjects have to identify a target amongst several distractors. In difficult VS tasks, it has been found that the reaction time (RT) is influenced by salience factors, such as the target-distractor similarity, and this finding is usually regarded as evidence for a guidance of attention by preattentive mechanisms. However, the use of RT measurements, a parameter which depends on multiple factors, allows only very indirect inferences about the underlying attentional mechanisms. The purpose of the present study was to determine the influence of salience factors on attentional guidance during VS, by measuring directly attentional allocation. We studied attention allocation by using a dual covert VS task in subjects who had 1) to detect a target amongst different items and 2) to report letters briefly flashed inside those items at different delays. As predicted, we showed that parallel processes guide attention towards the most relevant item by virtue of both goal-directed and stimulus-driven factors, and we demonstrated that this attentional selection is a prerequisite for target detection. In addition, we show that when the target is characterized by two features (conjunction VS), the goal-directed effects of both features are initially combined into a unique salience value, but at a later stage, grouping phenomena interact with the salience computation, and lead to the selection of a whole group of items. These results, in line with Guided Search Theory, show that efficient and rapid preattentive processes guide attention towards the most salient item, allowing to reduce the number of attentional shifts needed to find the target.

The role of the angular gyrus in visual conjunction search investigated using signal detection analysis and transcranial magnetic stimulation

Transcranial magnetic stimulation (TMS) delivered over the posterior parietal cortex increases choice reaction times in visual search for a target defined by a conjunction of features. Some recent studies of visual search have taken an approach based on signal detection theory, the findings of which are not addressed by studying the disruptive effects of TMS on reaction time. Here we investigated the role of the posterior parietal cortex in visual search by applying TMS while subjects performed unspeeded feature and conjunction visual search tasks matched for level of difficulty. TMS over the right, but not the left angular gyrus (AG) in the parietal cortex, nor vertex decreased subjects’ sensitivity on the conjunction but not the feature search task, as measured by the signal detection measure, d′. Changes in bias, specifically the tendency to make false positive responses, were less clear. We consider the findings in terms of four possible explanation: binding, attentional control, spatial localisation and visuomotor co-ordinate transformations.

Neuronal activity in superior colliculus signals both stimulus identity and saccade goals during visual conjunction search

Although we know that the process of saccade target selection is reflected in the activity of sensory-motor neurons within saccade executive centers, the description of this process at the neural level has yet to fully account for all selection outcomes. The current study sought to determine how neuronal activity in the intermediate layers of the superior colliculus (SC) determines correct saccade target selection by examining the activity of visuomovement neurons during both correct and error trials of monkeys performing a relatively difficult visual conjunction search task. We found that a stimulus presented in a neuron's response field, but not foveated, was associated with greater activity if it was the search target instead of a distractor, indicating that SC neurons could represent stimulus identity. Nevertheless, activity was greater when a saccade was made to a stimulus than when it was not, further implicating these neurons in selecting the saccade goal. Together with the related observation that, when the target fell in their response fields, SC neurons discharged significantly more if the monkey correctly selected it instead of a distractor, these results suggest that visual stimuli are selected when these neurons reach a critical activation level. Our findings show that the outcome of all visual search trials, regardless of the stimulus being selected, is predicted by SC neuronal activity.

Control of Visual Selection during Visual Search in the Human Brain

How do we find a target object in a cluttered visual scene? Targets carrying unique salient features can be found in parallel without directing attention, whereas targets defined by feature conjunctions or non-salient features need to be scrutinized in a serial attentional process in order to be identified. In this article, we review a series of experiments in which we used fMRI to probe the neural basis of this active search process in the human brain. In all experiments, we compared the fMRI signal between a difficult and an easy visual search (each performed without eye movements) in order to isolate neural activity reflecting the search process from other components such as stimulus responses and movement-related activity. The difficult search was either a conjunction search or a hard feature search and compared with an easy feature search, matched in visual stimulation and motor requirements. During both, the conjunction search and the hard feature search the frontal eye fields (FEF) and three parietal regions located in the intraparietal sulcus (IPS) were differentially activated: the anterior and posterior part of the intraparietal sulcus (AIPS, PIPS) as well as the junction of the intraparietal with the transverse occipital sulcus (IPTO). Only in PIPS, the modulation strength was most indistinguishable between conjunction and hard feature search. In a further experiment we showed that AIPS and IPTO are involved in visual conjunction search even in the absence of distractors; by contrast, the involvement of PIPS seems to depend on the presence of distractors. Taken together, these findings from these experiments demonstrate that all four key nodes of the human ’frontoparietal attention network’ are generally engaged in the covert selection process of visual search. But they also suggest that these areas play differential roles, perhaps reflecting different sub-processes in active search. We conclude by discussing a number of such sub-processes, such as the direction of spatial attention, visual feature binding, and the active suppression of distractors.

Repetition of distractor sets improves visual search performance in hemispatial neglect

Priming from repeated distractor sets, or search context, in conjunctive visual search was examined in four patients with hemispatial neglect. In the first experiment overall context was either changed or repeated while the target was always the same to control for any modulatory effect of target priming. Considerable priming was seen from repeated context. In the second experiment the context was either repeated on the left side, on the right side, on both sides, or the context was new. Priming from repeated context was found to arise from the left visual field, as well as the right visual field, as well as when overall context was repeated. Brief masked displays were used in experiment 3, the results again showing strong priming from repeated overall context. The results of the three experiments suggest that visual grouping, or perceptual organization, of distractor sets is relatively intact in the affected hemifield of parietal neglect patients. Furthermore, repetition of context may even temporarily ameliorate neglect symptoms in search. These findings are consistent with claims that grouping is distinct from attentional processing and that it operates at lower levels of the perceptual hierarchy.

Postural costs of performing cognitive tasks in non-coincident reference frames

Dual-task interactions in posture and cognitive tasks have been explained as a competition for spatial processing structures or as interference in the online sensorimotor adjustments required for sensory integration. Going beyond these general terms accounts, we propose that interference between spatial and temporal operations in posture-cognition arises at least partly from the need to share a common behavioral context, such as a spatial frame of reference. Using immersive visualization and motion-tracking techniques, we manipulated the spatial reference frames for a standing task and a conjunction visual search task into or out of coincidence. Aside from performance trade-offs due to task-load manipulations, performing visual search in a non-coincident reference frame led to cognitive task and postural task performance decrements (Experiment 1). Postural dual-task decrements were also observed when visual search was split between coincident and non-coincident frames and both frame conditions rendered identical in visual information relevant to posture control (Experiment 2). We concluded that the postural control costs observed for posture-cognition dual-tasking may in part reflect costs of keeping tasks' reference frames in register.

Bottom-up guidance in visual search for conjunctions.

Understanding the relative role of top-down and bottom-up guidance is crucial for models of visual search. Previous studies have addressed the role of top-down and bottom-up processes in search for a conjunction of features but with inconsistent results. Here, the author used an attentional capture method to address the role of top-down and bottom-up processes in conjunction search. The role of bottom-up processing was assayed by inclusion of an irrelevant-size singleton in a search for a conjunction of color and orientation. One object was uniquely larger on each trial, with chance probability of coinciding with the target; thus, the irrelevant feature of size was not predictive of the target's location. Participants searched more efficiently for the target when it was also the size singleton, and they searched less efficiently for the target when a nontarget was the size singleton. Although a conjunction target cannot be detected on the basis of bottom-up processing alone, participants used search strategies that relied significantly on bottom-up guidance in finding the target, resulting in interference from the irrelevant-size singleton.

Conjunctive Visual Search in Individuals With and Without Mental Retardation

A comprehensive understanding of the basic visual and cognitive abilities of individuals with mental retardation is critical for understanding the basis of mental retardation and for the design of remediation programs. We assessed visual search abilities in individuals with mild mental retardation and in MA- and CA-matched comparison groups. Our goal was to determine the effect of decreasing target-distracter disparities on visual search efficiency. Results showed that search rates for the group with mental retardation and the MA-matched comparisons were more negatively affected by decreasing disparities than were those of the CA-matched group. The group with mental retardation and the MA-matched group performed similarly on all tasks. Implications for theory and application are discussed.

Cross-trial priming in visual search for singleton conjunction targets: Role of repeated target and distractor features

Kristjánsson, Wang, and Nakayama (2002) demonstrated that visual search for conjunctively defined targets can be substantially expedited ("primed") when target and distractor features are repeated on consecutive trials. Two experiments were conducted to examine whether the search response time (RT) facilitation on target-present trials results from repetition of target-defining features, distractor features, or both. The experiments used a multiple conjunctive search paradigm (adapted from Kristjánsson et al., 2002), in which the target and distractor features were varied (i.e., repeated) independently of each other across successive trials. The RT facilitation was numerically largest when both target and distractor features were repeated, but not significantly larger than that when only distractor features were repeated. This indicates that cross-trial priming effects in conjunctive visual search result mainly from the repetition of distractor, rather than target, features.

Guidance of Eye Movements During Visual Conjunction Search: Local and Global Contextual Effects on Target Discriminability

The composition of a visual scene influences the ability of humans to select specific details within that scene for discrimination or foveation with saccadic eye movements. With the goal of establishing an animal model to investigate the neural mechanisms underlying the deployment of visual attention and the guidance of saccades during visual search, we studied the visual behavior of three monkeys while they performed a conjunction (color + form) search task similar to those used in human studies. We found that search performance declined when distractors adjacent to the target shared its color, thereby revealing that color was more discriminable than form in these displays and suggesting that monkeys perceptually grouped stimuli by proximity and similarity. Search performance also varied with the overall composition of the display. Most importantly, saccades were biased toward distractors sharing the target color when there were few of them within the display and away from those distractors when they were numerous. Last, the monkeys initiated saccades with a fixed latency, suggesting that their responses to the display were automatic and that search strategies did not involve attentional resources beyond those recruited for regulating saccades. We conclude that monkeys adapt their visual strategies, largely via bottom-up processes, to both the local and the global context of the search. These findings suggest that the visual behavior of monkeys is guided by strategies similar to those observed in humans.