Object Feature Binding Research Articles

Shifting expectations: Lapses in spatial attention are driven by anticipatory attentional shifts.

Attention is dynamic, constantly shifting between different locations - sometimes imperfectly. How do goal-driven expectations impact dynamic spatial attention? A previous study (Dowd & Golomb, Psychological Science, 30(3), 343-361, 2019) explored object-feature binding when covert attention needed to be either maintained at a single location or shifted from one location to another. In addition to revealing feature-binding errors during dynamic shifts of attention, this study unexpectedly found that participants sometimes made correlated errors on trials when they did not have to shift attention, mistakenly reporting the features and location of an object at a different location. The authors posited that these errors represent "spatial lapses" attention, which are perhaps driven by the implicit sampling of other locations in anticipation of having to shift attention. To investigate whether these spatial lapses are indeed anticipatory, we conducted a series of four experiments. We first replicated in Psychological Science, 30(3), the original finding of spatial lapses, and then showed that these spatial lapses were not observed in contexts where participants are not expecting to have to shift attention. We then tested contexts where the direction of attentional shifts was spatially predictable, and found that participants lapse preferentially to more likely shift locations. Finally, we found that spatial lapses do not seem to be driven by explicit knowledge of likely shift locations. Combined, these results suggest that spatial lapses of attention are induced by the implicit anticipation of making an attentional shift, providing further insight into the interplay between implicit expectations, dynamic spatial attention, and visual perception.

Combined expectancies: the role of expectations for the coding of salient bottom-up signals

The visual system forms predictions about upcoming visual features based on previous visual experiences. Such predictions impact on current perception, so that expected stimuli can be detected faster and with higher accuracy. A key question is how these predictions are formed and on which levels of processing they arise. Particularly, predictions could be formed on early levels of processing, where visual features are represented separately, or might require higher levels of processing, with predictions formed based on full object representations that involve combinations of visual features. In four experiments, the present study investigated whether the visual system forms joint prediction errors or whether expectations about different visual features such as color and orientation are formed independently. The first experiment revealed that task-irrelevant and implicitly learned expectations were formed independently when the features were separately bound to different objects. In a second experiment, no evidence for a mutual influence of both types of task-irrelevant and implicitly formed feature expectations was observed, although both visual features were assigned to the same objects. A third experiment confirmed the findings of the previous experiments for explicitly rather than implicitly formed expectations. Finally, no evidence for a mutual influence of different feature expectations was observed when features were assigned to a single centrally presented object. Overall, the present results do not support the view that object feature binding generates joint feature-based expectancies of different object features. Rather, the results suggest that expectations for color and orientation are processed and resolved independently at the feature level.

The Binding Problem after an eye movement.

Spatial attention is thought to be the "glue" that binds features together (e.g., Treisman & Gelade, 1980, Psychology, 12[1], 97-136)-but attention is dynamic, constantly moving across multiple goals and locations. For example, when a person moves her eyes, visual inputs that are coded relative to the eyes (retinotopic) must be rapidly updated to maintain stable world-centered (spatiotopic) representations. Here, we examined how dynamic updating of spatial attention after a saccadic eye movement affects object-feature binding. Immediately after a saccade, participants were simultaneously presented with four colored and oriented bars (one at a precued spatiotopic target location) and instructed to reproduce both the color and orientation of the target item. Object-feature binding was assessed by applying probabilistic mixture models to the joint distribution of feature errors: feature reports for the target item could be correlated (and thus bound together) or independent. We found that compared with holding attention without an eye movement, attentional updating after an eye movement produced more independent errors, including illusory conjunctions, in which one feature of the item at the spatiotopic target location was misbound with the other feature of the item at the initial retinotopic location. These findings suggest that even when only one spatiotopic location is task relevant, spatial attention-and thus object-feature binding-is malleable across and after eye movements, heightening the challenge that eye movements pose for the binding problem and for visual stability.

Object-Feature Binding Survives Dynamic Shifts of Spatial Attention

Visual object perception requires integration of multiple features; spatial attention is thought to be critical to this binding. But attention is rarely static—how does dynamic attention impact object integrity? Here, we manipulated covert spatial attention and had participants (total N = 48) reproduce multiple properties (color, orientation, location) of a target item. Object-feature binding was assessed by applying probabilistic models to the joint distribution of feature errors: Feature reports for the same object could be correlated (and thus bound together) or independent. We found that splitting attention across multiple locations degrades object integrity, whereas rapid shifts of spatial attention maintain bound objects. Moreover, we document a novel attentional phenomenon, wherein participants exhibit unintentional fluctuations—lapses of spatial attention—yet nevertheless preserve object integrity at the wrong location. These findings emphasize the importance of a single focus of spatial attention for object-feature binding, even when that focus is dynamically moving across the visual field.

Object-feature binding survives dynamic shifts of spatial attention

Successful object recognition requires the binding of different visual properties (e.g., color, shape, location) into an integrated object-level representation. Theories of feature-integration propose that spatial attention is crucial for binding—but attention is rarely static, instead dynamically shifting and splitting across multiple goals and locations. What happens to object-feature binding when attention must shift or split across multiple objects with multiple features? While maintaining central fixation, participants were briefly presented with an array of four colored, oriented bars. The target bar was defined by a spatial pre-cue that remained stable (Hold) or dynamically shifted from one location to another (Shift) before array presentation. In another condition, attention was split across two simultaneously pre-cued locations (Split). Participants were instructed to reproduce both the color and orientation (i.e., joint continuous-report) of the target item; they also performed a location report to confirm the target location. Object-feature binding was measured by applying probabilistic models to the joint distribution of feature errors: Errors in recalling both features of the same object could be correlated (and thus bound together) or independent (and unbound). Across multiple experiments, splitting attention across multiple objects degraded object integrity, resulting in unbound feature errors. In contrast, rapid shifts of spatial attention maintained object integrity—even when those shifts were inadvertent. For example, on Hold trials, participants sometimes misreported the target location, indicating a lapse of spatial attention. Yet on these trials, participants reported both the color and orientation bound to that incorrectly-attended location. Moreover, converging data from a parallel line of fMRI experiments reveal that neural reconstructions reflect the features of the attended object on a trial-by-trial basis, even when attention selects the incorrect object. Together, these results emphasize the importance of a single focus of spatial attention in object-feature binding, whether that focus is stable or dynamically shifting across multiple locations. Meeting abstract presented at VSS 2018

Feature-location binding in 3D: Feature judgments are biased by 2D location but not position-in-depth

A fundamental aspect of human visual perception is the ability to recognize and locate objects in the environment. Importantly, our environment is predominantly three-dimensional (3D), but while there is considerable research exploring the binding of object features and location, it is unknown how depth information interacts with features in the object binding process. A recent paradigm called the spatial congruency bias demonstrated that 2D location is fundamentally bound to object features, such that irrelevant location information biases judgments of object features, but irrelevant feature information does not bias judgments of location or other features. Here, using the spatial congruency bias paradigm, we asked whether depth is processed as another type of location, or more like other features. We initially found that depth cued by binocular disparity biased judgments of object color. However, this result seemed to be driven more by the disparity differences than the depth percept: Depth cued by occlusion and size did not bias color judgments, whereas vertical disparity information (with no depth percept) did bias color judgments. Our results suggest that despite the 3D nature of our visual environment, only 2D location information – not position-in-depth – seems to be automatically bound to object features, with depth information processed more similarly to other features than to 2D location.

Contribution of sublinear and supralinear dendritic integration to neuronal computations.

Nonlinear dendritic integration is thought to increase the computational ability of neurons. Most studies focus on how supralinear summation of excitatory synaptic responses arising from clustered inputs within single dendrites result in the enhancement of neuronal firing, enabling simple computations such as feature detection. Recent reports have shown that sublinear summation is also a prominent dendritic operation, extending the range of subthreshold input-output (sI/O) transformations conferred by dendrites. Like supralinear operations, sublinear dendritic operations also increase the repertoire of neuronal computations, but feature extraction requires different synaptic connectivity strategies for each of these operations. In this article we will review the experimental and theoretical findings describing the biophysical determinants of the three primary classes of dendritic operations: linear, sublinear, and supralinear. We then review a Boolean algebra-based analysis of simplified neuron models, which provides insight into how dendritic operations influence neuronal computations. We highlight how neuronal computations are critically dependent on the interplay of dendritic properties (morphology and voltage-gated channel expression), spiking threshold and distribution of synaptic inputs carrying particular sensory features. Finally, we describe how global (scattered) and local (clustered) integration strategies permit the implementation of similar classes of computations, one example being the object feature binding problem.

Evidence for a specific role for muscarinic receptors in crossmodal object recognition in rats

Acetylcholine (ACh) has been implicated in numerous cognitive functions, including multisensory feature binding. In the present study, we systematically assessed the involvement of cholinergic muscarinic receptors in several variations of an object recognition task for rats. In the standard spontaneous object recognition (SOR) task, tactile and visual properties of objects were freely available throughout the sample and choice phases. In the tactile- and visual-only unimodal SOR tasks, exploration in both phases was restricted to tactile and visual information, respectively. For the basic crossmodal object recognition (CMOR) task, sample object exploration was limited to tactile features, whereas choice objects were available only in the visual domain. In Experiment 1, pre-sample systemic administration of scopolamine (0.2mg/kg) disrupted performance on standard SOR, both unimodal SOR tasks, and basic CMOR, consistent with a role for muscarinic receptors in memory encoding. Conversely, in Experiment 2, pre-choice systemic scopolamine selectively impaired object recognition on the CMOR task. For Experiment 3, the inclusion of multimodal, but not unimodal pre-exposure to the to-be-remembered objects prevented scopolamine from disrupting performance on the CMOR task when given prior to the choice phase. These results suggest that ACh is necessary during the choice phase of the CMOR task to facilitate the binding of object features across sensory modalities, a function that is not required for the other tasks assessed. Multimodal object pre-exposure might preclude the requisite contribution of ACh in the choice phase by allowing rats to bind important visual and tactile object information prior to testing.

Multisensory Interactions between Auditory and Haptic Object Recognition

Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.

Storage and binding of object features in visual working memory

An influential conception of visual working memory is of a small number of discrete memory “slots”, each storing an integrated representation of a single visual object, including all its component features. When a scene contains more objects than there are slots, visual attention controls which objects gain access to memory.A key prediction of such a model is that the absolute error in recalling multiple features of the same object will be correlated, because features belonging to an attended object are all stored, bound together. Here, we tested participants’ ability to reproduce from memory both the color and orientation of an object indicated by a location cue. We observed strong independence of errors between feature dimensions even for large memory arrays (6 items), inconsistent with an upper limit on the number of objects held in memory.Examining the pattern of responses in each dimension revealed a gaussian distribution of error centered on the target value that increased in width under higher memory loads. For large arrays, a subset of responses were not centered on the target but instead predominantly corresponded to mistakenly reproducing one of the other features held in memory. These misreporting responses again occurred independently in each feature dimension, consistent with ‘misbinding’ due to errors in maintaining the binding information that assigns features to objects.The results support a shared-resource model of working memory, in which increasing memory load incrementally degrades storage of visual information, reducing the fidelity with which both object features and feature bindings are maintained.

A distributed cortical representation underlies crossmodal object recognition in rats.

The mechanisms by which the brain integrates the unimodal sensory features of an object into a comprehensive multimodal object representation are poorly understood. We have recently developed a procedure for assessing crossmodal object recognition (CMOR) and object feature binding in rats using a modification of the spontaneous object recognition (SOR) paradigm. Here we show for the first time that rats are capable of spontaneous crossmodal object recognition when they are asked to recognize a visually presented object having previously only explored the tactile features of that object. Moreover, rats with bilateral perirhinal cortex (PRh) lesions were impaired on the CMOR task and a visual-only, but not a tactile-only, version of SOR. Conversely, rats with bilateral posterior parietal cortex (PPC) lesions were impaired on the CMOR and tactile-only tasks but not the visual-only task. Finally, crossmodal object recognition ability was severely and selectively impaired in rats with unilateral lesions made to PRh and PPC in opposite hemispheres. Thus, spontaneous tactile-to-visual crossmodal object recognition in rats relies on an object representation that requires functional interaction between PRh and PPC, which appear to mediate the visual and tactile information-processing demands of the task, respectively. These results imply that, at least under certain conditions, the separate sensory features of an object are represented in a distributed manner in the cortex. The novel paradigm introduced here should be a valuable tool for further study of the neurobiological bases of crossmodal cognition and object feature binding.

Cognition and Consciousness: Kantian Affinities with Contemporary Vision Research

After providing a critique of Andreas Engel's neural mechanistic approach to object feature binding (OFB), I develop a Kantian approach to OFB that bears affinity with recent findings in cognitive psychology. I also address the diachronic object unity (DOU) problem and discuss the shortcomings of a purely neural mechanistic approach to this problem. Finally, I motivate a Kantian approach to DOU which suggests that DOU requires the persisting character of the cognizing subject. If plausible, the cognizing subject could make an explanatory contribution to our theory of unified consciousness and thus could not be eliminated on parsimonious grounds alone.

Brain responses to auditory and visual stimulus offset: Shared representations of temporal edges

Edges are crucial for the formation of coherent objects from sequential sensory inputs within a single modality. Moreover, temporally coincident boundaries of perceptual objects across different sensory modalities facilitate crossmodal integration. Here, we used functional magnetic resonance imaging in order to examine the neural basis of temporal edge detection across modalities. Onsets of sensory inputs are not only related to the detection of an edge but also to the processing of novel sensory inputs. Thus, we used transitions from input to rest (offsets) as convenient stimuli for studying the neural underpinnings of visual and acoustic edge detection per se. We found, besides modality-specific patterns, shared visual and auditory offset-related activity in the superior temporal sulcus and insula of the right hemisphere. Our data suggest that right hemispheric regions known to be involved in multisensory processing are crucial for detection of edges in the temporal domain across both visual and auditory modalities. This operation is likely to facilitate cross-modal object feature binding based on temporal coincidence.

Intrinsic Perspectives, Object Feature Binding, and Visual Consciousness

Van der Velde and I agree on two fundamental issues surrounding the vision-related binding problem and recent solutions that have been offered: (1) that tagging theories, such as neuronal synchrony, fail to account for object feature binding in visual consciousness; and (2) that feedforward— feedback processes in the visual cortical hierarchy play a role in generating a feature-unified object of visual consciousness. Van der Velde develops and discusses an important objection to tagging theories that could help to strengthen my critique of neuronal synchrony (and other tagging theories), and then argues that the cognitive subject makes no explanatory contribution to the unity of an object's features in visual consciousness. These issues are discussed in turn. By contrast, van Leeuwen takes a more critical approach to my article. A two-fold response to van Leeuwen is offered: first, the root of his perplexity is uncovered and then some specific objections that he poses to my critique of neuronal synchrony, as a purported solution of the object feature binding problem, are addressed.

Disambiguation, Binding, and the Unity of Visual Consciousness

Recent findings in neuroscience strongly suggest that an object's features (e.g., its color, texture, shape, etc.) are represented in separate areas of the visual cortex. Although represented in separate neuronal areas, somehow the feature representations are brought together as a single, unified object of visual consciousness. This raises a question of binding: how do neural activities in separate areas of the visual cortex function to produce a feature-unified object of visual consciousness? Several prominent neuroscientists have adopted neural synchrony and attention-based approaches to explain object feature binding. I argue that although neural synchrony and/or attentional mechanisms might function to disambiguate an object's features, it is difficult to see how either of these mechanisms could fully explain the unity of an object's features at the level of visual consciousness. After presenting a detailed critique of neural synchrony and attention-based approaches to object feature binding, I propose interactive hierarchical structuralism (IHS) . This view suggests that a unified percept (i.e., a feature-unified object of visual consciousness) is not reducible to the activity of any cognitive capacity or to any localized neural area, but emerges out of the interaction of visual information organized by spatial structuring capacities correlated with lower, higher, and intermediate levels of the visual hierarchy. After clarifying different notions of emergence and elaborating evidence for IHS, I discuss how IHS can be tested through transcranial magnetic stimulation and masking. In the final section I present some further implications/advantages of IHS.

The space of senses: impaired crossmodal interactions in a patient with Balint syndrome after bilateral parietal damage

Balint syndrome after bilateral parietal damage involves a severe disturbance of space representation including impaired oculomotor behaviour, optic ataxia, and simultanagnosia. Binding of object features into a unique spatial representation can also be impaired. We report a patient with bilateral parietal lesions and Balint syndrome, showing severe spatial deficits in several visual tasks predominantly affecting the left hemispace. In particular, we tested whether a loss of spatial representation would affect crossmodal interactions between simultaneous visual and tactile events occurring at the same versus different locations. A tactile discrimination task, where spatially congruent or incongruent visual cues were delivered near the patient’s hands, was used. Following stimulation of the left hand in the left side of space, we observed visuo-tactile interactions that were not modulated by spatially congruent conditions. In contrast, performance following stimulation of the right hand in the right side of space was affected in a spatially selective manner—facilitated for congruent stimuli and slowed for incongruent stimuli. To dissociate effects on somatotopic and spatiotopic coordinates, we crossed the patient’s hands during unimodal tactile discriminations. Tactile performance of the left hand improved when it was positioned in the right hemispace, whereas placing the right hand in left space produced no significant changes, suggesting that left-sided tactile inputs are coded with respect to a combination of limb- and trunk-centred coordinates. These data converge with recent findings in animals and healthy humans to indicate a critical role of the posterior parietal cortex in multimodal spatial integration, and in the fusion of different coordinates into a unified representation of space.

Perceptual organization, the disorganization syndrome, and context processing in chronic schizophrenia

Schizophrenia patients' perceptual organization abilities were assessed with a psychophysically well-controlled measure of contour integration. Compared with psychiatric and staff controls, schizophrenia patients were less able to detect contours comprising Gabor elements as the detection of these contours relied increasingly on long-range spatial interactions. Impaired task performance was also found to correlate significantly with higher levels of disorganized symptomatology. These data provide further evidence for impaired perceptual grouping in schizophrenia. In addition, the findings support the hypothesis that a common cortical processing algorithm involving contextual coordination is impaired in schizophrenia, leading to reduced binding of object features in vision, and reduced contextual disambiguation of linguistic information during thought and speech.