Objects In Visual Short-term Memory Research Articles

Neural correlates of visual short-term memory for objects with material categories

Behavioral and neuroscience studies have shown that we can easily identify material categories, such as metal and fabric. Not only the early visual areas but also higher-order visual areas including the fusiform gyrus are known to be engaged in material perception. However, the brain mechanisms underlying visual short-term memory (VSTM) for material categories are unknown. To address this issue, we examined the neural correlates of VSTM for objects with material categories using a change detection task. In each trial, participants viewed a sample display containing two, four, or six objects having six material categories and were required to remember the locations and types of objects. After a brief delay, participants were asked to detect an object change based on the images or material categories in the test display (image-based and material-based conditions). Neuronal activity in the brain was assessed using functional magnetic resonance imaging (MRI). Behavioral results showed that the number of objects encoded did not increase as a function of set size in either image-based or material-based conditions. By contrast, MRI data showed a difference between the image-based and material-based conditions in percent signal change observed in a priori region of interest, the fusiform face area (FFA). Thus, we failed to achieve our research aim. However, the brain activation in the FFA correlated with the activation in the precentral/postcentral gyrus, which is related to haptic processing. Our findings indicate that the FFA may be involved in VSTM for objects with material categories in terms of the difference between images and material categories and that this memory may be mediated by the tactile properties of objects.

Training Enhances Fidelity of Color Representations in Visual Long-Term Memory

Visual perception can be affected by training mental representations. However, it remains unclear if training procedures can also affect the quality of mental representations. To investigate if training enhances the fidelity of mental representations retrieved from visual long-term memory (VLTM), we used a task including object-color associations with a continuous response-space. We tested 15 participants in a training group and 15 participants in a control group. Training consisted of six training runs executed on 3 consecutive days. Before and after training, we assessed accessibility and fidelity of mental representations in VLTM and of single objects in visual short-term memory (VSTM). Not only accessibility to mental representation but also their fidelity increased across training and transferred to novel object-color associations in VLTM and VSTM after training. At the end of the training, fidelity of VLTM representations were virtually identical to fidelity of VSTM representations. We conclude that training object-color associations does not only improve the accessibility of VLTM representations, but also their fidelity based on perceptual plasticity of the visual system.

Stimulus- and Response-locked Posterior Contralateral Negativity Bisect Cognitive Operations in Visual Search.

We explored the flow of information during visual search by examining activity indexing visual attention (N2pc) and the subsequent processing of the selected objects in visual short-term memory (SPCN) time-locked to stimulus presentation and to the motor response. We measured event-related activity at posterior sites (PO7/PO8) for 96 participants during a simple visual search task. A response-locked posterior contralateral negativity (RLpcN) was observed with a scalp distribution similar to that of the N2pc and SPCN. The RLpcN was compared with the stimulus-locked activity (N2pc and SPCN) across experimental manipulations (targets were either closer or farther from fixation in visual space, and the response was either more frequent [75%] or less frequent [25%]) and across response speeds (EEG data were separated into tertiles by RT both within-subjects and between-subjects). The leading edge and early portion of the RLpcN appeared to reflect the initial deployment of attention (N2pc), whereas the later portion (up to peak amplitude) reflected subsequent processing of visual information (SPCN). SPCN and RLpcN also had similar modulations in amplitude for both analyses. Moreover, whereas very small N2pc and SPCN onset latency differences were observed when data were separated into tertiles by RT, there were large onset differences for the RLpcN, with earlier RLpcN onsets for longer RTs, suggesting that RT variance is in large determined by processing after the initial deployment of attention. The results show how we can bisect processing responsible for variations in RT relative to the onset of visual spatial attention.

The neural basis of precise visual short-term memory for complex recognisable objects

Recent evidence suggests that visual short-term memory (VSTM) capacity estimated using simple objects, such as colours and oriented bars, may not generalise well to more naturalistic stimuli. More visual detail can be stored in VSTM when complex, recognisable objects are maintained compared to simple objects. It is not yet known if it is recognisability that enhances memory precision, nor whether maintenance of recognisable objects is achieved with the same network of brain regions supporting maintenance of simple objects.We used a novel stimulus generation method to parametrically warp photographic images along a continuum, allowing separate estimation of the precision of memory representations and the number of items retained. The stimulus generation method was also designed to create unrecognisable, though perceptually matched, stimuli, to investigate the impact of recognisability on VSTM. We adapted the widely-used change detection and continuous report paradigms for use with complex, photographic images.Across three functional magnetic resonance imaging (fMRI) experiments, we demonstrated greater precision for recognisable objects in VSTM compared to unrecognisable objects. This clear behavioural advantage was not the result of recruitment of additional brain regions, or of stronger mean activity within the core network. Representational similarity analysis revealed greater variability across item repetitions in the representations of recognisable, compared to unrecognisable complex objects. We therefore propose that a richer range of neural representations support VSTM for complex recognisable objects.

Discrete capacity limits and neuroanatomical correlates of visual short-term memory for objects and spatial locations.

Working memory is responsible for keeping information in mind when it is no longer in view, linking perception with higher cognitive functions. Despite such crucial role, short-term maintenance of visual information is severely limited. Research suggests that capacity limits in visual short-term memory (VSTM) are correlated with sustained activity in distinct brain areas. Here, we investigated whether variability in the structure of the brain is reflected in individual differences of behavioral capacity estimates for spatial and object VSTM. Behavioral capacity estimates were calculated separately for spatial and object information using a novel adaptive staircase procedure and were found to be unrelated, supporting domain-specific VSTM capacity limits. Voxel-based morphometry (VBM) analyses revealed dissociable neuroanatomical correlates of spatial versus object VSTM. Interindividual variability in spatial VSTM was reflected in the gray matter density of the inferior parietal lobule. In contrast, object VSTM was reflected in the gray matter density of the left insula. These dissociable findings highlight the importance of considering domain-specific estimates of VSTM capacity and point to the crucial brain regions that limit VSTM capacity for different types of visual information. Hum Brain Mapp 38:767-778, 2017. © 2016 Wiley Periodicals, Inc.

The capacity and fidelity of visual short-term memory for objects and ensembles

The capacity and fidelity of visual short-term memory for objects and ensembles

The effect of spatial organization of targets and distractors on the capacity to selectively memorize objects in visual short-term memory.

We conducted a series of experiments to explore how the spatial configuration of objects influences the selection and the processing of these objects in a visual short-term memory task. We designed a new experiment in which participants had to memorize 4 targets presented among 4 distractors. Targets were cued during the presentation of distractor objects. Their locations varied according to 4 spatial configurations. From the first to the last configuration, the distance between targets’ locations was progressively increased. The results revealed a high capacity to select and memorize targets embedded among distractors even when targets were extremely distant from each other. This capacity is discussed in relation to the unitary conception of attention, models of split attention, and the competitive interaction model. Finally, we propose that the spatial dispersion of objects has different effects on attentional allocation and processing stages. Thus, when targets are extremely distant from each other, attentional allocation becomes more difficult while processing becomes easier. This finding implicates that these 2 aspects of attention need to be more clearly distinguished in future research.

Distinguishing between lateralized and nonlateralized brain activity associated with visual short-term memory: fMRI, MEG, and EEG evidence from the same observers

Previous functional neuroimaging studies have shown that maintenance of centrally presented objects in visual short-term memory (VSTM) leads to bilateral increases of BOLD activations in IPS/IOS cortex, while prior electrophysiological work suggests that maintaining stimuli encoded from a single hemifield leads to a sustained posterior contralateral negativity (SPCN) in electrophysiology and magnetoencephalography. These two findings have never been investigated using the same physiological measures. We recorded the BOLD response using fMRI, magnetoencephalography (MEG), and electrophysiology (EEG), while subjects encoded visual stimuli from a single hemifield of a balanced display. The EEG showed an SPCN. However, no SPCN-like activation was observed in the BOLD signals. The BOLD response in parietal cortex remained bilateral, even after unilateral encoding of the stimuli, but MEG showed both bilateral and contralateral activations, each likely reflecting a sub portion of the neuronal populations participating in the maintenance of information in VSTM. Contrary to the assumption that BOLD, EEG, and MEG responses – that were each linked to the maintenance of information in VSTM – are markers of the same neuronal processes, our findings suggest that each technique reveals a somewhat distinct but overlapping neural signature of the mechanisms supporting visual short-term memory.

Updating feature information about objects in visual short-term memory

Updating feature information about objects in visual short-term memory

Strategic control of visual short-term memory during scene viewing

During scene viewing, the eyes and attention are directed serially to objects of interest, and visual short-term memory (VSTM) is used retain information from recently attended objects (H lli th 2004) I th t t d i d h th l t t i ll Experiment 1: Results 100 100 Experiment 2: Results 100 Experiment 3: Results o ngwor , . n e presen s u y, we exam ne w e er peop e can s ra eg ca y control the content of VSTM during scene viewing, retaining task-relevant objects in VSTM even as attention and the eyes are directed to subsequent objects. Participants viewed a set of real-world objects presented serially within a 3-D rendered scene. One object in the sequence was cued by a tone as “to-be-remembered”. At the end of the sequence, memory for the visual form of one object was tested. Participants exhibited tight control over the content of VSTM, successfully protecting task-relevant objects from subsequent interference. Such strategic maintenance of objects in VSTM could play an important role in real-world visual behavior, especially when object information must be maintained across shifts of attention and the eyes to other objects (such as when comparing two er ce nt ag e C or re ct 70 80 90 Cue Position 2 Cue Position 3 Neutral Pe rc en ta ge C or re ct

Updating objects in visual short-term memory

Updating objects in visual short-term memory

Representing objects in visual short-term memory: The roles of the human intra-parietal sulcus and the lateral occipital complex

Representing objects in visual short-term memory: The roles of the human intra-parietal sulcus and the lateral occipital complex

Subliminally Presented and Stored Objects Capture Spatial Attention

When objects disappear from view, we can still bring them to mind, at least for brief periods of time, because we can represent those objects in visual short-term memory (VSTM) (Sperling, 1960; Cowan, 2001). A defining characteristic of this representation is that it is topographic, that is, it preserves a spatial organization based on the original visual percept (Vogel and Machizawa, 2004; Astle et al., 2009; Kuo et al., 2009). Recent research has also shown that features or locations of visual items that match those being maintained in conscious VSTM automatically capture our attention (Awh and Jonides, 2001; Olivers et al., 2006; Soto et al., 2008). But do objects leave some trace that can guide spatial attention, even without participants intentionally remembering them? Furthermore, could subliminally presented objects leave a topographically arranged representation that can capture attention? We presented objects either supraliminally or subliminally and then 1 s later re-presented one of those objects in a new location, as a "probe" shape. As participants made an arbitrary perceptual judgment on the probe shape, their covert spatial attention was drawn to the original location of that shape, regardless of whether its initial presentation had been supraliminal or subliminal. We demonstrate this with neural and behavioral measures of memory-driven attentional capture. These findings reveal the existence of a topographically arranged store of "visual" objects, the content of which is beyond our explicit awareness but which nonetheless guides spatial attention.

The attentional selection in visual search within short-term memory representations

During the perception of a visual scene, attention can be directed to parts of the space (Posner and Cohen, 1984), distinct features (Treisman and Gelade, 1980) and, as recently shown, also to entire objects (Desimone and Duncan, 1995; Valdes-Sosa et al., 1998; O'Craven et al., 1999; Schoenfeld et al., 2003). Depending on the goals of the observer, those types of selection are flexibly employed to eliminate ambiguity and to shape out wanted from unwanted visual information (Hopf et al., 2005; Stoppel et al., 2007). It is becoming increasingly clear that attentional selection can also operate upon internal representations held in visual short-term memory (VSTM) beyond the phase of perceptual encoding. Importantly, these representations appear to maintain at least some of the spatial configuration of the initial perceptual scene (Gratton, 1998; Jiang et al., 2000). Very recent work has shown that the selection of a target within a search array maintained in VSTM draws on similar mechanisms as during perception. For example, the identification of a color target within that array was observed to elicit an N2pc component in the electrophysiological response (Kuo et al., 2009). The N2pc is a well established signature of attentional selection in visual search (Luck and Hillyard, 1994) which reflects spatial biasing of activity in contralateral posterior parietal and occipital-temporal areas during target selection and the suppression of distracters (Hopf et al., 2000, 2002). A recent study by Astle et al. (2009) investigated not only whether the spatial representation of an array of items is maintained in VSTM, but went on to look for the mechanism underlying the selection of individual item features stored in VSTM. This is an important issue because studies have shown that when items are stored in VSTM, the individual features are bound together to form integrated objects (Vogel et al., 2001; Awh et al., 2007). This could lead to conflicting information at the level of feature selection because features requiring selection may be bound to features to be discarded from processing. This is the situation Astle et al. (2009) address in one of their experiments. The subjects memorized two items of different shape and color and were subsequently cued to either recall whether a certain color or shape was present in the array (VSTM search). In another experiment, subjects were first cued to a particular shape or color and then asked to perform the search in a presented array (perceptual search). Astle et al. (2009) compared the ERP correlates of VSTM search to those of perceptual search and observed that both were associated with a lateralized ERP response, suggesting that VSTM, like perceptual search, involves spatially organized attentional selection. Surprisingly, in contrast to the perceptual search, the lateralized response seen in VSTM search was of opposite polarity to the typical N2pc component. This is intriguing and at the same time surprising, as this contrasts with the previous demonstration of a typical N2pc component being associated with VSTM search (Kuo et al., 2009). The authors suggest that while during perceptual search participants can search the array at a feature-specific level, VSTM search requires these processes to occur at an object-specific level. In contrast to the study by Kuo et al. (2009), in the present experiment the binding of features into objects in VSTM led to conflicting requirements of feature selection, namely, to the necessity to simultaneously boost and suppress features that are part of the same object. Whether this could entirely explain an inversed polarity of the N2pc component is still to be determined in further studies. Nonetheless, the data of Astle et al. (2009) points to different mechanisms of feature selection in perceptual and VSTM search, especially when conflicting feature-level information interferes with the target selection.

Orienting attention to objects in visual short-term memory

We measured electroencephalographic activity during visual search of a target object among objects available to perception or among objects held in visual short-term memory (VSTM). For perceptual search, a single shape was shown first (pre-cue) followed by a search-array, and the task was to decide whether the pre-cue was or was not in the search-array. For search of VSTM, a search-array was shown first followed by a single shape (post-cue), and the task was to decide whether the post-cue was or was not in the previously displayed search-array. We focused on early lateralized electrical brain activity over posterior and temporal areas time-locked to search-arrays in pre-cue trials and to post-cues in post-cue trials. In Experiment 1, search-arrays were composed of two lateralized shapes, displayed in the upper/lower two quadrants of the monitor. In Experiment 2, search-arrays were composed of four shapes, displayed at the corners of an imaginary square centered on fixation. In pre-cue trials, we observed an N2pc of about equal amplitude and latency for search-arrays composed of two or four shapes. In post-cue trials, we observed N2pc-like activity with search-arrays composed of two shapes, that was however substantially attenuated with search-arrays composed of four shapes. For many aspects, attending to a perceptual object was functionally and neurally analogous to attending to an object held in VSTM, suggesting that spatial selective attention biases search of objects during both ongoing perception and retention.

Updating objects in visual short-term memory is feature selective

The purpose of this study was to examine whether the process of updating information in visual short-term memory (VSTM) is object based. We investigated whether modifying the memory of one feature of an object would automatically promote refreshing the memory of all of its other features. The results showed that the facilitative effect of updating was specific to the updated feature of an object and did not spread to its nonupdated features. This feature-selective effect suggests that updating VSTM is not object based (Experiment 1), even though storage was object based (Experiment 2). Control experiments ruled out strategy-based (Experiment 3) and stimulus-related (Experiments 4-6) accounts. Feature-selective updating may indicate that the mechanism used to modify the contents of memory may have a different basis than that used to encode or store information in memory.

Oscillatory activity in parietal and dorsolateral prefrontal cortex during retention in visual short‐term memory: Additive effects of spatial attention and memory load

We used whole-head magnetoencephalography to study the representation of objects in visual short-term memory (VSTM) in the human brain. Subjects remembered the location and color of either two or four colored disks that were encoded from the left or right visual field (equal number of distractors in the other visual hemifield). The data were analyzed using time-frequency methods, which enabled us to discover a strong oscillatory activity in the 8-15 Hz band during the retention interval. The study of the alpha power variation revealed two types of responses, in different brain regions. The first was a decrease in alpha power in parietal cortex, contralateral to the stimuli, with no load effect. The second was an increase of alpha power in parietal and lateral prefrontal cortex, as memory load increased, but without interaction with the hemifield of the encoded stimuli. The absence of interaction between side of encoded stimuli and memory load suggests that these effects reflect distinct underlying mechanisms. A novel method to localize the neural generators of load-related oscillatory activity was devised, using cortically-constrained distributed source-localization methods. Some activations were found in the inferior intraparietal sulcus (IPS) and intraoccipital sulcus (IOS). Importantly, strong oscillatory activity was also found in dorsolateral prefrontal cortex (DLPFC). Alpha oscillatory activity in DLPFC was synchronized with the activity in parietal regions, suggesting that VSTM functions in the human brain may be implemented via a network that includes bilateral DLPFC and bilateral IOS/IPS as key nodes.

Human posterior parietal cortex maintains color, shape and motion in visual short-term memory

We used functional magnetic resonance imaging (fMRI) to investigate the neural substrate of visual short-term memory for objects defined by features processed in the dorsal and the ventral visual streams. Here we adopted the conventional delayed recognition task, whereas in addition to the more commonly used visual features of color and shape, motion direction was applied to define an item. Our behavioral results indicated that the capacity limit of visual short-term memory of motion direction was approximately two, which was significantly lower than those of color and shape, about three or four. We also found that storage capacity was significantly reduced when subjects were required to retain all three features superimposed in space. Meanwhile, fMRI results revealed that activity in the posterior part of the superior parietal lobe was memory–load dependent for all three features indicating that it collects and stores visual information from both the two visual processing streams, whereas the anterior part was load dependent only for motion.

Dissociable neural mechanisms supporting visual short-term memory for objects

Using visual information to guide behaviour requires storage in a temporary buffer, known as visual short-term memory (VSTM), that sustains attended information across saccades and other visual interruptions. There is growing debate on whether VSTM capacity is limited to a fixed number of objects or whether it is variable. Here we report four experiments using functional magnetic resonance imaging that resolve this controversy by dissociating the representation capacities of the parietal and occipital cortices. Whereas representations in the inferior intra-parietal sulcus (IPS) are fixed to about four objects at different spatial locations regardless of object complexity, those in the superior IPS and the lateral occipital complex are variable, tracking the number of objects held in VSTM, and representing fewer than four objects as their complexity increases. These neural response patterns were observed during both VSTM encoding and maintenance. Thus, multiple systems act together to support VSTM: whereas the inferior IPS maintains spatial attention over a fixed number of objects at different spatial locations, the superior IPS and the lateral occipital complex encode and maintain a variable subset of the attended objects, depending on their complexity. VSTM capacity is therefore determined both by a fixed number of objects and by object complexity.

Distributed cortical systems in visual short-term memory revealed by event-related functional magnetic resonance imaging.

The spatio-temporal distribution of brain activity as revealed by non-invasive functional imaging helps to elucidate the neuronal encoding and processing strategies required by complex cognitive tasks. We investigated visual short-term memory for objects, places and conjunctions in humans using event-related time-resolved functional magnetic resonance imaging that permitted segregation of encoding, retention and retrieval phases. All conditions were accompanied by the activation of a widespread network of parietal and prefrontal areas during the retention phase, but this retention-related activity showed additional modulations depending on task instructions. These modulations confirmed a posterior-anterior and right-left dissociation for spatial versus non-spatial memory and revealed that conjunction memory does not rely on a linear addition of the component processes.