Human Visual Short-term Memory Research Articles

Visual and Semantic Contributions to Visual Short-Term Memory.

Liu et al. recently demonstrated novel neural evidence for visual and semantic contributions to the encoding and maintenance of object information in a delayed match-to-sample task. Their data highlight the close interaction between sensory experience and prior semantic knowledge in human visual short-term memory for naturalistic stimuli.

Stable maintenance of multiple representational formats in human visual short-term memory

Visual short-term memory (VSTM) enables humans to form a stable and coherent representation of the external world. However, the nature and temporal dynamics of the neural representations in VSTM that support this stability are barely understood. Here we combined human intracranial electroencephalography (iEEG) recordings with analyses using deep neural networks and semantic models to probe the representational format and temporal dynamics of information in VSTM. We found clear evidence that VSTM maintenance occurred in two distinct representational formats which originated from different encoding periods. The first format derived from an early encoding period (250 to 770 ms) corresponded to higher-order visual representations. The second format originated from a late encoding period (1,000 to 1,980 ms) and contained abstract semantic representations. These representational formats were overall stable during maintenance, with no consistent transformation across time. Nevertheless, maintenance of both representational formats showed substantial arrhythmic fluctuations, i.e., waxing and waning in irregular intervals. The increases of the maintained representational formats were specific to the phases of hippocampal low-frequency activity. Our results demonstrate that human VSTM simultaneously maintains representations at different levels of processing, from higher-order visual information to abstract semantic representations, which are stably maintained via coupling to hippocampal low-frequency activity.

How does aging influence object-location and name-location binding during a visual short-term memory task?

Objective: Age-related impairments in human visual short-term memory (VSTM) may reflect a reduced ability to retain bound object representations, viz., object form, name, spatial, and temporal location (so called ‘memory sources’). Our objective is to examine how healthy aging affects VSTM in a battery of memory recognition tasks in which sequentially presented objects, locations, and names (as auditory stimuli) were learned, with one component cued at test.Methods: Thirty-six young healthy adults (18–30 years) and 36 normally aging older adults (>60 years with no underlying health and vision issues) completed five VSTM tasks: 1. Object recognition for two or four objects; 2. Spatial location recognition for two or four objects; 3. Bound object-location recognition for two or four objects; 4. Object recognition with location priming for two or four objects; 5. Bound name (auditory)-location (cross-modal) recognition for four objects.Results: Significantly lower performance for older adults was found in spatial location recognition [task 2, p = 0.03], bound object-location recognition [task 3, p = 0.001], object recognition with location priming [task 4, p = 0.02], and bound name-location recognition [task 5, p = 0.001, independent samples t-test] tasks. A significant age group-task interaction was found (p = 0.02).Conclusion: Performance for all tests except test 1 was impaired in older adults. Lower performance for older adults was most significant in VSTM tasks requiring object-location (visual only) or name-location (auditory and visual) binding. The findings are compatible with the ‘memory source’ model, demonstrating that age-related binding performance is influenced by spatial coding and location priming deficits.

How does aging affect the human visual short-term memory task for object-location and name-location binding?

How does aging affect the human visual short-term memory task for object-location and name-location binding?

The Retention and Disruption of Colour Information in Human Visual Short Term Memory

The Retention and Disruption of Colour Information in Human Visual Short Term Memory

Decoding the content of visual short-term memory under distraction in occipital and parietal areas.

Recent studies have provided conflicting accounts regarding where in the human brain visual short-term memory (VSTM) content is stored, with strong univariate fMRI responses reported in superior intraparietal sulcus (IPS) but robust multivariate decoding reported in occipital cortex. Given the continuous influx of information in everyday vision, VSTM storage under distraction is often required. We found that neither distractor presence nor predictability during the memory delay affected behavioral performance. Similarly, superior IPS exhibited consistent decoding of VSTM content across all distractor manipulations and had multivariate responses that closely tracked behavioral VSTM performance. However, occipital decoding of VSTM content was significantly modulated by distractor presence and predictability. Furthermore, we found no effect of target-distractor similarity on VSTM behavioral performance, further challenging the role of sensory regions in VSTM storage. Overall, consistent with previous univariate findings, these results show that superior IPS, not occipital cortex, plays a central role in VSTM storage.

Testing visual short-term memory of pigeons (Columba livia) and a rhesus monkey (Macaca mulatta) with a location change detection task

Change detection is commonly used to assess capacity (number of objects) of human visual short-term memory (VSTM). Comparisons with the performance of non-human animals completing similar tasks have shown similarities and differences in object-based VSTM, which is only one aspect ("what") of memory. Another important aspect of memory, which has received less attention, is spatial short-term memory for "where" an object is in space. In this article, we show for the first time that a monkey and pigeons can be accurately trained to identify location changes, much as humans do, in change detection tasks similar to those used to test object capacity of VSTM. The subject's task was to identify (touch/peck) an item that changed location across a brief delay. Both the monkey and pigeons showed transfer to delays longer than the training delay, to greater and smaller distance changes than in training, and to novel colors. These results are the first to demonstrate location-change detection in any non-human species and encourage comparative investigations into the nature of spatial and visual short-term memory.

Distributed Patterns of Activity in Sensory Cortex Reflect the Precision of Multiple Items Maintained in Visual Short-Term Memory

Traditionally, load sensitivity of sustained, elevated activity has been taken as an index of storage for a limited number of items in visual short-term memory (VSTM). Recently, studies have demonstrated that the contents of a single item held in VSTM can be decoded from early visual cortex, despite the fact that these areas do not exhibit elevated, sustained activity. It is unknown, however, whether the patterns of neural activity decoded from sensory cortex change as a function of load, as one would expect from a region storing multiple representations. Here, we use multivoxel pattern analysis to examine the neural representations of VSTM in humans across multiple memory loads. In an important extension of previous findings, our results demonstrate that the contents of VSTM can be decoded from areas that exhibit a transient response to visual stimuli, but not from regions that exhibit elevated, sustained load-sensitive delay-period activity. Moreover, the neural information present in these transiently activated areas decreases significantly with increasing load, indicating load sensitivity of the patterns of activity that support VSTM maintenance. Importantly, the decrease in classification performance as a function of load is correlated with within-subject changes in mnemonic resolution. These findings indicate that distributed patterns of neural activity in putatively sensory visual cortex support the representation and precision of information in VSTM.

Functional relationships for investigating cognitive processes

Functional relationships (from systematic manipulation of critical variables) are advocated for revealing fundamental processes of (comparative) cognition—through examples from my work in psychophysics, learning, and memory. Functional relationships for pigeon wavelength (hue) discrimination revealed best discrimination at the spectral points of hue transition for pigeons—a correspondence (i.e., functional relationship) similar to that for humans. Functional relationships for learning revealed: Item-specific or relational learning in matching to sample as a function of the pigeons’ sample-response requirement, and same/different abstract-concept learning as a function of the training set size for rhesus monkeys, capuchin monkeys, and pigeons. Functional relationships for visual memory revealed serial position functions (a 1st order functional relationship) that changed systematically with retention delay (a 2nd order relationship) for pigeons, capuchin monkeys, rhesus monkeys, and humans. Functional relationships for rhesus-monkey auditory memory also revealed systematic changes in serial position functions with delay, but these changes were opposite to those for visual memory. Functional relationships for proactive interference revealed interference that varied as a function of a ratio of delay times. Functional relationships for change detection memory revealed (qualitative) similarities and (quantitative) differences in human and monkey visual short-term memory as a function of the number of memory items. It is concluded that these findings were made possible by varying critical variables over a substantial portion of the manipulable range to generate functions and derive relationships.

Visual object complexity limits pigeon short-term memory

The study of visual memory has repeatedly shown qualitatively similar visual short-term memory (VSTM) systems between human and many nonhuman species. In studies of human VSTM using change detection, increasing visual object complexity has an inverse effect on accuracy. In the current study, we assessed the functional relationship between visual object complexity and memory performance in visual change detection in pigeons and humans. Visual object complexity was quantified for each object type within each species using visual target search. Change detection performance was inversely related to object complexity in both species, suggesting that pigeon VSTM, like human VSTM, is limited by visual object complexity. Human participants were able to use a verbal-labeling strategy to mitigate some of the effect of visual object complexity, suggesting a qualitative difference in how the two species may solve certain visual discriminations. Considering the visual complexity of novel objects may also help explain previous failures to transfer relational rules to novel visual objects.

Unleashing Potential: Transcranial Direct Current Stimulation over the Right Posterior Parietal Cortex Improves Change Detection in Low-Performing Individuals

The limits of human visual short-term memory (VSTM) have been well documented, and recent neuroscientific studies suggest that VSTM performance is associated with activity in the posterior parietal cortex. Here we show that artificially elevating parietal activity via positively charged electric current through the skull can rapidly and effortlessly improve people's VSTM performance. This artificial improvement, however, comes with an interesting twist: it interacts with people's natural VSTM capability such that low performers who tend to remember less information benefitted from the stimulation, whereas high performers did not. This behavioral dichotomy is explained by event-related potentials around the parietal regions: low performers showed increased waveforms in N2pc and contralateral delay activity (CDA), which implies improvement in attention deployment and memory access in the current paradigm, respectively. Interestingly, these components are found during the presentation of the test array instead of the retention interval, from the parietal sites ipsilateral to the target location, thus suggesting that transcranial direct current stimulation (tDCS) was mainly improving one's ability to suppress no-change distractors located on the irrelevant side of the display during the comparison stage. The high performers, however, did not benefit from tDCS as they showed equally large waveforms in N2pc and CDA, or SPCN (sustained parietal contralateral negativity), before and after the stimulation such that electrical stimulation could not help any further, which also accurately accounts for our behavioral observations. Together, these results suggest that there is indeed a fixed upper limit in VSTM, but the low performers can benefit from neurostimulation to reach that maximum via enhanced comparison processes, and such behavioral improvement can be directly quantified and visualized by the magnitude of its associated electrophysiological waveforms.

Human Visual Short-Term Memory Precision Can Be Varied at Will When the Number of Retained Items Is Low

It has been debated whether human visual working memory is limited by the number of items or the precision with which they are represented. In the research reported here, we show that the precision of working memory can be flexibly and willfully controlled, but only if the number of retained items is low. Electroencephalographic recordings revealed that a neural marker for visual working memory (contralateral delay activity, or CDA) that is known to increase in amplitude with the number of retained items was also affected by the precision with which items were retained. However, willfully enhanced precision increased CDA amplitude only when the number of retained items was low. These results show that both the number and the (willfully controlled) precision of retained items constrain visual working memory: People can enhance the precision of their visual working memory, but only for a few items.

The Temporal Evolution of Electromagnetic Markers Sensitive to the Capacity Limits of Visual Short-Term Memory

An electroencephalographic (EEG) marker of the limited contents of human visual short-term memory (VSTM) has previously been described. Termed contralateral delay activity, this consists of a sustained, posterior, negative potential that correlates with memory load and is greatest contralateral to the remembered hemifield. The current investigation replicates this finding and uses magnetoencephalography (MEG) to characterize its magnetic counterparts and their neural generators as they evolve throughout the memory delay. A parametric manipulation of memory load, within and beyond capacity limits, allows separation of signals that asymptote with behavioral VSTM performance from additional responses that contribute to a linear increase with set-size. Both EEG and MEG yielded bilateral signals that track the number of objects held in memory, and contralateral signals that are independent of memory load. In MEG, unlike EEG, the contralateral interaction between hemisphere and item load is much weaker, suggesting that bilateral and contralateral markers of memory load reflect distinct sources to which EEG and MEG are differentially sensitive. Nonetheless, source estimation allowed both the bilateral and the weaker contralateral capacity-limited responses to be localized, along with a load-independent contralateral signal. Sources of global and hemisphere-specific signals all localized to the posterior intraparietal sulcus during the early delay. However the bilateral load response peaked earlier and its generators shifted later in the delay. Therefore the hemifield-specific response may be more closely tied to memory maintenance while the global load response may be involved in initial processing of a limited number of attended objects, such as their individuation or consolidation into memory.

A hemodynamic correlate of lateralized visual short-term memories

Neuroimaging studies attempting to isolate the neural substrate of visual short-term memory in humans have concentrated on the behavior of neurons populating the posterior part of the parietal cortex as a possible source of visual short-term memory capacity limits. Using a standard change-detection task, fMRI studies have shown that maintenance of bilaterally encoded objects elicited bilateral increases of hemodynamic activation in the intra-parietal and intra-occipital sulci (IPS–IOS) proportional to the number of objects retained in visual short-term memory. We used a spatially cued variant of the change-detection task to record hemodynamic responses to unilaterally encoded objects using functional near-infrared spectroscopy (fNIRS). Electrophysiological studies that employed this task have shown that maintenance of unilaterally encoded objects elicited posterior unilateral (contralateral) increase in event-related negativity proportional to the number of objects retained in visual short-term memory. We therefore examined whether contralateral increases in oxy-hemoglobin concentration correlated with the number of retained objects. Contrary to the idea that bilateral increases in BOLD responses and unilateral increases in event-related negativity may be different reflections of the same underlying neural/functional processing, memory-related increases in oxy-hemoglobin concentration were found bilaterally even when objects had to be encoded unilaterally. The present findings suggest that EEG and fMRI/fNIRS techniques reveal distinct neural signatures of the mechanisms supporting visual short-term memory.

A behavioural investigation of human visual short term memory for colour

We examined visual short term memory (VSTM) for colour using a delayed-match-to-sample paradigm. In these experiments we measured the effects of increasing inter-stimulus interval (ISI), varying between 0 and 10 s, on the ability of five colour normal human observers to make colour matches between a reference and subsequently presented test stimuli. The coloured stimuli used were defined by different chromatic axes on the isoluminant plane of DKL colour space. In preliminary experiments we used a hue scaling procedure to identify a total of 12 colour stimuli which served as reference hues in the colour memory experiments: four stimuli were exemplars of red, green, blue and yellow colour appearance categories, four were located between these categories and a further four were located on the cardinal axes that isolated the activity of the cone-opponent mechanisms. Our results demonstrate that there is a reduction in the ability of observers to make accurate colour matches with increasing ISIs and that this reduced performance was similar for all colour stimuli. However, the shifts in hue that were measured between the reference and matched test stimuli were significantly greater for the cardinal stimuli compared to those measured for the stimuli defined by the hue scaling procedure. This deterioration in the retention of hue in VSTM for stimuli that isolate cone-opponent mechanisms may be a reflection of the reorganisation of colour processing that occurs in the cortex where colour appearance mechanisms become more prominent.

Dual impact of extra-foveal processing in human visual short-term memory

Dual impact of extra-foveal processing in human visual short-term memory

Theta bandwidth determines human visual working memory capacity Human visual short term memory (VSTM) is capacity limited1. The neurophysiological basis for this limitation is, however, currently unknown. Here we apply the principle of efficient information transfer, derived from communications theory, to provide a mechanistic account of VSTM capacity. Shannon's information theory states that to increase the capacity of information encoded

Theta bandwidth determines human visual working memory capacity Human visual short term memory (VSTM) is capacity limited1. The neurophysiological basis for this limitation is, however, currently unknown. Here we apply the principle of efficient information transfer, derived from communications theory, to provide a mechanistic account of VSTM capacity. Shannon's information theory states that to increase the capacity of information encoded

Differential Neural Responses during Performance of Matching and Nonmatching to Sample Tasks at Two Delay Intervals

Visual short-term memory in humans and animals is frequently assessed using delayed matching to sample (DMTS) and delayed nonmatching to sample (DNMTS) tasks across variable delay intervals. Although these tasks depend on certain common mechanisms, there are behavioral differences between them, and neuroimaging provides a means of assessing explicitly whether this is underpinned by differences at a neural level. Findings of delay-dependent deficits, after lesions in humans and animals, suggest that the neural implementation of these tasks may also critically depend on the delay interval. In this study we determined whether there were differential neural responses associated with DMTS and DNMTS tasks at two different delay intervals using functional magnetic resonance imaging. Ten healthy volunteers were studied under four test conditions: DMTS and DNMTS at 5 and 15 sec delay. The main effect of DMTS compared with DNMTS across both delay intervals was associated with significant activation in bilateral head of caudate and medial orbitofrontal cortex. By contrast, DNMTS compared with DMTS was associated with significant activation in mediodorsal thalamus, bilateral lateral orbitofrontal cortex, and left premotor cortex. The main effect of short compared with long delay, across both tasks, was associated with significantly greater activity in occipital and parietal cortices. By contrast, long compared with short delay was associated with significantly greater activity in temporal and ventrolateral frontal cortices. We conclude that DMTS and DNMTS are not equivalent and furthermore that the precise neural implementation of these tasks is a dynamic function of delay interval.

Representations in human visual short-term memory: an event-related brain potential study

Behavioral measures and event-related potentials (ERPs) were recorded from 12 subjects while performing three delayed matching-to-sample tasks. The task instructions indicated whether stimulus locations, shapes or conjunctions of locations and shapes had to be memorized and matched against a probe. Memory load was varied trial-by-trial by cueing one or two of three stimuli for memorization, followed by one probe. ERPs during memorization and probe response times varied as a function of memory load (posterior negative and anterior positive ERP wave), but not between single-feature and conjunction of features, nor was an interaction found between them. The results support and extend hypotheses by Luck and Vogel (Luck, S.J. and Vogel, E.K., The capacity of visual working memory for features and conjunctions. Nature, 390 (1997) 279–281.) that visual short-term memory stores conjunctions of features rather than single features.

Measurements and source estimations of extremely low frequency brain magnetic fields in a short-term memory task by a whole-head neurogradiometer

To investigate the characteristics of visual short-term memory in humans, brain magnetic fields evoked during a delayed paired comparison task were recorded using a whole-head neuromagnetometer. The visual stimulus consisted of a circle with different colors in each quadrant. In the memory condition, subjects reacted with the index finger, when the first stimulus (Sample) was identical in color configuration to the second stimulus (Test), and with the middle finger when they differed. For the control condition, the subjects ignored the Sample, and moved the index or middle finger alternately in response to the Test. Extremely low frequency components of brain magnetic fields were observed 500 ms after the Sample onset in the temporal and/or the occipital region in the memory condition, but not in the control condition. Sources for the low frequency components were localized in the inferior part of the occipital lobe, in the vicinity of the supramarginal gyrus and the angular gyrus, and the inferior frontal gyrus. The results suggest that the activities in the inferior part of the occipital lobe controls the storage process of short-term visual memory.