Models Of Visual Working Memory Research Articles

A dynamic neural resource model bridges sensory and working memory

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or ‘iconic’ memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

A dynamic neural resource model bridges sensory and working memory.

Probing memory of a complex visual image within a few hundred milliseconds after its disappearance reveals significantly greater fidelity of recall than if the probe is delayed by as little as a second. Classically interpreted, the former taps into a detailed but rapidly decaying visual sensory or 'iconic' memory (IM), while the latter relies on capacity-limited but comparatively stable visual working memory (VWM). While iconic decay and VWM capacity have been extensively studied independently, currently no single framework quantitatively accounts for the dynamics of memory fidelity over these time scales. Here, we extend a stationary neural population model of VWM with a temporal dimension, incorporating rapid sensory-driven accumulation of activity encoding each visual feature in memory, and a slower accumulation of internal error that causes memorized features to randomly drift over time. Instead of facilitating read-out from an independent sensory store, an early cue benefits recall by lifting the effective limit on VWM signal strength imposed when multiple items compete for representation, allowing memory for the cued item to be supplemented with information from the decaying sensory trace. Empirical measurements of human recall dynamics validate these predictions while excluding alternative model architectures. A key conclusion is that differences in capacity classically thought to distinguish IM and VWM are in fact contingent upon a single resource-limited WM store.

Scaling models of visual working memory to natural images

Over the last few decades, psychologists have developed precise quantitative models of human recall performance in visual working memory (VWM) tasks. However, these models are tailored to a particular class of artificial stimulus displays and simple feature reports from participants (e.g., the color or orientation of a simple object). Our work has two aims. The first is to build models that explain people’s memory errors in continuous report tasks with natural images. Here, we use image generation algorithms to generate continuously varying response alternatives that differ from the stimulus image in natural and complex ways, in order to capture the richness of people’s stored representations. The second aim is to determine whether models that do a good job of explaining memory errors with natural images also explain errors in the more heavily studied domain of artificial displays with simple items. We find that: (i) features taken from state-of-the-art deep encoders predict trial-level difficulty in natural images better than several reasonable baselines; and (ii) the same visual encoders can reproduce set-size effects and response bias curves in the artificial stimulus domains of orientation and color. Moving forward, our approach offers a scalable way to build a more generalized understanding of VWM representations by combining recent advances in both AI and cognitive modeling.

A spatially continuous diffusion model of visual working memory

We present results from five visual working memory (VWM) experiments in which participants were briefly shown between 2 and 6 colored squares. They were then cued to recall the color of one of the squares and they responded by choosing the color on a continuous color wheel. The experiments provided response proportions and response time (RT) measures as a function of angle for the choices. Current VWM models for this task include discrete models that assume an item is either within working memory or not and resource models that assume that memory strength varies as a function of the number of items. Because these models do not include processes that allow them to account for RT data, we implemented them within the spatially continuous diffusion model (SCDM, Ratcliff, 2018) and use the experimental data to evaluate these combined models. In the SCDM, evidence retrieved from memory is represented as a spatially continuous normal distribution and this drives the decision process until a criterion (represented as a 1-D line) is reached, which produces a decision. Noise in the accumulation process is represented by continuous Gaussian process noise over spatial position. The models that fit best from the discrete and resource-based classes converged on a common model that had a guessing component and that allowed the height of the normal memory-strength distribution to vary with number of items. The guessing component was implemented as a regular decision process driven by a flat evidence distribution, a zero-drift process. The combination of choice and RT data allows models that were not identifiable based on choice data alone to be discriminated.

Mapping visual working memory models to a theoretical framework.

The body of research on visual working memory (VWM)-the system often described as a limited memory store of visual information in service of ongoing tasks-is growing rapidly. The discovery of numerous related phenomena, and the many subtly different definitions of working memory, signify a challenge to maintain a coherent theoretical framework to discuss concepts, compare models and design studies. A lack of robust theory development has been a noteworthy concern in the psychological sciences, thought to be a precursor to the reproducibility crisis (Oberauer & Lewandowsky, Psychonomic Bulletin & Review, 26, 1596-1618, 2019). I review the theoretical landscape of the VWM field by examining two prominent debates-whether VWM is object-based or feature-based, and whether discrete-slots or variable-precision best describe VWM limits. I share my concerns about the dualistic nature of these debates and the lack of clear model specification that prevents fully determined empirical tests. In hopes of promoting theory development, I provide a working theory map by using the broadly encompassing memory for latent representations model (Hedayati et al., Nature Human Behaviour, 6, 5, 2022) as a scaffold for relevant phenomena and current theories. I illustrate how opposing viewpoints can be brought into accordance, situating leading models of VWM to better identify their differences and improve their comparison. The hope is that the theory map will help VWM researchers get on the same page-clarifying hidden intuitions and aligning varying definitions-and become a useful device for meaningful discussions, development of models, and definitive empirical tests of theories.

Uncovering the Neurobiological Consequences of High-Voltage Electrical Field Exposure on the Visual Working Memory of Macaques and Also Using Spiking Neural Network Model

High-voltage (HV) power transmission lines running near cities and villages can cause severe damage (Mental and physical). Due to the magnetic and electric fields they produce. This study aimed to investigate the effects of high-voltage (HV) electric fields on the spiking neural network model of the brain and biological and behavioral models of visual working memory. To achieve this goal, macaques were studied for their cognitive functions, expression of the NMDA receptor gene, MRI-assisted analysis of brain anatomy, and variations in blood sodium and potassium concentrations. The experimental group of macaques was exposed to a 3kV/m high-voltage field for four hours a day for one month. Computational models were then evaluated using experimental parameters. According to the results, it was observed that being exposed to high-voltage electric fields led to a reduction in the expression of the NMDA receptor gene, as well as a decrease in the levels of Sodium and potassium ions in the blood plasma. Additionally, analysis assisted by MRI showed a decrease in the volume of the hippocampus and amygdala after exposure to the electric field. In conclusion, the results of cognitive, genetic, blood, and MRI tests, along with the spiking neural network model, elucidate the mechanism of the visual working memory deterioration in macaques due to high-voltage electric field exposure.

Don't hide the instruction manual: A dynamic trade-off between using internal and external templates during visual search.

Visual search is typically studied by requiring participants to memorize a template initially, for which they subsequently search in a crowded display. Search in daily life, however, often involves templates that remain accessible externally, and may therefore be (re)attended for just-in-time encoding or to refresh internal template representations. Here, we show that participants indeed use external templates during search when given the chance. This behavior was observed during both simple and complex search, scaled with task difficulty, and was associated with improved performance. Furthermore, we show that participants used external sampling not only to offload memory, but also as a means of verifying whether the template was remembered correctly at the end of trials. We conclude that the external world may not only provide the challenge (e.g., distractors), but may dynamically ease search. These results argue for extensions of state-of-the-art models of search, because external sampling seems to be used frequently, in at least two ways and is actually beneficial for task performance. Our findings support a model of visual working memory that emphasizes a resource-efficient trade-off between storing and (re)attending external information.

Attentional prioritisation and facilitation for similar stimuli in visual working memory

Visual working memory (VWM) allows for the brief retention of approximately three to four items. Interestingly, when these items are similar to each other in a feature domain, memory recall performance is elevated compared to when they are dissimilar. This similarity benefit is currently not accounted for by models of VWM. Previous research has suggested that this similarity benefit may arise from selective attentional prioritisation in the maintenance phase. However, the similarity effect has not been contrasted under circumstances where dissimilar item types can adequately compete for memory resources. In Experiment 1, similarity benefits were seen for all-similar over all-dissimilar displays. This was also seen in mixed displays, change detection performance was higher when one of the two similar items changed, compared to when the dissimilar item changed. Surprisingly, the similarity effect was stronger in these mixed displays then when comparing the all-similar and all-dissimilar. Experiment 2 investigated this further by examining how attention was allocated in the memory encoding phase via eye movements. Results revealed that attention prioritised similar over dissimilar items in the mixed displays. Similar items were more likely to receive the first fixation and were fixated more often than dissimilar items. Furthermore, dwell times were elongated for dissimilar items, suggesting that encoding was less efficient. These results suggest that there is an attentional strategy towards prioritising similar items over dissimilar items, and that this strategy’s influence can be observed in the memory encoding phase.

Evaluating models of visual working memory in change detection: Discrete-slots or non-diagnostic data?

Evaluating models of visual working memory in change detection: Discrete-slots or non-diagnostic data?

Breaking the cardinal rule: The impact of interitem interaction and attentional priority on the cardinal biases in orientation working memory.

Although it is not typically assumed in influential models of visual working memory (WM), representations in WM are systematically biased by multiple factors. Orientation representations are biased away from the cardinal axis (i.e., cardinal bias) and they are biased away from or toward the other orientation simultaneously held in WM (i.e., interitem interaction). The present study investigated the extent to which these two bias mechanisms interact in WM. In Experiment 1, participants remembered two sequentially presented orientations and reproduced both orientations after a short delay. Cardinal biases were assessed separately for the trials where the two mechanisms produce biases in the same direction (i.e., congruent trials) and the trials where they produce biases in the opposite direction (i.e., incongruent trials). Whereas congruent trials exhibited a typical cardinal bias, incongruent trials exhibited no cardinal bias, demonstrating that the cardinal bias was canceled out by the interitem interaction. Follow-up experiments extended these results by manipulating attentional priority for the two orientations by means of precue (Experiment 2) and postcue (Experiment 3). In both experiments, attentionally prioritized items exhibited a typical cardinal bias irrespective of the congruency whereas attentionally unprioritized items exhibited a reversal of the cardinal bias in the incongruent trials, demonstrating that selective attention modulates the influence of the interitem interaction. Together, these results suggest that WM leverages information about specific stimuli and their relationship to support a given behavioral goal.

Memory Fidelity Reveals Qualitative Changes in Interactions Between Items in Visual Working Memory

Memory for objects in a display sometimes reveals attraction—the objects are remembered as more similar to one another than they actually were—and sometimes reveals repulsion—the objects are remembered as more different from one another. The conditions that lead to these opposing memory biases are poorly understood; there is no theoretical framework that explains these contrasting dynamics. In three experiments (each N = 30 adults), we demonstrate that memory fidelity provides a unifying dimension that accommodates the existence of both types of visual working memory interactions. We show that either attraction or repulsion can arise simply as a function of manipulations of memory fidelity. We also demonstrate that subjective ratings of fidelity predict the presence of attraction or repulsion on a trial-by-trial basis. We discuss how these results bear on computational models of visual working memory and contextualize these results within the literature of attraction and repulsion effects in long-term memory and perception.

How do neural processes give rise to cognition? Simultaneously predicting brain and behavior with a dynamic model of visual working memory.

There is consensus that activation within distributed functional brain networks underlies human thought. The impact of this consensus is limited, however, by a gap that exists between data-driven correlational analyses that specify where functional brain activity is localized using functional magnetic resonance imaging (fMRI), and neural process accounts that specify how neural activity unfolds through time to give rise to behavior. Here, we show how an integrative cognitive neuroscience approach may bridge this gap. In an exemplary study of visual working memory, we use multilevel Bayesian statistics to demonstrate that a neural dynamic model simultaneously explains behavioral data and predicts localized patterns of brain activity, outperforming standard analytic approaches to fMRI. The model explains performance on both correct trials and incorrect trials where errors in change detection emerge from neural fluctuations amplified by neural interaction. Critically, predictions of the model run counter to cognitive theories of the origin of errors in change detection. Results reveal neural patterns predicted by the model within regions of the dorsal attention network that have been the focus of much debate. The model-based analysis suggests that key areas in the dorsal attention network such as the intraparietal sulcus play a central role in change detection rather than working memory maintenance, counter to previous interpretations of fMRI studies. More generally, the integrative cognitive neuroscience approach used here establishes a framework for directly testing theories of cognitive and brain function using the combined power of behavioral and fMRI data. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Understanding occipital and parietal contributions to visual working memory: Commentary on Xu (2020)

ABSTRACT In her commentary, Xu [2020. Revisit once more the sensory storage account of visual working memory. Visual Cognition, 28(5–8), 433–446] admonishes the reader that “To have a full understanding of the cognitive mechanisms underlying VWM [visual working memory], both behavioral and neural evidence needs to be taken into account. This is a must, and not a choice, for any study that attempts to capture the nature of VWM” (p. 11). Although we don’t disagree with this statement, our overall assessment of this commentary is that it, itself, fails to satisfy several “musts” and, consequently, does not pose a serious challenge for the sensory recruitment framework for understanding visual working memory. These “musts” include accurately characterizing the framework being critiqued, not favouring verbal models and intuition at the expense of formal quantitative models, and providing even-handed interpretation of the work of others. We’ll conclude with a summary of how the sensory recruitment framework can be incorporated into a broader working model of visual working memory.

Incidental encoding of visual information in temporal reference frames in working memory

Visual events are structured in space and time, yet models of visual working memory (VWM) have largely relied on tasks emphasizing spatial aspects. Here, we show that temporal properties of visual events are incidentally encoded along with spatial properties. In five experiments, participants performed change-detection tasks, in which items had unique spatial and temporal coordinates at encoding. Crucially, neither space nor time was task-relevant. The key manipulation concerned the retrieval context: The test array was identical to the memory array either in its entire spatiotemporal structure, or only its spatial or temporal structure. Removing spatial or temporal information at retrieval resulted in costs, indicating that memory relied on both spatial and temporal context in which items were initially perceived. Encoding of spatiotemporal structure occurred incidentally, not strategically, as it was robust even when the retrieval context was perfectly predictable. However, spatial and temporal inter-item spacings influenced the weighting of spatial and temporal information: It favoured the domain in which items were more widely spaced, facilitating their individuation and, likely, access to representations. Across individuals, the weighting of spatial and temporal information varied substantially, but it remained consistent across sessions, suggesting stable preferences for coding in the spatial or temporal domain. No comparable incidental encoding occurred for other task-irrelevant feature dimensions (size or colour). We propose that temporal structure serves as fundamental a function in VWM as spatial structure, scaffolding events that unfold over time.

Psychophysical scaling reveals a unified theory of visual memory strength.

Almost all models of visual memory implicitly assume that errors in mnemonic representations are linearly related to distance in stimulus space. Here we show that neither memory nor perception are appropriately scaled in stimulus space; instead, they are based on a transformed similarity representation that is nonlinearly related to stimulus space. This result calls into question a foundational assumption of extant models of visual working memory. Once psychophysical similarity is taken into account, aspects of memory that have been thought to demonstrate a fixed working memory capacity of around three or four items and to require fundamentally different representations-across different stimuli, tasks and types of memory-can be parsimoniously explained with a unitary signal detection framework. These results have substantial implications for the study of visual memory and lead to a substantial reinterpretation of the relationship between perception, working memory and long-term memory.

Nonlinear effects of spatial connectedness implicate hierarchically structured representations in visual working memory

• Spatial connectedness non-linearly influences performance in change detection task. • Heterogeneity of stimulus structure hinders Visual Working Memory performance. • Representations in Visual Working Memory possess hierarchical part/whole structure. Five experiments investigated the role of spatial connectedness between a pair of objects presented in the change detection task for the actual capacity of visual working memory (VWM) in healthy young adults (total N = 405). Three experiments yielded a surprising nonlinear relationship between the proportion of pair-wise connected objects and capacity, with the highest capacity observed for homogenous displays, when either all objects were connected or disjointed. A drop in capacity, ranging from an average of a quarter of an object out of three objects maintained in VWM, was noted when only some objects were connected while others were disjointed. As indicated by another two experiments, this effect was specific to double-feature encoding, but disappeared when single visual features had to be memorized. No existing theoretical model of VWM can directly explain this novel effect. Overall, the nonlinear effect of spatial connectedness implies that representations in VWM possess hierarchical structure defined by wholes, parts, features, and their relations, and the heterogeneity of such a structure hinders VWM performance, while homogeneity facilitates it.

Functional dissociation of anterior cingulate cortex and intraparietal sulcus in visual working memory

Previous electrophysiological studies of lateralized visual working memory (VWM) identified an ERP component, defined as contralateral delay activity (CDA), directly modulated by the number of items held in memory. One of the main candidate as the cortical source of this ERP component is the inferior intraparietal sulcus (IPS). Moreover, previous neuroimaging studies put forth evidence for the presence of a distributed VWM network involving also prefrontal areas and in particular the anterior cingulate cortex (ACC). Nonetheless, the understanding of the functional role of ACC is still debated. We recorded the high-density EEG in 20 healthy participants undergoing a VWM and a control task. Explorative cluster-based permutation statistics confirmed the posterior memory load dependent CDA modulation, but also identified an additional anterior cluster of electrodes whose amplitude was modulated by memory load. The source reconstruction revealed a memory load dependent activation in the IPS but also in the ACC, suggesting that these two areas might be nodes of a fronto-parietal circuit underlying VWM maintenance. Crucially, parietal and prefrontal areas showed a temporal dissociation, since IPS was more engaged in the early phase of visual information storage while the ACC was more active during the late phase. This pattern suggests a functional dissociation between the parietal cortex, which is involved in encoding and storage of information, and prefrontal areas, subserving cognitive control processes, including the boosting and protection of information from decay. Remarkably, the connection strength between IPS and ACC predicted the individual number of items held in memory. These findings are discussed within the theoretical account of a neural distributed model of VWM.

I will never forget you: Direct forgetting and the 3-state model of visual working memory.

I will never forget you: Direct forgetting and the 3-state model of visual working memory.

Unambiguous evidence in favor of a signal detection model of visual working memory

Unambiguous evidence in favor of a signal detection model of visual working memory

Electroencephalographic evidence for improved visual working memory performance during standing and exercise.

While a substantial body of research has investigated the effects of aerobic exercise on cognitive performance, few have monitored exercise-concurrent cognitive processes via electroencephalography and fewer still using an event-related potential (ERP) approach. As such, little is known regarding how the temporal dynamics of cognitive processing are influenced during aerobic activity. Here, we aimed to elucidate the influence of aerobic exercise on the temporal dynamics of concurrent visual working memory (VWM) performance. Participants performed a VWM retro-cue task at rest and during aerobic exercise across two postural modalities: seated (using a stationary bicycle) and standing upright (using a treadmill). Three consecutive phases of the VWM processing pipeline were assessed by means of lateralized ERPs: access of VWM representations, response selection, and response execution. Aerobic exercise and upright posture were found to have significant effects on VWM performance, facilitating processing speed in the retro-cue task. This facilitation arose primarily at an intermediary stage between the phases of accessing VWM representations and response selection. Our findings hold implications not only for understanding the influence of aerobic activity on VWM, but also for contemporary models of VWM that are built exclusively on data recorded during stationary, seated conditions.