Contents Of Visual Working Memory Research Articles

Non-visual spatial strategies are effective for maintaining precise information in visual working memory

Visual working memory content is commonly thought to be composed of a precise visual representation of stimulus information (e.g., color, shape). Nevertheless, previous research has shown that individuals represent this visual information in different formats, historically dichotomized into “verbal” and “visual” formats. With growing popular knowledge of aphantasia, or the absence of sensory mental imagery, recent studies have demonstrated that individuals with aphantasia perform similarly to individuals with typical imagery on visual working memory tasks. This suggest that the use of non-visual strategies may be sufficient to perform visual working memory tasks, which were previously thought to be strictly visual. To investigate the effects of different strategies on performance in a visual working memory task, we recruited individuals across the visual imagery spectrum and tested their ability to identify relatively small (3°), medium (6°), or large (10°) changes in the degree of orientation of gratings held in working memory. Subsequently, participants indicated the extent to which they used five different strategies: visual, spatial, verbal, semantic, and sensorimotor. Results revealed that individuals with aphantasia and typical imagery performed similarly to each other across all task difficulty levels. Individuals with typical imagery dominantly used visuospatial strategies, but surprisingly, individuals with aphantasia overwhelmingly preferred the use of non-visual spatial and sensorimotor strategies over verbal strategies. These results suggest that non-visual spatial and sensorimotor strategies can be adopted in visual working memory tasks and these strategies are equally effective as visuospatial strategies. This calls for a rethinking of the “visual” versus “verbal” dichotomy, and provides evidence for the use of other non-visual mental representations in working memory tasks.

Task Termination Triggers Spontaneous Removal of Information From Visual Working Memory

Working memory (WM) is a goal-directed memory system that actively maintains a limited amount of task-relevant information to serve the current goal. By this definition, WM maintenance should be terminated after the goal is accomplished, spontaneously removing no-longer-relevant information from WM. Past studies have failed to provide direct evidence of spontaneous removal of WM content by allowing participants to engage in a strategic reallocation of WM resources to competing information within WM. By contrast, we provide direct neural and behavioral evidence that visual WM content can be largely removed less than 1 s after it becomes obsolete, in the absence of a strategic allocation of resources (total N = 442 adults). These results demonstrate that visual WM is intrinsically a goal-directed system, and spontaneous removal provides a means for capacity-limited WM to keep up with ever-changing demands in a dynamic environment.

Neuronal representation of visual working memory content in the primate primary visual cortex.

The human ability to perceive vivid memories as if they "float" before our eyes, even in the absence of actual visual stimuli, captivates the imagination. To determine the neural substrates underlying visual memories, we investigated the neuronal representation of working memory content in the primary visual cortex of monkeys. Our study revealed that neurons exhibit unique responses to different memory contents, using firing patterns distinct from those observed during the perception of external visual stimuli. Moreover, this neuronal representation evolves with alterations in the recalled content and extends beyond the retinotopic areas typically reserved for processing external visual input. These discoveries shed light on the visual encoding of memories and indicate avenues for understanding the remarkable power of the mind's eye.

Prospection of Potential Actions during Visual Working Memory Starts Early, Is Flexible, and Predicts Behavior.

For visual working memory to serve upcoming behavior, it is crucial that we prepare for the potential use of working-memory contents ahead of time. Recent studies have demonstrated how the prospection and planning for an upcoming manual action starts early after visual encoding, and occurs alongside visual retention. Here, we address whether such "output planning" in visual working memory flexibly adapts to different visual-motor mappings, and occurs even when an upcoming action will only potentially become relevant for behavior. Human participants (female and male) performed a visual-motor working memory task in which they remembered one or two colored oriented bars for later (potential) use. We linked, and counterbalanced, the tilt of the visual items to specific manual responses. This allowed us to track planning of upcoming behavior through contralateral attenuation of β band activity, a canonical motor-cortical EEG signature of manual-action planning. The results revealed how action encoding and subsequent planning alongside visual working memory (1) reflect anticipated task demands rather than specific visual-motor mappings, (2) occur even for actions that will only potentially become relevant for behavior, and (3) are associated with faster performance for the encoded item, at the expense of performance to other working-memory content. This reveals how the potential prospective use of visual working memory content is flexibly planned early on, with consequences for the speed of memory-guided behavior.SIGNIFICANCE STATEMENT It is increasingly studied how visual working memory helps us to prepare for the future, in addition to how it helps us to hold onto the past. Recent studies have demonstrated that the planning of prospective actions occurs alongside encoding and retention in working memory. We show that such early "output planning" flexibly adapts to varying visual-motor mappings, occurs both for certain and potential actions, and predicts ensuing working-memory guided behavior. These results highlight the flexible and future-oriented nature of visual working memory, and provide insight into the neural basis of the anticipatory dynamics that translate visual representations into adaptive upcoming behavior.

Parietal-driven visual working memory representation in occipito-temporal cortex

Human fMRI studies have documented extensively that the content of visual working memory (VWM) can be reliably decoded from fMRI voxel response patterns during the delay period in both the occipito-temporal cortex (OTC), including early visual areas (EVC), and the posterior parietal cortex (PPC).1,2,3,4 Further work has revealed that VWM signal in OTC is largely sustained by feedback from associative areas such as prefrontal cortex (PFC) and PPC.4,5,6,7,8,9 It is unclear, however, if feedback during VWM simply restores sensory representations initially formed in OTC or if it can reshape the representational content of OTC during VWM delay. Taking advantage of a recent finding showing that object representational geometry differs between OTC and PPC in perception,10 here we find that, during VWM delay, the object representational geometry in OTC becomes more aligned with that of PPC during perception than with itself during perception. This finding supports the role of feedback in shaping the content of VWM in OTC, with the VWM content of OTC more determined by information retained in PPC than by the sensory information initially encoded in OTC.

VWM-based bias in conscious access can be extended to a new sandwich masking task and real-life stimuli.

Previous studies using breaking continuous flash suppression observed that the content of visual working memory (VWM) influences the priority for accessing visual awareness. However, most studies have used simple stimuli, whereas real-life objects are typically more meaningful and contain more perceptual information than simple objects. In this study, we intermixed a delayed match-to-sample task to manipulate the content of VWM, and a breaking repeated masking suppression (b-RMS) task to investigate whether this memory-based effect in conscious access can be extended to a new sandwich masking task and real-life stimuli. The results revealed that memory-congruent objects broke RMS faster than incongruent objects for both simple and real-life objects. Specifically, for simple objects, color-matching targets broke RMS faster than color-mismatching targets, whereas state-matching targets broke RMS faster than state-mismatching targets for real-life objects. These results suggest that the faster detection of VWM-matching over VWM mismatching stimuli-which has been mostly studied using only one type of task (b-CFS) and stimulus (colored shapes)-extends to (1) yet another masking technique (b-RMS) and to (2) a novel stimulus type (real-life objects), providing evidence that memory-based biases in conscious access are a ubiquitous phenomenon.

Decoding of Working Memory Contents in Auditory Cortex Is Not Distractor-Resistant.

Working memory enables the temporary storage of relevant information in the service of behavior. Neuroimaging studies have suggested that sensory cortex is involved in maintaining contents in working memory. This raised the question of how sensory regions maintain memory representations during the exposure to distracting stimuli. Multivariate pattern analysis of fMRI signals in visual cortex has shown that the contents of visual working memory could be decoded concurrently with passively viewed distractors. The present fMRI study tested whether this finding extends to auditory working memory and to active distractor processing. We asked participants to memorize the pitch of a target sound and to compare it with a probe sound presented after a 13 s delay period. In separate conditions, we compared a blank delay phase (no distraction) with either passive listening to, or active processing of, an auditory distractor presented throughout the memory delay. Consistent with previous reports, pitch-specific memory information could be decoded in auditory cortex during the delay in trials without distraction. In contrast, decoding of target sounds in early auditory cortex dropped to chance level during both passive and active distraction. This was paralleled by memory performance decrements under distraction. Extending the analyses beyond sensory cortex yielded some evidence for memory content-specific activity in inferior frontal and superior parietal cortex during active distraction. In summary, while our findings question the involvement of early auditory cortex in the maintenance of distractor-resistant working memory contents, further research should elucidate the role of hierarchically higher regions.SIGNIFICANCE STATEMENT Information about sensory features held in working memory can be read out from hemodynamic activity recorded in human sensory cortices. Moreover, visual cortex can in parallel store visual content and process newly incoming, task-irrelevant visual input. The present study investigated the role of auditory cortex for working memory maintenance under distraction. While memorized sound frequencies could be decoded in auditory cortex in the absence of distraction, auditory distraction during the delay phase impaired memory performance and prevented decoding of information stored in working memory. Apparently, early auditory cortex is not sufficient to represent working memory contents under distraction that impairs performance. However, exploratory analyses indicated that, under distraction, higher-order frontal and parietal regions might contribute to content-specific working memory storage.

Mountains of memory in a sea of uncertainty: Sampling the external world despite useful information in visual working memory

A large part of research on visual working memory (VWM) has traditionally focused on estimating its maximum capacity. Yet, humans rarely need to load up their VWM maximally during natural behavior, since visual information often remains accessible in the external world. Recent work, using paradigms that take into account the accessibility of information in the outside world, has indeed shown that observers utilize only one or two items in VWM before sampling from the external world again. One straightforward interpretation of this finding is that, in daily behavior, much fewer items are memorized than the typically reported capacity limits. Here, we first investigate whether this lower reliance on VWM when information is externally accessible might instead reflect resampling before VWM is actually depleted. To this aim we devised an online task, in which participants copied a model (six items in a 4x4 grid; always accessible) in an adjacent empty 4x4 grid. A key aspect of our paradigm is that we (unpredictably) interrupted participants just before inspection of the model with a 2-alternative-forced-choice (2-AFC) question, probing their VWM content. Critically, we observed above-chance performance on probes appearing just before model inspection. This finding shows that the external world was resampled, despite VWM still containing relevant information. We then asked whether increasing the cost of sampling causes participants to load up more information in VWM or, alternatively, to squeeze out more information from VWM (at the cost of making more errors). To manipulate the cost of resampling, we made it more difficult (specifically, more time-consuming) to access the model. We show that with increased cost of accessing the model (which lead to fewer, but longer model inspections), participants could place more items correctly immediately after sampling, and they kept attempting to place items for longer after their first error. These findings demonstrate that participants both encoded more information in VWM and made attempts to squeeze out more information from VWM when sampling became more costly. We argue that human observers constantly evaluate how certain they are of their VWM contents, and only use that VWM content of which their certainty exceeds a context-dependent “action threshold”. This threshold, in turn, depends on the trade-off between the cost of resampling and the benefits of making an action. We argue that considering the interplay between the available VWM contents and a context-dependent action threshold, is key for reconciling the traditional VWM literature with VWM use in our day-to-day behavior.

Perceptual comparisons modulate memory biases induced by new visual inputs.

It is well-established that stimulus-specific information in visual working memory (VWM) can be systematically biased by new perceptual inputs. These memory biases are commonly attributed to interference that arises when perceptual inputs are physically similar to VWM contents. However, recent work has suggested that explicitly comparing the similarity between VWM contents and new perceptual inputs modulates the size of memory biases above and beyond stimulus-driven effects. Here, we sought to directly investigate this modulation hypothesis by comparing the size of memory biases following explicit comparisons to those induced when new perceptual inputs are ignored (Experiment 1) or maintained in VWM alongside target information (Experiment 2). We found that VWM reports showed larger attraction biases following explicit perceptual comparisons than when new perceptual inputs were ignored or maintained in VWM. An analysis of participants' perceptual comparisons revealed that memory biases were amplified after perceptual inputs were endorsed as similar-but not dissimilar-to one's VWM representation. These patterns were found to persist even after accounting for variability in the physical similarity between the target and perceptual stimuli across trials, as well as the baseline memory precision between the distinct task demands. Together, these findings illustrate a causal role of perceptual comparisons in modulating naturally-occurring memory biases.

Tracking Proactive Interference in Visual Memory.

The current contents of visual working memory can be disrupted by previously formed memories. This phenomenon is known as proactive interference, and it can be used to index the availability of old memories. However, there is uncertainty about the robustness and lifetime of proactive interference, which raises important questions about the role of temporal factors in forgetting. The present study assessed different factors that were expected to influence the persistence of proactive interference over an inter-trial interval in the visual recent probes task. In three experiments, participants encoded arrays of targets and then determined whether a single probe matched one of those targets. On some trials, the probe matched an item from the previous trial (a “recent negative”), whereas on other trials the probe matched a more distant item (a “non-recent negative”). Prior studies have found that recent negative probes can increase errors and slow response times in comparison to non-recent negative probes, and this offered a behavioral measure of proactive interference. In Experiment 1, factors of array size (the number of targets to be encoded) and inter-trial interval (300 ms vs. 8 s) were manipulated in the recent probes task. There was a reduction in proactive interference when a longer delay separated trials on one measure, but only when participants encoded two targets. When working memory capacity was strained by increasing the array size to four targets, proactive interference became stronger after the long delay. In Experiment 2, the inter-trial interval length was again manipulated, along with stimulus novelty (the number of stimuli used in the experiment). Proactive interference was modestly stronger when a smaller number of stimuli were used throughout the experiment, but proactive interference was minimally affected by the inter-trial interval. These findings are problematic for temporal models of forgetting, but Experiment 3 showed that proactive interference also resisted disruption produced by a secondary task presented within the inter-trial interval. Proactive interference was constantly present and generally resilient to the different manipulations. The combined data suggest a relatively durable, passive representation that can disrupt current working memory under a variety of different circumstances.

Guidance of attention from visual working memory is feature-based, not object-based: Implications for models of feature binding.

A classic question in visual working memory (VWM) research is whether features from the same object are bound directly in an integrated representation or are maintained separately and bound only indirectly though shared location. Here, we examined this question using a novel method that probed the effects of VWM on the guidance of attention (rather than requiring explicit access to VWM content, as has typically been used). Participants remembered two color-shape conjunction objects. During a retention-interval search task, they searched for a target letter among distractor letters superimposed over color-shape conjunction items. There were two critical conditions. In the same-object-match condition, one search item matched both the color and shape of a single remembered object. In the different-object-match condition, one search item matched the color from one remembered object and the shape from the other. Robust effects of VWM-based guidance were observed, both when probing the incidental guidance of attention (Experiments 1 and 2) and the strategic guidance of attention (Experiment 3). Critically, in none of the experiments was the magnitude of guidance greater for same-object-match than for different-object-match. The results indicate that the representational units of guidance from VWM are individual features rather than integrated objects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Interactions Between Visual Working Memory, Attention, and Color Categories: A Pupillometry Study.

Recent studies have found that visual working memory (VWM) for color shows a categorical bias: observers typically remember colors as more prototypical to the category they belong to than they actually are. Here, we further examine color-category effects on VWM using pupillometry. Participants remembered a color for later reproduction on a color wheel. During the retention interval, a colored probe was presented, and we measured the pupil constriction in response to this probe, assuming that the strength of constriction reflects the visual saliency of the probe. We found that the pupil initially constricted most strongly for non-matching colors that were maximally different from the memorized color; this likely reflects a lack of visual adaptation for these colors, which renders them more salient than memory-matching colors (which were shown before). Strikingly, this effect reversed later in time, such that pupil constriction was more prolonged for memory-matching colors as compared to non-matching colors; this likely reflects that memory-matching colors capture attention more strongly, and perhaps for a longer time, than non-matching colors do. We found no effects of color categories on pupil constriction: after controlling for color distance, (non-matching) colors from the same category as the memory color did not result in a different pupil response as compared to colors from a different category; however, we did find that behavioral responses were biased by color categories. In summary, we found that pupil constriction to colored probes reflects both visual adaptation and VWM content, but, unlike behavioral measures, is not notably affected by color categories.

Joint representation of working memory and uncertainty in human cortex

Neural representations of visual working memory (VWM) are noisy, and thus, decisions based on VWM are inevitably subject to uncertainty. However, the mechanisms by which the brain simultaneously represents the content and uncertainty of memory remain largely unknown. Here, inspired by the theory of probabilistic population codes, we test the hypothesis that the human brain represents an item maintained in VWM as a probability distribution over stimulus feature space, thereby capturing both its content and uncertainty. We used a neural generative model to decode probability distributions over memorized locations from fMRI activation patterns. We found that the mean of the probability distribution decoded from retinotopic cortical areas predicted memory reports on a trial-by-trial basis. Moreover, in several of the same mid-dorsal stream areas, the spread of the distribution predicted subjective trial-by-trial uncertainty judgments. These results provide evidence that VWM content and uncertainty are jointly represented by probabilistic neural codes.

Perceptual stimuli with novel bindings interfere with visual working memory

What influences the extent to which perceptual information interferes with the contents of visual working memory? In two experiments using a combination of change detection and continuous reproduction tasks, I show that binding novelty is a key factor in producing interference. In Experiment 2, participants viewed arrays of colored circles, then completed consecutive change detection and recall tests of their memory for stochastically independent items from the same array. When the probe used in the change detection test was novel (i.e., required a “change” response), subsequent recall performance was worse than in trials with matching (i.e., “no change”) probes, irrespective of whether or not the same item was tested in both phases. In Experiment 2, participants viewed arrays of oriented arrows, then completed a change detection (requiring memory) or direction judgement (not requiring memory) test, followed by recalling a stochastically independent item. Again, novel probes in the first phase led to worse recall, irrespective of whether the initial task required memory. This effect held whether the probe was wholly novel (i.e., a new feature presented at any location) or simply involved a novel binding (i.e., an old feature presented at a new location). These findings highlight the role of novelty in visual interference, consistent with the assumptions of computational models of WM, and suggest that new bindings of old information are sufficient to produce such interference.

Visual working memory load plays limited, to no role in encoding distractor objects during visual search

ABSTRACT Previous research raised the counterintuitive hypothesis that searching for multiple potential targets leads to increased incidental encoding of distractors. Are these previously reported findings due to increased visual working memory (VWM) engagement, or less precise target templates? In four experiments, we examined the effect of VWM load during visual search on incidental encoding of distractors. Consecutive target repetitions indirectly reduce template-related VWM demands but failed to reduce recognition for distractors relative to conditions where the targets were novel. Distractors that were subsequently recognized attracted longer cumulative dwell time, regardless of search condition. When placed in a dual-task situation where search was performed while holding a working memory load, recognition for distractors was marginally improved relative to a search task without additional VWM demands. We ruled out the possibility that the dual-task was not sufficiently difficult to trigger the scrutiny of distractors required for significant encoding benefits by showing a decrement to encoding when search time was limited. This suggests that widening the attentional set is the crucial factor in improved incidental encoding given that observers can assign differential status to various contents of VWM. Thus, utilizing VWM resources in general appears insufficient to meaningfully improve incidental memory.

Visual working memory content influences correspondence processes.

Representing objects as continuous across time requires the establishment of correspondence, whereby current stimuli are represented as deriving from the same object as earlier stimuli. Spatiotemporal continuity and surface-feature similarity play important roles in these correspondence processes. Because objects are often represented across extended periods of time, visual working memory (VWM) content should also play a role in object correspondence. We tested this prediction using Ternus motion. Displays consisted of three-disk arrays that shifted horizontally by one position between frames. Depending on how correspondence is resolved, Ternus displays are perceived as group motion, where all three disks appear to move together, or element motion, where one disk appears to jump across the others. Reports of which motion is perceived provide an index of how correspondence was resolved. Ternus displays were adapted such that the color of some disks biased element motion while the color of others biased group motion. Maintaining one or the other of the colors in VWM for later report systematically biased which type of motion was perceived (Experiments 1 and 2). When color was incidental to the VWM task, however, it did not (Experiment 3). These results confirm that VWM content contributes to object correspondence. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How retaining objects containing multiple features in visual working memory regulates the priority for access to visual awareness.

The content of visual working memory influences the access to visual awareness. Thus far, research has focused on retention of a single feature, whereas memoranda in real life typically contain multiple features. Here, we intermixed a delayed match-to-sample task to manipulate VWM content, and a breaking Continuous Flash Suppression (b-CFS) task to measure prioritization for visual awareness. Observers memorized either the color (Exp. 1), the shape (Exp. 2) or both the features (Exp. 3) of an item and indicated the location of a suppressed target. We observed that color-matching targets broke suppression faster than color-mismatching targets both when color was memory relevant or irrelevant. Shape only impacted priority for visual awareness through an interaction with color. We conclude that: (1) VWM can regulate the priority of visual information to access visual awareness along a single feature dimension; (2) different features of a memorandum vary in their potency to impact access to visual awareness, and the more dominant feature may even suppress the effect of the less dominant feature; (3) even stimuli that match an irrelevant feature dimension of the memorandum can be prioritized for visual awareness.

Stimulus-Specific Visual Working Memory Representations in Human Cerebellar Lobule VIIb/VIIIa.

fMRI research has revealed that cerebellar lobule VIIb/VIIIa exhibits load-dependent activity that increases with the number of items held in visual working memory (VWM). However, it remains unclear whether these cerebellar responses reflect processes specific to VWM or more general visual attentional mechanisms. To investigate this question, we examined whether cerebellar activity during the delay period of a VWM task is selective for stimuli held in working memory. A sample of male and female human subjects performed a VWM continuous report task in which they were retroactively cued to remember the direction of motion of moving dot stimuli. Cerebellar lobule VIIb/VIIIa delay-period activation accurately decoded the direction of the remembered stimulus, as did frontal and parietal regions of the dorsal attention network. Arguing against a motor explanation, no other cerebellar area exhibited stimulus specificity, including the oculomotor vermis, a key area associated with eye movement control. Finer-scale analysis revealed that the medial portion of lobule VIIb and to a lesser degree the lateral most portion of lobules VIIb and VIIIa, which exhibit robust resting state connectivity with frontal and parietal regions of the dorsal attention network, encoded the identity of the remembered stimulus, while intermediate portions of lobule VIIb/VIIIa did not. These findings of stimulus-specific coding of VWM within lobule VIIb/VIIIa indicate for the first time that the distributed network responsible for the encoding and maintenance of mnemonic representations extends to the cerebellum.SIGNIFICANCE STATEMENT There is considerable debate concerning where in the brain the contents of visual working memory (VWM) are stored. To date, this literature has primarily focused on the role of regions located within cerebral cortex. There is growing evidence for cerebellar involvement in higher-order cognitive functions including working memory. While the cerebellum has been previously shown to be recruited by VWM paradigms, it is unclear whether any portion of cerebellum actively encodes and maintains mnemonic representations. The present study demonstrates that cerebellar lobule VIIb/VIIIa activity patterns are selective for remembered stimuli and that this selectivity persists in the absence of perceptual input. These findings provide novel evidence for the participation of cerebellar structures in the persistent storage of visual information.

How the content of visual working memory regulates the priority for access to visual awareness for memoranda with multiple features

How the content of visual working memory regulates the priority for access to visual awareness for memoranda with multiple features

Visual working memory content influences correspondence processes [Research data and description

Visual working memory content influences correspondence processes [Research data and description