Visual Working Memory Encoding Research Articles

External cues improve visual working memory encoding in the presence of salient distractors in schizophrenia.

People with schizophrenia (PSZ) are impaired in attentional prioritization of non-salient but relevant stimuli over salient distractors during visual working memory (VWM) encoding. Conversely, guidance of top-down attention by external predictive cues is intact. Yet, it is unknown whether this preserved ability can help PSZ encode more information in the presence of salient distractors. We employed a visuospatial change-detection task using four Gabor patches with differing orientations in 66 PSZ and 74 healthy controls (HCS). Two Gabor patches flickered which were designated either as targets or distractors and either a predictive or a non-predictive cue was displayed to manipulate top-down attention, resulting in four conditions. We observed significant effects of group, salience and cue as well as significant interactions of salience by cue, group by salience and group by cue. Across all conditions, PSZ stored significantly less information in VWM than HCS. PSZ stored significantly less non-flickering than flickering information with a non-predictive cue. However, PSZ stored significantly more flickering and non-flickering information with a predictive cue. Our findings indicate that control of attentional selection is impaired in schizophrenia. We demonstrate that additional top-down information significantly improves performance in PSZ. The observed deficit in attentional control suggests a disturbance of GABAergic inhibition in early visual areas. Moreover, our findings are indicative of a mechanism for enhancing attentional control in PSZ, which could be utilized by pro-cognitive interventions. Thus, the current paradigm is suitable to reveal both preserved and compromised cognitive component processes in schizophrenia.

A discrete component in visual working memory encoding.

Working memory (WM) is a central cognitive bottleneck, which has primarily been attributed to its well-known storage limit. However, relatively little is known about the processing limit during the initial memory encoding stage, which may also constrain various cognitive processes. The present study introduces a novel method using dynamic stimulus presentation and hierarchical Bayesian modeling to quantitatively estimate visual WM encoding speed. Participants performed a delayed-estimation task with two memory items continuously changing color hues in perceptually unnoticeable steps. Across three experiments, the recall errors systematically shifted toward the direction of color change, providing a proxy measure of encoding speed. Importantly, the observed shifts were best characterized by a temporal lag during the encoding of different items, supported by a mixture of two distributions with credibly distinct encoding times. A supplementary model-free analysis further confirmed the discrete encoding component in visual WM for multiple items. These findings shed light on the temporal dynamics of WM encoding processes. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Location has a privilege, but it is limited: Evidence from probing task-irrelevant location.

We investigated the extents of automaticity in location and orientation encoding in visual working memory (VWM) by manipulating their task relevance and assessing the amount of resource recruited by their encoding. Across five experiments, participants were surprised with a location report trial (Experiment 1A, 2A, and 3) or an orientation report trial (Experiment 2A and 2B) at a point when only the item's color had been task-relevant. This was followed by control trials to assess the memory quality of color when location or orientation had become task-relevant. We found the surprise trial performance to be significantly worse than the first control trial for both location and orientation, although to a greater extent for orientation for which there was virtually no measurable information from the subjects' reports. This was the case even when encoding was the only incidental memory process before the control trials (Experiment 2A and 2B), and the surprise memory costs cannot be attributed to the unexpectedness inherent to the surprise question (Experiment 3). The control trials revealed a consistent reduction of color memory only in the orientation experiments. These results suggest that although location encoding is more automatic than orientation, neither is encoded in a fully automatic manner. Our results show that incidentally encoded location is only coarse-grained, constraining the spatial precision of space-based indexing systems. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Differential aspects of attention predict the depth of visual working memory encoding: Evidence from pupillometry.

What determines how much one encodes into visual working memory? Traditionally, encoding depth is considered to be indexed by spatiotemporal properties of gaze, such as gaze position and dwell time. Although these properties inform about where and how long one looks, they do not necessarily inform about the current arousal state or how strongly attention is deployed to facilitate encoding. Here, we found that two types of pupillary dynamics predict how much information is encoded during a copy task. The task involved encoding a spatial pattern of multiple items for later reproduction. Results showed that smaller baseline pupil sizes preceding and stronger pupil orienting responses during encoding predicted that more information was encoded into visual working memory. Additionally, we show that pupil size reflects not only how much but also how precisely material is encoded. We argue that a smaller pupil size preceding encoding is related to increased exploitation, whereas larger pupil constrictions signal stronger attentional (re)orienting to the to-be-encoded pattern. Our findings support the notion that the depth of visual working memory encoding is the integrative outcome of differential aspects of attention: how alert one is, how much attention one deploys, and how long it is deployed. Together, these factors determine how much information is encoded into visual working memory.

The role of encoding history in visual working memory encoding: Evidence from attribute amnesia

The role of encoding history in visual working memory encoding: Evidence from attribute amnesia

Encoding specificity instead of online integration of real-world spatial regularities for objects in working memory

Most objects show high degrees of spatial regularity (e.g. beach umbrellas appear above, not under, beach chairs). The spatial regularities of real-world objects benefit visual working memory (VWM), but the mechanisms behind this spatial regularity effect remain unclear. The “encoding specificity” hypothesis suggests that spatial regularity will enhance the visual encoding process but will not facilitate the integration of information online during VWM maintenance. The “perception-alike” hypothesis suggests that spatial regularity will function in both visual encoding and online integration during VWM maintenance. We investigated whether VWM integrates sequentially presented real-world objects by focusing on the existence of the spatial regularity effect. Throughout five experiments, we manipulated the presentation (simultaneous vs. sequential) and regularity (with vs. without regularity) of memory arrays among pairs of real-world objects. The spatial regularity of memory objects presented simultaneously, but not sequentially, improved VWM performance. We also examined whether memory load, verbal suppression and masking, and memory array duration hindered the spatial regularity effect in sequential presentation. We found a stable absence of the spatial regularity effect, suggesting that the participants were unable to integrate real-world objects based on spatial regularities online. Our results support the encoding specificity hypothesis, wherein the spatial regularity of real-world objects can enhance the efficiency of VWM encoding, but VWM cannot exploit spatial regularity to help organize sampled sequential information into meaningful integrations.

Object-based encoding in visual working memory: A critical revisit.

Visual working memory (VWM) is responsible for the temporal retention and manipulation of visual information. It has been suggested that VWM employs an object-based encoding (OBE) manner to extract highly discriminable information from visual perception: Whenever one feature dimension of the objects is selected for entry into VWM, the other task-irrelevant highly discriminable dimension is also extracted into VWM involuntarily. However, the task-irrelevant feature in OBE studies might reflect a high capacity fragile VWM (FVWM) trace that stores maskable sensory representations. To directly compare the VWM storage hypothesis and the FVWM storage hypothesis, we used a unique characteristic of FVWM that the representations in FVWM could be erased by backward masks presented at the original locations of the memory array. We required participants to memorise the orientations of three coloured bars while ignoring their colours, and presented backward masks during the VWM maintenance interval. In four experiments, we consistently observed that the OBE occurs regardless of the presentation of the backward masks, except when even the task-relevant features in VWM were significantly interrupted by immediate backward masks, suggesting that the task-irrelevant features of objects are stored in VWM rather than in FVWM.

Event-based encoding of biological motion and location in visual working memory.

We make use of discrete yet meaningful events to orient ourselves to the dynamic environment. Among these events, biological motion, referring to the movements of animate entities, is one of the most biologically salient. We usually encounter biological motions of multiple human beings taking place simultaneously at distinct locations. How we encode biological motions into visual working memory (VWM) to form a coherent experience of the external world and guide our social behaviour remains unclear. This study for the first time addressed the VWM encoding mechanism of biological motions and their corresponding locations. We tested an event-based encoding hypothesis for biological motion and location: When one element of an event is required to be memorised, the irrelevant element of an event will also be extracted into VWM. We presented participants with three biological motions at different locations and required them to memorise only the biological motions or their locations while ignoring the other dimension. We examined the event-based encoding by probing a distracting effect: If the event-based encoding took place, the change of irrelevant dimension in the probe would lead to a significant distraction and impair the performance of detecting target dimension. We found significant distracting effects, which lasted for 3 s but vanished at 6 s, regardless of the target dimension (biological motions vs. locations, Experiment 1) and the exposure time of memory array (1 s vs. 3 s, Experiment 2). These results together support an event-based encoding mechanism during VWM encoding of biological motions and their corresponding locations.

Neural markers of visual working memory encoding and maintenance track attentional prioritization

Neural markers of visual working memory encoding and maintenance track attentional prioritization

Contralateral delay activity tracks the storage of visually presented letters and words.

Electrophysiological studies have demonstrated that the maintenance of items in visual working memory (VWM) is indexed by the contralateral delay activity (CDA), which increases in amplitude as the number of objects to remember increases, plateauing at VWM capacity. Previous work has primarily utilized simple visual items, such as colored squares or picture stimuli. Despite the frequent use of verbal stimuli in seminal investigations of visual attention and memory, it is unknown whether temporary storage of letters and words also elicit a typical load-sensitive CDA. Given their close associations with language and phonological codes, it is possible that participants store these stimuli phonologically, and not visually. Participants completed a standard visual change-detection task while their ERPs were recorded. Experiment 1 compared the CDA elicited by colored squares compared to uppercase consonants, and Experiment 2 compared the CDA elicited by words compared to colored bars. Behavioral accuracy of change detection decreased with increasing set size for colored squares, letters, and words. We found that a capacity-limited CDA was present for colored squares, letters, and word arrays, suggesting that the visual codes for letters and words were maintained in VWM, despite the potential for transfer to verbal working memory. These results suggest that, despite their verbal associations, letters and words elicit the electrophysiological marker of VWM encoding and storage.

P.3.010 - Imbalance in the deployment of neural resources during visual working memory encoding in schizophrenia as a possible intermediate phenotype

Visual working memory (VWM) deficits are a prominent impairment in patients with schizophrenia (SCZ) and are associated with an inefficient encoding process [1] and with neuronal dysfunctions [2]. Both behavioural and neurofunctional impairments can be observed in unaffected first-degree relatives of patients (REL) as well and are thus discussed as a possible intermediate phenotype of the disorder [3,4]. In this study we investigated the characteristics of WM encoding and its neural substrates in SCZ and REL to evaluate their potential role as an intermediate phenotype of the disorder. We examined n = 25 SCZ, n = 22 REL and n = 25 healthy control subjects (CON). All subjects underwent whole brain fMRI scans in a 3 Tesla MR system. During the fMRI scan, a VWM change-detection task was presented, which showed a sample array consisting of 3 red lines of different orientation that should be memorized. After a delay interval of either 100ms, 400ms or 800ms (SOA = stimulus onset asynchrony) a pattern mask was shown, followed by another delay period and a test array. The longer the delay interval until masking, the more durable the VWM representations were expected to be. Response accuracy served as an estimate of the grade of VWM encoding. The behavioural data was analyzed using a two-way univariate ANOVA with repeated measures and post hoc tests. For group statistics of the event-related fMRI data we conducted a random-effects GLM (RFX-GLM) and a subsequent ANOVA with 'group' as a between-subjects factor (3 levels) and 'SOA' as within-subjects factor (3 levels). For behavioural performance in the VWM task we found significant main effects of 'group' (p = 0.026) and 'SOA' (p = 0.013). The longer the SOA interval, the greater the response accuracy across groups (significant linear trend; p = 0.009). Response accuracy was significantly lower in SCZ compared to CON (p = 0.011) and REL (p = 0.041). We observed aberrant neural activation during VWM encoding for both SCZ and REL compared to CON: across SOA conditions SCZ showed widespread neural deactivation, including the DLPFC and precentral gyrus, as well as increased activity of the anterior cingulate. In REL, we observed a shift in neural activation in regard to the length of the SOA interval: at SOA 100 REL showed only decreased activity in the superior frontal gyrus and fusiform gyrus; at SOA 400 we observed decreased activity in the middle frontal gyrus and increased activity in the superior frontal gyrus; and at SOA 800 REL showed only increased activity in the inferior frontal gyrus. The findings indicate that impaired VWM in SCZ may emerge from an insufficient engagement of the functional WM network with concomitant anterior cingulate hyperactivity during encoding. In REL, the shift from decreased to increased activity might reflect a compensatory process: hypoactivity might point towards fundamental neural dysfunctions—at lighter processing demands local hyperactivity might ensure accurate VWM performance. Neural dysfunctions during VWM encoding should thus be considered as a candidate intermediate phenotype of SCZ.

Fechner's law in metacognition: A quantitative model of visual working memory confidence.

Although visual working memory (VWM) has been studied extensively, it is unknown how people form confidence judgments about their memories. Peirce (1878) speculated that Fechner's law-which states that sensation is proportional to the logarithm of stimulus intensity-might apply to confidence reports. Based on this idea, we hypothesize that humans map the precision of their VWM contents to a confidence rating through Fechner's law. We incorporate this hypothesis into the best available model of VWM encoding and fit it to data from a delayed-estimation experiment. The model provides an excellent account of human confidence rating distributions as well as the relation between performance and confidence. Moreover, the best-fitting mapping in a model with a highly flexible mapping closely resembles the logarithmic mapping, suggesting that no alternative mapping exists that accounts better for the data than Fechner's law. We propose a neural implementation of the model and find that this model also fits the behavioral data well. Furthermore, we find that jointly fitting memory errors and confidence ratings boosts the power to distinguish previously proposed VWM encoding models by a factor of 5.99 compared to fitting only memory errors. Finally, we show that Fechner's law also accounts for metacognitive judgments in a word recognition memory task, which is a first indication that it may be a general law in metacognition. Our work presents the first model to jointly account for errors and confidence ratings in VWM and could lay the groundwork for understanding the computational mechanisms of metacognition. (PsycINFO Database Record

Local and interregional alpha EEG dynamics dissociate between memory for search and memory for recognition

Attention during visual search is thought to be guided by an active visual working memory (VWM) representation of the search target. We tested the hypothesis that a VWM representation used for searching a target among competing information (a “search template”) is distinct from VWM representations used for simple recognition tasks, without competition. We analyzed EEG from 20 human participants while they performed three different VWM-based visual detection tasks. All tasks started with identical lateralized VWM cues, but differed with respect to the presence and nature of competing distractors during the target display at test, where participants performed a simple recognition task without distractors, or visual search in pop-out (distinct) and serial (non-distinct) search displays. Performance was worst for non-distinct search, and best for simple recognition. During the one second delay period between cue and test, we observed robust suppression of EEG dynamics in the alpha (8–14Hz) band over parieto-occipital sites contralateral to the relevant VWM item, both in terms of local power as well as interregional phase synchrony within a posterior-parietal network. Importantly, these lateralization dynamics were more strongly expressed prior to search compared to simple recognition. Furthermore, before the VWM cue, alpha phase synchrony between prefrontal and mid-posterior-parietal sites was strongest for non-distinct search, reflecting enhanced anticipatory control prior to VWM encoding. Directional connectivity analyses confirmed this effect to be in an anterior-to-posterior direction. Together, these results provide evidence for frontally mediated top-down control of VWM in preparation of visual search.

Induced and Evoked Human Electrophysiological Correlates of Visual Working Memory Set-Size Effects at Encoding.

The ability to encode, store, and retrieve visually presented objects is referred to as visual working memory (VWM). Although crucial for many cognitive processes, previous research reveals that VWM strictly capacity limited. This capacity limitation is behaviorally observable in the set size effect: the ability to successfully report items in VWM asymptotes at a small number of items. Research into the neural correlates of set size effects and VWM capacity limits in general largely focus on the maintenance period of VWM. However, we previously reported that neural resources allocated to individual items during VWM encoding correspond to successful VWM performance. Here we expand on those findings by investigating neural correlates of set size during VWM encoding. We hypothesized that neural signatures of encoding-related VWM capacity limitations should be differentiable as a function of set size. We tested our hypothesis using High Density Electroencephalography (HD-EEG) to analyze frequency components evoked by flickering target items in VWM displays of set size 2 or 4. We found that set size modulated the amplitude of the 1st and 2nd harmonic frequencies evoked during successful VWM encoding across frontal and occipital-parietal electrodes. Frontal sites exhibited the most robust effects for the 2nd harmonic (set size 2 > set size 4). Additionally, we found a set-size effect on the induced power of delta-band (1–4 Hz) activity (set size 2 > set size 4). These results are consistent with a capacity limited VWM resource at encoding that is distributed across to-be-remembered items in a VWM display. This resource may work in conjunction with a task-specific selection process that determines which items are to be encoded and which are to be ignored. These neural set size effects support the view that VWM capacity limitations begin with encoding related processes.

Adverse orienting effects on visual working memory encoding and maintenance

Visual working memory (VWM) representations can be strengthened by pre-cues presented before, and retro-cues presented after, the memory display, providing evidence that attentional orienting plays a role in memory encoding and maintenance. It is unknown whether attentional orienting to VWM stimuli can also have adverse effects (known as inhibition of return; IOR), as has been found for perceptual-cueing tasks. If so, this would provide further evidence for common attentional orienting mechanisms for mnemonic and perceptual representations. In Experiment 1, we used pre-cueing and demonstrated an increased encoding probability, but not precision, at short SOAs, but probability decreased at long SOAs, reminiscent of the classic IOR findings. In Experiment 2, we used retro-cueing and showed that it improved memory performance, unless attention was cued back to the center of the display by a second cue. In this case, the deleterious effects were on precision, indicating that the item was still retained, but its quality of representation suffered. Together, these results provide further evidence for universal spatial attentional mechanisms operating on perceptual as well as mnemonic representations.

Electrophysiology reveals different mechanisms of attentional filtering during visual working memory encoding and retention

Electrophysiology reveals different mechanisms of attentional filtering during visual working memory encoding and retention

The relationship between visual attention and visual working memory encoding: A dissociation between covert and overt orienting.

There is substantial debate over whether visual working memory (VWM) and visual attention constitute a single system for the selection of task-relevant perceptual information or whether they are distinct systems that can be dissociated when their representational demands diverge. In the present study, we focused on the relationship between visual attention and the encoding of objects into VWM. Participants performed a color change-detection task. During the retention interval, a secondary object, irrelevant to the memory task, was presented. Participants were instructed either to execute an overt shift of gaze to this object (Experiments 1-3) or to attend it covertly (Experiments 4 and 5). Our goal was to determine whether these overt and covert shifts of attention disrupted the information held in VWM. We hypothesized that saccades, which typically introduce a memorial demand to bridge perceptual disruption, would lead to automatic encoding of the secondary object. However, purely covert shifts of attention, which introduce no such demand, would not result in automatic memory encoding. The results supported these predictions. Saccades to the secondary object produced substantial interference with VWM performance, but covert shifts of attention to this object produced no interference with VWM performance. These results challenge prevailing theories that consider attention and VWM to reflect a common mechanism. In addition, they indicate that the relationship between attention and VWM is dependent on the memorial demands of the orienting behavior. (PsycINFO Database Record

Different effects of color-based and location-based selection on visual working memory.

In the present study, we investigated how feature- and location-based selection influences visual working memory (VWM) encoding and maintenance. In Experiment 1, cue type (color, location) and cue timing (precue, retro-cue) were manipulated in a change detection task. The stimuli were color-location conjunction objects, and binding memory was tested. We found a significantly greater effect for color precues than for either color retro-cues or location precues, but no difference between location pre- and retro-cues, consistent with previous studies (e.g., Griffin & Nobre in Journal of Cognitive Neuroscience, 15, 1176-1194, 2003). We also found no difference between location and color retro-cues. Experiment 2 replicated the color precue advantage with more complex color-shape-location conjunction objects. Only one retro-cue effect was different from that in Experiment 1: Color retro-cues were significantly less effective than location retro-cues in Experiment 2, which may relate to a structural property of multidimensional VWM representations. In Experiment 3, a visual search task was used, and the result of a greater location than color precue effect suggests that the color precue advantage in a memory task is related to the modulation of VWM encoding rather than of sensation and perception. Experiment 4, using a task that required only memory for individual features but not for feature bindings, further confirmed that the color precue advantage is specific to binding memory. Together, these findings reveal new aspects of the interaction between attention and VWM and provide potentially important implications for the structural properties of VWM representations.

The steady-state visual evoked potential reveals neural correlates of the items encoded into visual working memory

Visual working memory (VWM) capacity limitations are estimated to be ~4 items. Yet, it remains unclear why certain items from a given memory array may be successfully retrieved from VWM and others are lost. Existing measures of the neural correlates of VWM cannot address this question because they measure the aggregate processing of the entire stimulus array rather than neural signatures of individual items. Moreover, this cumulative processing is usually measured during the delay period, thereby reflecting the allocation of neural resources during VWM maintenance. Here, we use the steady-state visual evoked potential (SSVEP) to identify the neural correlates of individual stimuli at VWM encoding and test two distinct hypotheses: the focused-resource hypothesis and the diffuse-resource hypothesis, for how the allocation of neural resources during VWM encoding may contribute to VWM capacity limitations. First, we found that SSVEP amplitudes were larger for stimuli that were later remembered than for items that were subsequently forgotten. Second, this pattern generalized so that the SSVEP amplitudes were also larger for the unprobed stimuli in correct compared to incorrect trials. These data are consistent with the diffuse-resource view in which attentional resources are broadly allocated across the whole stimulus array. These results illustrate the important role encoding mechanisms play in limiting the capacity of VWM.

The When and Where of Working Memory Dysfunction in Early-Onset Schizophrenia—A Functional Magnetic Resonance Imaging Study

Behavioral evidence indicates that working memory (WM) in schizophrenia is already impaired at the encoding stage. However, the neurophysiological basis of this primary deficit remains poorly understood. Using event-related fMRI, we assessed differences in brain activation and functional connectivity during the encoding, maintenance and retrieval stages of a visual WM task with 3 levels of memory load in 17 adolescents with early-onset schizophrenia (EOS) and 17 matched controls. The amount of information patients could store in WM was reduced at all memory load levels. During encoding, activation in left ventrolateral prefrontal cortex (VLPFC) and extrastriate visual cortex, which in controls positively correlated with the amount of stored information, was reduced in patients. Additionally, patients showed disturbed functional connectivity between prefrontal and visual areas. During retrieval, right inferior VLPFC hyperactivation was correlated with hypoactivation of left VLPFC in patients during encoding. Visual WM encoding is disturbed by a failure to adequately engage a visual-prefrontal network critical for the transfer of perceptual information into WM. Prefrontal hyperactivation appears to be a secondary consequence of this primary deficit. Isolating the component processes of WM can lead to more specific neurophysiological markers for translational efforts seeking to improve the treatment of cognitive dysfunction in schizophrenia.