The attentional selection in visual search within short-term memory representations

Abstract

During the perception of a visual scene, attention can be directed to parts of the space (Posner and Cohen, 1984), distinct features (Treisman and Gelade, 1980) and, as recently shown, also to entire objects (Desimone and Duncan, 1995; Valdes-Sosa et al., 1998; O'Craven et al., 1999; Schoenfeld et al., 2003). Depending on the goals of the observer, those types of selection are flexibly employed to eliminate ambiguity and to shape out wanted from unwanted visual information (Hopf et al., 2005; Stoppel et al., 2007). It is becoming increasingly clear that attentional selection can also operate upon internal representations held in visual short-term memory (VSTM) beyond the phase of perceptual encoding. Importantly, these representations appear to maintain at least some of the spatial configuration of the initial perceptual scene (Gratton, 1998; Jiang et al., 2000). Very recent work has shown that the selection of a target within a search array maintained in VSTM draws on similar mechanisms as during perception. For example, the identification of a color target within that array was observed to elicit an N2pc component in the electrophysiological response (Kuo et al., 2009). The N2pc is a well established signature of attentional selection in visual search (Luck and Hillyard, 1994) which reflects spatial biasing of activity in contralateral posterior parietal and occipital-temporal areas during target selection and the suppression of distracters (Hopf et al., 2000, 2002). A recent study by Astle et al. (2009) investigated not only whether the spatial representation of an array of items is maintained in VSTM, but went on to look for the mechanism underlying the selection of individual item features stored in VSTM. This is an important issue because studies have shown that when items are stored in VSTM, the individual features are bound together to form integrated objects (Vogel et al., 2001; Awh et al., 2007). This could lead to conflicting information at the level of feature selection because features requiring selection may be bound to features to be discarded from processing. This is the situation Astle et al. (2009) address in one of their experiments. The subjects memorized two items of different shape and color and were subsequently cued to either recall whether a certain color or shape was present in the array (VSTM search). In another experiment, subjects were first cued to a particular shape or color and then asked to perform the search in a presented array (perceptual search). Astle et al. (2009) compared the ERP correlates of VSTM search to those of perceptual search and observed that both were associated with a lateralized ERP response, suggesting that VSTM, like perceptual search, involves spatially organized attentional selection. Surprisingly, in contrast to the perceptual search, the lateralized response seen in VSTM search was of opposite polarity to the typical N2pc component. This is intriguing and at the same time surprising, as this contrasts with the previous demonstration of a typical N2pc component being associated with VSTM search (Kuo et al., 2009). The authors suggest that while during perceptual search participants can search the array at a feature-specific level, VSTM search requires these processes to occur at an object-specific level. In contrast to the study by Kuo et al. (2009), in the present experiment the binding of features into objects in VSTM led to conflicting requirements of feature selection, namely, to the necessity to simultaneously boost and suppress features that are part of the same object. Whether this could entirely explain an inversed polarity of the N2pc component is still to be determined in further studies. Nonetheless, the data of Astle et al. (2009) points to different mechanisms of feature selection in perceptual and VSTM search, especially when conflicting feature-level information interferes with the target selection.