Multiple Object Permanence Tracking: Maintenance, Retrieval and Transformation of Dynamic Object Representations

Abstract

Our visual world is composed of multiple dynamic objects with various visual features. For efficient interaction with the world, the visual system needs to keep binding of object features and update them as their dynamic changes. Given severe limitation of our visual short-term memory (VSTM) (Luck & Vogel, 1997; Pashler, 1988), it is a challenge to understand how the visual system deals with this binding problem in dynamic environment. In this chapter, I will review research on this issue, mainly focused on experimental studies using the paradigm called “multiple object permanence tracking” (Imaruoka et al., 2005; Saiki, 2002, 2003a, 2003b, 2007; Saiki & Miyatsuji, 2007, in press). Transformation of object representations in dynamic environment has been investigated mainly using multiple object tracking task (MOT) (Pylyshyn & Storm, 1988; Shcoll & Pylyshyn, 1998). In MOT, a dozen of identical objects (dots) are randomly moving around on the display, and observers required to track a subset of these objects. Although research with MOT revealed various properties of object representations used by visual cognition mechanisms, the issue of binding various object features into an object representation remains unclear, because MOT only manipulates spatiotemporal location of objects, not other features. To address the issue of feature binding in dynamic environment, multiple object permanence tracking (MOPT) task used objects with different colors and shapes, and investigated how these objects’ features are bound together in dynamic displays. This chapter will describe five topics investigated with MOPT paradigm. First, how feature binding is maintained over dynamic movement of multiple objects? A series of experiments revealed that our ability of keeping binding of objects’ color, shape and their spatiotemporal locations was significantly impaired when objects move (Saiki, 2003a, 2003b). Importantly, object motion was quite slow and predictable, so that the impairment was not due to failure of tracking of objects per se. Second, memory for feature binding was evaluated more strictly (Saiki & Miyatsuji, 2007). Switch detection task used in previous work showed that task performance was quite good when objects were stationary. However, simple switch detection task may overestimate our ability, and a more strict test revealed that even if objects were stationary, our ability of maintaining feature binding was much more limited than previous studies suggested. Third, is memory maintenance, or memory retrieval responsible for the performance impairment in MOPT task? To test this, I used retrieval cues