Multi-part Objects Research Articles

Contributed Session II: Neural correlates of serial processing during divided attention across multipart objects.

Judgments of multiple simultaneously presented stimuli produce a variety of divided attention effects. For example, participants can detect colors in two locations as well as in one, but can recognize only one masked word at a time (White, Palmer, & Boynton, Psych Science 2018). Here, we ask whether judging objects with interchangeable parts, similar to letters in words, also produces performance deficits consistent with serial processing, and which brain areas subserve this process. In a probe recognition task, participants discriminated abstract objects made of Duplo™ parts. On each trial, two objects were presented, with either one or both objects cued as relevant (single- and dual-task conditions, respectively). The distractor probes were made of the same parts in a different order. The difference in performance between the single- and dual-task conditions was 17±1% (n=13), consistent with the prediction of a serial model and rejecting the fixed-capacity parallel model. This result suggests that there is a visual brain area where information can be processed about only one object at a time. Currently, we are using fMRI to examine activity in object-selective regions of the human lateral occipital cortex. To seek evidence of serial processing, we are assessing how stimulus-related modulation is mediated by selective attention; the area of interest should show a modulation for only the attended stimulus location.

The Ties that Bind: Agnosia, Neglect and Selective Attention to Visual Scale.

Historical and contemporary treatments of visual agnosia and neglect regard these disorders as largely unrelated. It is thought that damage to different neural processes leads directly to one or the other condition, yet apperceptive variants of agnosia and object-centered variants of neglect share remarkably similar deficits in the quality of conscious experience. Here we argue for a closer association between "apperceptive" variants of visual agnosia and "object-centered" variants of visual neglect. We introduce a theoretical framework for understanding these conditions based on "scale attention", which refers to selecting boundary and surface information at different levels of the structural hierarchy in the visual array. We review work on visual agnosia, the cortical structures and cortico-cortical pathways that underlie visual perception, visuospatial neglect and object-centered neglect, and attention to scale. We highlight direct and indirect pathways involved in these disorders and in attention to scale. The direct pathway involves the posterior vertical segments of the superior longitudinal fasciculus that are positioned to link the established dorsal and ventral attentional centers in the parietal cortex with structures in the inferior occipitotemporal cortex associated with visual apperceptive agnosia. The connections in the right hemisphere appear to be more important for visual conscious experience, whereas those in the left hemisphere appear to be more strongly associated with the planning and execution of visually guided grasps directed at multi-part objects such as tools. In the latter case, semantic and functional information must drive the selection of the appropriate hand posture and grasp points on the object. This view is supported by studies of grasping in patients with agnosia and in patients with neglect that show that the selection of grasp points when picking up a tool involves both scale attention and semantic contributions from inferotemporal cortex. The indirect pathways, which include the inferior fronto-occipital and horizontal components of the superior longitudinal fasciculi, involve the frontal lobe, working memory and the "multiple demands" network, which can shape the content of visual awareness through the maintenance of goal- and task-based abstractions and their influence on scale attention. Recent studies of human cortico-cortical pathways necessitate revisions to long-standing theoretical views on visual perception, visually guided action and their integrations. We highlight findings from a broad sample of seemingly disparate areas of research to support the proposal that attention to scale is necessary for typical conscious visual experience and for goal-directed actions that depend on functional and semantic information. Furthermore, we suggest that vertical pathways between the parietal and occipitotemporal cortex, along with indirect pathways that involve the premotor and prefrontal cortex, facilitate the operations of scale attention.

The influence of object structure on visual short-term memory for multipart objects.

Numerous studies have shown that more visual features can be stored in visual short-term memory (VSTM) when they are encoded from fewer objects (Luck & Vogel, 1997, Nature, 390, 279-281; Olson & Jiang, 2002, Perception & Psychophysics, 64[7], 1055-1067). This finding has been consistent for simple objects with one surface and one boundary contour, but very few experiments have shown a clear performance benefit when features are organized as multipart objects versus spatially dispersed single-feature objects. Some researchers have suggested multipart object integration is not mandatory because of the potential ambiguity of the display (Balaban & Luria, 2015, Cortex, 26(5), 2093-2104; Luria & Vogel, 2014, Journal of Cognitive Neuroscience, 26[8], 1819-1828). For example, a white bar across the middle of a red circle could be interpreted as two objects, a white bar occluding a red circle, or as a single two-colored object. We explore whether an object benefit can be found by disambiguating the figure-ground organization of multipart objects using a luminance gradient and linear perspective to create the appearance of a unified surface. Also, we investigated memory for objects with a visual feature indicated by a hole, rather than an additional surface on the object. Results indicate the organization of multipart objects can influence VSTM performance, but the effect is driven by how the specific organization allows for use of global ensemble statistics of the memory array rather than a memory benefit for local object representations.

The structure of illusory conjunctions reveals hierarchical binding of multipart objects.

The world around us is filled with complex objects, full of color, motion, shape, and texture, and these features seem to be represented separately in the early visual system. Anne Treisman pointed out that binding these separate features together into coherent conscious percepts is a serious challenge, and she argued that selective attention plays a critical role in this process. Treisman also showed that, consistent with this view, outside the focus of attention we suffer from illusory conjunctions: misperceived pairings of features into objects. Here we used Treisman's logic to study the structure of pre-attentive representations of multipart, multicolor objects, by exploring the patterns of illusory conjunctions that arise outside the focus of attention. We found consistent evidence of some pre-attentive binding of colors to their parts, and weaker evidence of binding multiple colors of the same object. The extent to which such hierarchical binding occurs seems to depend on the geometric structure of multipart objects: Objects whose parts are easier to separate seem to exhibit greater pre-attentive binding. Together, these results suggest that representations outside the focus of attention are not entirely a "shapeless bundles of features," but preserve some meaningful object structure.

Which properties of the visual stimuli predict the type of representation used in mental rotation?

Two modes of internal representation, holistic and piecemeal transformation, have been reported as a means to perform mental rotation (MR) tasks. The stimulus complexity effect has been proposed as an indicator to disentangle between these two representation types. However, the complexity effect has not been fully confirmed owing to the fact that different performances could result from different types of stimuli. Moreover, whether the non-mirror foils play a role in forcing participants to encode all the information from the stimuli in MR tasks is still under debate. This study aims at testing the association between these two common types of representation with different stimuli in MR tasks. First, the numbers of segments and vertices in polygon stimuli were manipulated to test which property of the visual stimuli is more likely to influence the representation in MR tasks. Second, the role of non-mirror foils was examined by comparing the stimulus complexity effect in both with- and without-non-mirror foils conditions. The results revealed that the segment number affected the slope of the linear function relating response times to rotation angle, but the vertex number in the polygons did not. This suggests that a holistic representation was more likely to be adopted in processing integrated objects, whereas a piecemeal transformation was at play in processing multi-part objects. In addition, the stimulus complexity effect was observed in the with-non-mirror foils condition but not in the without-non-mirror foils one, providing a direct evidence to support the role of non-mirror foils in MR tasks.

Does Extensive Training at Individuating Novel Objects in Adulthood Lead to Visual Expertise? The Role of Facelikeness.

Human adults have a rich visual experience thanks to seeing human faces since birth, which may contribute to the acquisition of perceptual processes that rapidly and automatically individuate faces. According to a generic visual expertise hypothesis, extensive experience with nonface objects may similarly lead to efficient processing of objects at the individual level. However, whether extensive training in adulthood leads to visual expertise remains debated. One key issue is the extent to which the acquisition of visual expertise depends on the resemblance of objects to faces in terms of the spatial configuration of parts. We therefore trained naive human adults to individuate a large set of novel parametric multipart objects. Critically, one group of participants trained with the objects in a "facelike" stimulus orientation, whereas a second group trained with the same objects but with the objects rotated 180° in the picture plane into a "nonfacelike" orientation. We used a fast periodic visual stimulation EEG protocol to objectively quantify participants' ability to discriminate untrained exemplars before and after training. EEG responses associated with the frequency of identity change in a fast stimulation sequence, which reflects rapid and automatic perceptual processes, were observed over lateral occipital sites for both groups before training. There was a significant, albeit small, increase in these responses after training but only for the facelike group and only to facelike stimuli. Our findings indicate that perceived facelikeness plays a role in visual expertise and highlight how the adult perceptual system exploits familiar spatial configurations when learning new object categories.

Multi-part objects yield no change detection benefit for color and orientation even when parts are unambiguously integrated in the display

Multi-part objects yield no change detection benefit for color and orientation even when parts are unambiguously integrated in the display

Stereo disparity facilitates view generalization during shape recognition for solid multipart objects.

Current theories of object recognition in human vision make different predictions about whether the recognition of complex, multipart objects should be influenced by shape information about surface depth orientation and curvature derived from stereo disparity. We examined this issue in five experiments using a recognition memory paradigm in which observers (N = 134) memorized and then discriminated sets of 3D novel objects at trained and untrained viewpoints under either mono or stereo viewing conditions. In order to explore the conditions under which stereo-defined shape information contributes to object recognition we systematically varied the difficulty of view generalization by increasing the angular disparity between trained and untrained views. In one series of experiments, objects were presented from either previously trained views or untrained views rotated (15°, 30°, or 60°) along the same plane. In separate experiments we examined whether view generalization effects interacted with the vertical or horizontal plane of object rotation across 40° viewpoint changes. The results showed robust viewpoint-dependent performance costs: Observers were more efficient in recognizing learned objects from trained than from untrained views, and recognition was worse for extrapolated than for interpolated untrained views. We also found that performance was enhanced by stereo viewing but only at larger angular disparities between trained and untrained views. These findings show that object recognition is not based solely on 2D image information but that it can be facilitated by shape information derived from stereo disparity.

Late development of metric part-relational processing in object recognition.

Four experiments with unfamiliar objects examined the remarkably late consolidation of part-relational relative to part-based object recognition (Jüttner, Wakui, Petters, Kaur, & Davidoff, 2013). Our results indicate a particularly protracted developmental trajectory for the processing of metric part relations. Schoolchildren aged 7 to 14 years and adults were tested in 3-Alternative-Forced-Choice tasks to judge the correct appearance of upright and inverted newly learned multipart objects that had been manipulated in terms of individual parts or part relations. Experiment 1 showed that even the youngest tested children were close to adult levels of performance for recognizing categorical changes of individual parts and relative part position. By contrast, Experiment 2 demonstrated that performance for detecting metric changes of relative part position was distinctly reduced in young children compared with recognizing metric changes of individual parts, and did not approach the latter until 11 to 12 years. A similar developmental dissociation was observed in Experiment 3, which contrasted the detection of metric relative-size changes and metric part changes. Experiment 4 showed that manipulations of metric size that were perceived as part (rather than part-relational) changes eliminated this dissociation. Implications for theories of object recognition and similarities to the development of face perception are discussed.

Hierarchical binding in multi-part objects

Hierarchical binding in multi-part objects

Developmental trajectories of part-based and configural object recognition in adolescence.

Three experiments assessed the development of children's part and configural (part-relational) processing in object recognition during adolescence. In total, 312 school children aged 7-16 years and 80 adults were tested in 3-alternative forced choice (3-AFC) tasks. They judged the correct appearance of upright and inverted presented familiar animals, artifacts, and newly learned multipart objects, which had been manipulated either in terms of individual parts or part relations. Manipulation of part relations was constrained to either metric (animals, artifacts, and multipart objects) or categorical (multipart objects only) changes. For animals and artifacts, even the youngest children were close to adult levels for the correct recognition of an individual part change. By contrast, it was not until 11-12 years of age that they achieved similar levels of performance with regard to altered metric part relations. For the newly learned multipart objects, performance was equivalent throughout the tested age range for upright presented stimuli in the case of categorical part-specific and part-relational changes. In the case of metric manipulations, the results confirmed the data pattern observed for animals and artifacts. Together, the results provide converging evidence, with studies of face recognition, for a surprisingly late consolidation of configural-metric relative to part-based object recognition.

Transfer of object category knowledge across visual and haptic modalities: Experimental and computational studies

We study people’s abilities to transfer object category knowledge across visual and haptic domains. If a person learns to categorize objects based on inputs from one sensory modality, can the person categorize these same objects when the objects are perceived through another modality? Can the person categorize novel objects from the same categories when these objects are, again, perceived through another modality? Our work makes three contributions. First, by fabricating Fribbles (3-D, multi-part objects with a categorical structure), we developed visual-haptic stimuli that are highly complex and realistic, and thus more ecologically valid than objects that are typically used in haptic or visual-haptic experiments. Based on these stimuli, we developed the See and Grasp data set, a data set containing both visual and haptic features of the Fribbles, and are making this data set freely available on the world wide web. Second, complementary to previous research such as studies asking if people transfer knowledge of object identity across visual and haptic domains, we conducted an experiment evaluating whether people transfer object category knowledge across these domains. Our data clearly indicate that we do. Third, we developed a computational model that learns multisensory representations of prototypical 3-D shape. Similar to previous work, the model uses shape primitives to represent parts, and spatial relations among primitives to represent multi-part objects. However, it is distinct in its use of a Bayesian inference algorithm allowing it to acquire multisensory representations, and sensory-specific forward models allowing it to predict visual or haptic features from multisensory representations. The model provides an excellent qualitative account of our experimental data, thereby illustrating the potential importance of multisensory representations and sensory-specific forward models to multisensory perception.

The relative weight of shape and non-rigid motion cues in object perception: A model of the parameters underlying dynamic object discrimination

Shape and motion are two dominant cues for object recognition, but it can be difficult to investigate their relative quantitative contribution to the recognition process. In the present study, we combined shape and non-rigid motion morphing to investigate the relative contributions of both types of cues to the discrimination of dynamic objects. In Experiment 1, we validated a novel parameter-based motion morphing technique using a single-part three-dimensional object. We then combined shape morphing with the novel motion morphing technique to pairs of multipart objects to create a joint shape and motion similarity space. In Experiment 2, participants were shown pairs of morphed objects from this space and responded "same" on the basis of motion-only, shape-only, or both cues. Both cue types influenced judgments: When responding to only one cue, the other cue could be ignored, although shape cues were more difficult to ignore. When responding on the basis of both cues, there was an overall bias to weight shape cues more than motion cues. Overall, our results suggest that shape influences discrimination more than motion even when both cue types have been made quantitatively equivalent in terms of their individual discriminability.

Selecting multipart objects: Is uniformity necessary?

Selecting multipart objects: Is uniformity necessary?

Detection of unexpected multi-part objects from segmented contour maps

A novel method is proposed to detect multi-part objects of unknown specific shape and appearance in natural images. It consists in first extracting a strictly over-segmented map of circular arcs and straight-line segments from an edge map. Each obtained constant-curvature contour primitive has an unknown origin which may be the external boundary of an interesting object, the textured or marked region enclosed by that boundary, or the external background region. The following processing steps identify, in a systematic yet efficient way, which groups of ordered contour primitives form a complete boundary of proper multi-part shape. Multiple detections are ranked with the top boundaries best satisfying a combination of global shape grouping criteria. Experimental results confirm the unique potential of the method to identify, in images of variable complexity, actual boundaries of multi-part objects as diverse as an airplane, a stool, a bicycle, a fish, and a toy truck.

Modulation of Viewpoint Effects in Object Recognition by Shape and Motion Cues

In three experiments, we examined the role of structural similarity and different types of motion on the efficiency of performing same--different shape judgments across changes in viewpoints. In all experiments, participants judged whether two novel, multi-part objects were structurally identical, and they were to ignore any viewpoint or motion differences between the objects. In experiment 1, participants were affected by viewpoint differences more for structurally similar than structurally distinct objects, but this interaction was mitigated by rigid motion. In experiments 2 and 3, we used only structurally similar objects that moved only some of their parts, either in a similar way between objects within a pair or in distinctive ways. Participants' recognition performance was facilitated by this articulated motion relative to both static and scrambled controls. We conclude that coherent motion facilitates generalisation across different views of dynamic objects under some conditions.

Qualitative part-based models in content-based image retrieval

A qualitative, volumetric part-based model is proposed to improve the categorical invariance and viewpoint invariance in content-based image retrieval, and a novel two-step part-categorization method is presented to build it. The method consists first in transforming parts extracted from a segmented contour primitive map and then categorizing the transformed parts using interpretation rules. The first step allows noisy extracted parts to be transformed to the domain of a simple classifier. The second step computes features of the transformed parts for categorization. Content-based image retrieval experiments using real images of complex multi-part objects confirm that a model built from the categorized parts improves both the categorical invariance and the viewpoint invariance. It does so by directly addressing the fundamental limits of low-level models.

Modulation of Neural Activity during Observational Learning of Actions and Their Sequential Orders

How does the brain transform perceptual representations of others' actions into motor representations that can be used to guide behavior? Here we used functional magnetic resonance imaging to record human brain activity while subjects watched others construct multipart objects under varied task demands. We find that relative to resting baseline, passive action observation increases activity within inferior frontal and parietal cortices implicated in action encoding (mirror system) and throughout a distributed network of areas involved in motor representation, including dorsal premotor cortex, pre-supplementary motor area, cerebellum, and basal ganglia (experiments 1 and 2). Relative to passive observation, these same areas show increased activity when subjects observe with the intention to subsequently reproduce component actions using the demonstrated sequential procedures (experiment 1). Observing the same actions with the intention of reproducing component actions, but without the requirement to use the demonstrated sequential procedure, increases activity in the same regions, although to a lesser degree (experiment 2). These findings demonstrate that when attempting to learn behaviors through observation, the observers' intentions modulate responses in a widely distributed network of cortical and subcortical regions implicated previously in action encoding and/or motor representation. Among these regions, only activity within the right intraparietal sulcus predicts the accuracy with which observed procedures are subsequently performed. Successful formation of motor representations of sequential procedures through observational learning is dependent on computations implemented within this parietal region.

Effects of varying stimulus size on object recognition in pigeons.

The authors investigated the pigeon's ability to generalize object discrimination performance to smaller and larger versions of trained objects. In Experiment 1, they taught pigeons with line drawings of multipart objects and later tested the birds with both larger and smaller drawings. The pigeons exhibited significant generalization to new sizes, although they did show systematic performance decrements as the new size deviated from the original. In Experiment 2, the authors tested both linear and exponential size changes of computer-rendered basic shapes to determine which size transformation produced equivalent performance for size increases and decreases. Performance was more consistent with logarithmic than with linear scaling of size. This finding was supported in Experiment 3. Overall, the experiments suggest that the pigeon encodes size as a feature of objects and that the representation of size is most likely logarithmic.

Three-month-old infants’ object recognition across changes in viewpoint using an operant learning procedure

Object knowledge refers to the understanding that all objects share certain properties. Various components of object knowledge (e.g., object occlusion, object causality) have been examined in human infants to determine its developmental origins. Viewpoint invariance—the understanding that an object viewed from different viewpoints is still the same object—is one area of object knowledge, however, that has received less attention. To this end, infants’ capacity for viewpoint-invariant perception of multi-part objects was investigated. Three-month-old infants were tested for generalization to an object displayed on a mobile that differed only in orientation (i.e., viewpoint) from a training object. Infants were given experience with a wide range of object views (Experiment 1) or a more restricted range during training (Experiment 2). The results showed that infants generalized between a horizontal and vertical viewpoint (Experiment 1) that they could clearly discriminate between in other contexts (i.e., with restricted view experience, Experiment 2). Overall, the outcome shows that training experience with multiple viewpoints plays an important role in infants’ ability to develop a general percept of an object's 3D structure and promotes viewpoint-invariant perception of multi-part objects; in contrast, restricting training experience impedes viewpoint-invariant recognition of multi-part objects.