Scene hierarchy structures object representations while flexibly adapting to varying task demands

Abstract

Objects are arranged in the environment following certain rules which we have termed “scene grammar”, that we exploit to perceive and interact with objects in an efficient way. We have recently proposed that this organization is hierarchical: objects are grouped together based on the scene they typically appear in (e.g., bathroom); based on the different spatial clusters of objects within a scene (“phrases”, e.g., the sink phrase); and based on the status they have within a phrase, where one object (“anchor”, e.g., the sink) typically holds strong predictions about the identity and position of other objects (“local objects”, e.g., a toothbrush). Here we investigate if mental representations of objects reflect this hierarchy and which behavioural goal strengthens this organization. To do so, we collected pairwise similarity judgments for triplets of everyday objects, using an odd-one-out task with different task instructions: in one case, similarity had to be judged based on action goals performed with these objects; in another case, similarity was judged based on visual appearance of objects; finally, similarity ratings were collected without explicit instructions. Results showed that measures of scene hierarchy had stronger impact on similarity judgments for the “action goal” task and the “no instructions” task than for the “visual appearance” task; some aspects of the hierarchy, like “phrase” organization, were even more important to make similarity judgments based on action goals than without explicit instructions. Moreover, measures that estimate visual features of objects had a stronger effect on similarity when judged based on visual appearance than in the other tasks. To conclude, our study shows that scene hierarchy shapes mental representation of objects and that this cognitive organization can flexibly change according to behavioural goals. Besides, it suggests that scene hierarchy is exploited with the purpose of efficient interaction with objects in the environment.