Scene Complexity Captures the Detail Trace of Visual Episodic Memory

Abstract

Memory research has established that human observers use schemas, mental constructs that represent scene elements and relationships between those elements, as the basis for encoding information into visual episodic memory. Newly developed computational methods now allow the extraction of this schema information from the human observer via two-dimensional maps known as Visual Memory Schema (VMS) maps (Kyle-Davidson, Bors & Evans, 2019). VMS maps reveal which regions in a scene image are responsible for a human recalling that scene. VMS Maps are highly consistent among observers and are not explained solely by computational saliency models or eye-fixations. However, previous work also suggests that humans, in addition to schemas, retain rich detail of the scene, and the presence of that detailed information aids in later recognition of the same. Findings show that scenes considered to be more detailed (scenes with man-made elements) are remembered better than less detailed scenes (natural scene). Similarly, images that have detail removed (compared to a source image) appear less memorable than images that maintain the original detail (Baddeley & Evans, 2019). We investigated whether perceptual image complexity could stand as an analogue for the detail trace in visual memory, exploring how the complexity of an image relates to that image's memorability. Our approach was to conduct a behavioural experiment gathering two-dimensional complexity information from humans for a scene dataset that already has corresponding schema information (VMS Maps), which allowed us to explore complexity and memorability from a two-dimensional perspective. Next, we trained a neural network to predict complexity scores, and investigate this relationship for another scene memory dataset, now comparing the predicted complexity scores with ground truth memory for the images. The findings indicate that both extracted, and predicted perceptual complexity of the scene, relates to how well observers remember that same scene.