Signal to source assignment as a fundamental constraint on visual working memory

Abstract

A serious computational problem faced by human vision is to assign signals to sources. Received signals come from a priori unknown sources and the same signal content must be used to infer the presence of objects, to assign causal responsibility for individual signals to particular objects, and to infer the properties of those objects. Research on visual working memory often takes for granted signal to source assignment. We investigated assignment empirically through a modification to the color delayed estimation task: participants encoded the colors of a set of squares. After a delay, in half of trials, they reported as precisely as they could the color of a probed square. In 25% of trials, they were asked only to click in every location they remembered an object, and in 25% they reported the number of objects seen. In both the latter conditions, numerical reports showed significant variability, increasing with more objects. These results suggest that segmenting a display into the right number of objects is not always accomplished accurately. We used a clustering model to relate these results more directly to assignment. The model received fifty noisy samples from each object in a memory display. Samples were distributed spatially with eccentricity-dependent Gaussian variance and in color space following a von Mises centered on the color of each source object. The model then used a modified DBSCAN algorithm to cluster samples, effectively identifying samples thought to share a source. Finally, the model reported the number of clusters identified, their centroids, and an estimate of the source color for each cluster. Responses were significantly correlated with those of human observers. These results suggest that using visual working memory involves uncertainty about the association between received signals and their possible sources, uncertainty that places independent constraints on human capabilities. Meeting abstract presented at VSS 2016