Positional binding with distributed representations

Abstract

Positional binding specifies feature positions for an image (or for text). We show how to incorporate position into a fully distributed vector formed from Vector Quantization, or add position to a vector formed from a Vector Symbolic Architecture. The method guarantees that small shifts in position result in small changes to the representation vector, and does not require an increase in vector size. The incorporation of positional binding improves performance on CIFAR-10 and on a new database of noisy abstract face images, which we hereby make public. For Deep Learning approaches, we emphasize the importance of positional binding, and this sheds light on why multiple layers and pooling are beneficial.