Learning Topographic Representations of Nature Images with Pairwise Cumulant

Abstract

In this paper, we propose a model for natural images to learn topographic representations and complex cell properties. Different from the estimation of traditional models, e.g., pooling the outputs of filters in neighboring regions, our method maximizes a simple form of binary relations between two adjacent complex cells--"pairwise cumulant", which contains the favorable nonlinearity as high order cumulant, and can exploit the "sparseness" and "correlation" of cells in primary visual cortex. By means of choosing nonlinearity properly, our model is related to cumulant-based ICA model, and the derived fixed-point algorithm is close to the well-known FastICA algorithm. The local convergence analysis proves that our fixed-point algorithm is cubic convergence and experiments on nature images show its high efficiency than traditional algorithms. Besides, simulations demonstrate the effectiveness of our model in capturing nonlinear dependencies among these neighboring complex cells. The learnt filters preserve properties of complex cells, and their orientation, spatial frequency and location change smoothly over the topographic map. In addition, these learnt filters can be used as feature descriptors. They produce features that are invariant to object transformations, and achieve better results than traditional models on digit recognition tasks.