Columnar Machine: Fast estimation of structured sparse codes

Abstract

Abstract Ever since the discovery of columnar structures, their function remained enigmatic. As a potential explanation for this puzzling function, we introduce the ‘Columnar Machine’. We join two neural network types, Structured Sparse Coding (SSC) of generative nature exploiting sparse groups of neurons and Feed-Forward Networks (FFNs) into one architecture. Memories supporting recognition can be quickly loaded into SSC via supervision or can be learned by SSC in a self-organized manner. However, SSC evaluation is slow. We train FFNs for predicting the sparse groups and then the representation is computed by fast undercomplete methods. This two step procedure enables fast estimation of the overcomplete group sparse representations. The suggested architecture works fast and it is biologically plausible. Beyond the function of the minicolumnar structure it may shed light onto the role of fast feed-forward inhibitory thalamocortical channels and cortico-cortical feed-back connections. We demonstrate the method for natural image sequences where we exploit temporal structure and for a cognitive task where we explain the meaning of unknown words from their contexts.