Modified deep attractor neural networks for variability compensation in recognition tasks

Abstract

Some brain functions can be described with attractor formation in the human neural system. Various biologically-inspired models have employed this characteristic for memory representation, pattern completion and, noise reduction. Attractor neural networks are amongst these models capable of settling to stable patterns called attractors using their recurrent connections. Deep attractor neural networks (DANets) are relatively new models that utilize their attractors to separate the sources in the mixtures. In this paper, a modified version of DANets, variability compensator attractor neural network (VCANets); is proposed. This model compensates variabilities which are a common source of performance degradation in recognition tasks. For this purpose, significant modifications were applied to the structure and training procedure of DANets, including attractors' calculation, multi-task learning, curriculum learning, mining of challenging samples, and special mini-batch training procedure. Experiments on the MNIST and MNIST-C datasets for handwritten digit recognition and FARSDAT dataset for acoustic landmark recognition show that VCANet can effectively improve recognition accuracy and reduce unwanted variabilities.