Training energy-based single-layer Hopfield and oscillatory networks with unsupervised and supervised algorithms for image classification

Abstract

This paper investigates how to solve image classification with Hopfield neural networks (HNNs) and oscillatory neural networks (ONNs). This is a first attempt to apply ONNs for image classification. State-of-the-art image classification networks are multi-layer models trained with supervised gradient back-propagation, which provide high-fidelity results but require high energy consumption and computational resources to be implemented. On the contrary, HNN and ONN networks are single-layer, requiring less computational resources, however, they necessitate some adaptation as they are not directly applicable for image classification. ONN is a novel brain-inspired computing paradigm that performs low-power computation and is attractive for edge artificial intelligence applications, such as image classification. In this paper, we perform image classification with HNN and ONN by exploiting their auto-associative memory (AAM) properties. We evaluate precision of HNN and ONN trained with state-of-the-art unsupervised learning algorithms. Additionally, we adapt the supervised equilibrium propagation (EP) algorithm to single-layer AAM architectures, proposing the AAM-EP. We test and validate HNN and ONN classification on images of handwritten digits using a simplified MNIST set. We find that using unsupervised learning, HNN reaches 65.2%, and ONN 59.1% precision. Moreover, we show that AAM-EP can increase HNN and ONN precision up to 67.04% for HNN and 62.6% for ONN. While intrinsically HNN and ONN are not meant for classification tasks, to the best of our knowledge, these are the best-reported precisions of HNN and ONN performing classification of images of handwritten digits.