Neural Network for Low-Memory IoT Devices and MNIST Image Recognition Using Kernels Based on Logistic Map

Andrei Velichko

doi:10.3390/electronics9091432

Andrei Velichko

Open Access

https://doi.org/10.3390/electronics9091432

Copy DOI

Abstract

This study presents a neural network which uses filters based on logistic mapping (LogNNet). LogNNet has a feedforward network structure, but possesses the properties of reservoir neural networks. The input weight matrix, set by a recurrent logistic mapping, forms the kernels that transform the input space to the higher-dimensional feature space. The most effective recognition of a handwritten digit from MNIST-10 occurs under chaotic behavior of the logistic map. The correlation of classification accuracy with the value of the Lyapunov exponent was obtained. An advantage of LogNNet implementation on IoT devices is the significant savings in memory used. At the same time, LogNNet has a simple algorithm and performance indicators comparable to those of the best resource-efficient algorithms available at the moment. The presented network architecture uses an array of weights with a total memory size from 1 to 29 kB and achieves a classification accuracy of 80.3–96.3%. Memory is saved due to the processor, which sequentially calculates the required weight coefficients during the network operation using the analytical equation of the logistic mapping. The proposed neural network can be used in implementations of artificial intelligence based on constrained devices with limited memory, which are integral blocks for creating ambient intelligence in modern IoT environments. From a research perspective, LogNNet can contribute to the understanding of the fundamental issues of the influence of chaos on the behavior of reservoir-type neural networks.

Highlights

In the age of neural networks and Internet of Things (IoT), the search for new neural network architectures capable of operating on devices with small amounts of memory (10s of kB of RAM) is becoming an urgent agenda [1,2,3]
The constrained devices form the basis for ambient intelligence (AmI) in IoT environments [5], and can be divided into three categories based on code and memory sizes: class 0
The current study presents a new architecture for a neural network, where complex dynamics are simulated by the application of logistic mapping to the multilayer weights of a feedforward network

Summary

Introduction

In the age of neural networks and Internet of Things (IoT), the search for new neural network architectures capable of operating on devices with small amounts of memory (10s of kB of RAM) is becoming an urgent agenda [1,2,3]. Training of neural networks is the process of optimal selection of the weight coefficients of neuron couplings and filter parameters that occupy a significant amount of memory. The architecture of the LogNNet network, an algorithm for finding weights using logistic mapping and a method for assessing the accuracy of classification of handwritten digits from the MNIST-10 database are described. LogNNet is that the proposed network can beand used to implement artificial intelligence based on compared with other neural well-known algorithms, suggestions to improve functionality and to reduce constrained with limited are integral blocks forindicate the creation of proposed. Neural network can be used to implement artificial intelligence based on constrained devices with limited memory, which are integral blocks for the creation of AmI and modern IoT environments

Data and Methods

Results

Discussion

Conclusions