Research on Sparsity of Output Synapses in Echo State Networks

Xiaohui Mu,Lixiang Li,Xiangyu He

doi:10.1155/2018/1984524

Xiaohui Mu, Lixiang Li + Show 1 more

Open Access

https://doi.org/10.1155/2018/1984524

Copy DOI

Abstract

This paper presents an improved model of echo state networks (ESNs) and gives the definitions of energy consumption, energy efficiency, etc. We verify the existence of redundant output synaptic connections by numerical simulations. We investigate the relationships among energy consumption, prediction step, and the sparsity of ESN. At the same time, the energy efficiency and the prediction steps are found to present the same variation trend when silencing different synapses. Thus, we propose a computationally efficient method to locate redundant output synapses based on energy efficiency of ESN. We find that the neuron states of redundant synapses can be linearly represented by the states of other neurons. We investigate the contributions of redundant and core output synapses to the performance of network prediction. For the prediction task of chaotic time series, the predictive performance of ESN is improved about hundreds of steps by silencing redundant synapses.

Highlights

Artificial Intelligence is a branch of science and it studies and develops theories, methods, techniques, and applications for simulating and extending human intelligence [1]
This paper presents an improved model of echo state networks (ESNs) and gives the definitions of energy consumption, energy efficiency, etc
We have analyzed the reason for the existence of redundant output synaptic connections in ESN from the perspectives of mathematical analysis and numerical simulations

Summary

Introduction

Artificial Intelligence is a branch of science and it studies and develops theories, methods, techniques, and applications for simulating and extending human intelligence [1]. ESN approach contains the learning and prediction processes It is comprised by input neurons, sparse reservoir, and output neurons. Few of the existing algorithms based on ESN studied the problems of energy consumption and energy efficiency and reduced redundant synapses in the output connections. To find out and remove redundant synapses in the output connections, further to improve the predictive performance of ESN, this paper proposes an improved ESN model. We analyze the relationship between energy consumption and sparsity, the relationship between predicted steps and sparsity, and the relationship between energy efficiency and sparsity in ESN and verify the existence of redundant synapses. Compared to a fully connected network, the predictive performance is improved about hundreds of steps by silencing redundant synapses for the task of chaotic time series prediction

Materials and Methods

Results

C Predicted Steps

Findings

Discussion

Conclusions