Dynamics of stochastic artificial neurons

Abstract

Up–down saturating counters driven by stochastic signals (Bernoulli processes) are used to implement stochastic neurons in digital pulsed neural networks. The transfer functions of these neurons relate the probabilities of the output and input pulses and can approximate either sigmoidal or gaussian activation functions. The output is in general a simple combinational logic function of the various bits of the b-bit counter. The dynamics of the state transitions are presented in this work. The times required for the state transitions to converge to equilibrium (stationary states) depend upon the input Bernoulli probability p in a complex manner. The sigmoidal or gaussian approximations improve with the number of states N=2 b of the counter, and convergence times increase approximately as ( N−1) 2 as expected for a biased random walk or diffusion process.