Information loss in a non-linear neuronal model

Abstract

The Fisher information forms a classical analytical tool for the problem of inferring an unknown deterministic parameter of a given neuronal input by observing and measuring the response of the neuron. Alas, given the complex non-linear response of a neuron, the form of the conditional output probability function can render direct calculation of the Fisher information rather difficult. Using the Cauchy–Schwarz inequality, an alternative information measure has been recently proposed (Stein and Nossek, 2017) which serves as a conservative approximation to the Fisher information. The alternative information measure can be calculated using the first four output moments, which in many cases can be easily evaluated by measuring neuronal output statistics. In an application of this alternative information measure, we demonstrate here how to conservatively establish the intrinsic inference capability of a non-linear neuron model with a hyperbolic tangent activation function, when the analytic form of the parametric output statistic is not available in closed form. We found that for neuronal input variables corrupted by either Gaussian or Poisson noise, the information loss of the neuron increases with an increase in the slope (i.e. non-linearity) of the activation function, with a steeper increase for a Gaussian as compared to a Poisson input. The current approach can be used to study the effect of noise on both the input signal and neuronal output in neurons with other activation functions.