NEURON MODEL SIGMOID ACTIVATION FUNCTION BASED ON MULTI-OPTIONAL FUNCTIONS ENTROPY CONDITIONAL OPTIMIZATION DOCTRINE

Abstract

It is made an attempt to discover an explainable plausible reason for a neuron activation function, of a sigmoid type function like logistic function, substantiation in terms of the multi-optional conditional optimality doctrine for the special hybrid-optional effectiveness functions uncertainty. In the studied case, the input-output mapping is stipulated by the entropy of the activation function conditional optimal distribution in regards with the induced local field of the neuron. It is proposed to evaluate the direction of uncertainty with the combined hybrid relative pseudo-entropy function. This is a new insight into the scientific substantiation of the well-known dependency derived in another way. The developed theoretical contemplations and mathematical derivations are verified with numerical simulation and plotted diagrams.