Step, Ramp, Delta, and Differentiable Activation Functions Obtained Using Percolation Equations

Abstract

This paper presents two new analytical equations, the Two Exponent Phenomenological Percolation Equation (TEPPE) and the Single Exponent Phenomenological Percolation Equation (SEPPE) which, for the proper choice of parameters, approximate the widely used Heaviside Step Function. The plots of the equations presented in the figures in this paper show some, but by no means all, of the step, ramp, delta, and differentiable activation functions that can be obtained using the percolation equations. By adjusting the parameters these equations can give linear, concave, and convex ramp functions, which are basic signals in systems used in engineering and management. The equations are also Analytic Activation Functions, the form or nature of which can be varied by changing the parameters. Differentiating these functions gives delta functions, the height and width of which depend on the parameters used. The TEPPE and SEPPE and their derivatives are presented in terms of the conductivity (<img src=image/13428023_01.gif>) owing to their original use in describing the electrical properties of binary composites, but are applicable to other percolative phenomena. The plots in the figures presented are used to show the response <img src=image/13428023_02.gif> (composite conductivity) for the parameters <img src=image/13428023_03.gif> (higher conductivity component of the composite), <img src=image/13428023_04.gif> (lower conductivity component of the composite) and <img src=image/13428023_05.gif>, the volume fraction of the higher conductivity component in the composite. The additional parameters are the critical volume fraction, <img src=image/13428023_06.gif>, which determines the position of the step or delta function on the <img src=image/13428023_07.gif> axis and one or two exponents <img src=image/13428023_08.gif>, and <img src=image/13428023_09.gif>.