Non-monotonic Piecewise Linear Activation Functions Research Articles

Multistability of state-dependent switching neural networks with discontinuous nonmonotonic piecewise linear activation functions

This paper presents the theoretical results on the multistability of state-dependent switching neural networks with discontinuous nonmonotonic piecewise linear activation functions. For n-neurons switching model, this paper shows that neural networks have 7n equilibrium points, 6n of which are located at the continuous points of activation functions and others are located at the discontinuous points of activation functions. Among these equilibrium points, 4n or 5n are stable and others are unstable, which depend on the relationship between the switching threshold and the discontinuous points of the activation functions. Compared with existing results, this paper reveals that switching threshold and discontinuous character are crucial in increasing the number of equilibrium points. Two examples are presented to verify the theoretical results.

Multistability and attraction basins of discrete-time neural networks with nonmonotonic piecewise linear activation functions

This paper is concerned with multistability and attraction basins of discrete-time neural networks with nonmonotonic piecewise linear activation functions. Under some reasonable conditions, the addressed networks have (2m+1)n equilibrium points. (m+1)n of which are locally asymptotically stable, and the others are unstable. The attraction basins of the locally asymptotically stable equilibrium points are given in the form of hyperspherical regions. These results here, which include existence, uniqueness, locally asymptotical stability, instability and attraction basins of the multiple equilibrium points, generalize and improve the earlier publications. Finally, an illustrative example with numerical simulation is given to show the feasibility and the effectiveness of the theoretical results. The theoretical results and illustrative example indicate that the activation functions improve the storage capacity of neural networks significantly.

Multistability and instability of competitive neural networks with non-monotonic piecewise linear activation functions

This paper addresses the issue of multistability for competitive neural networks. First, a general class of continuous non-monotonic piecewise linear activation functions is introduced. Then, based on the fixed point theorem, the contraction mapping theorem and the eigenvalue properties of strict diagonal dominance matrix, it is shown that under some conditions, such n-neuron competitive neural networks have exactly 5n equilibrium points, among which 3n equilibrium points are locally exponentially stable and the others are unstable. Moreover, it is revealed that the neural networks with non-monotonic piecewise linear activation functions introduced in this paper can have greater storage capacity than the ones with Mexican-hat-type activation function and nondecreasing saturated activation function. In addition, unlike most existing multistability results of neural networks with nondecreasing activation functions, the location of those obtained 3n locally stable equilibrium points in this paper is more flexible. Finally, a numerical example is provided to illustrate and validate the theoretical findings via comprehensive computer simulations.

Coexistence and local [formula omitted]-stability of multiple equilibrium points for memristive neural networks with nonmonotonic piecewise linear activation functions and unbounded time-varying delays

In this paper, the coexistence and dynamical behaviors of multiple equilibrium points are discussed for a class of memristive neural networks (MNNs) with unbounded time-varying delays and nonmonotonic piecewise linear activation functions. By means of the fixed point theorem, nonsmooth analysis theory and rigorous mathematical analysis, it is proven that under some conditions, such n-neuron MNNs can have 5n equilibrium points located in ℜn, and 3n of them are locally μ-stable. As a direct application, some criteria are also obtained on the multiple exponential stability, multiple power stability, multiple log-stability and multiple log–log-stability. All these results reveal that the addressed neural networks with activation functions introduced in this paper can generate greater storage capacity than the ones with Mexican-hat-type activation function. Numerical simulations are presented to substantiate the theoretical results.

Complete Stability of Neural Networks With Nonmonotonic Piecewise Linear Activation Functions

This brief studies the complete stability of neural networks with nonmonotonic piecewise linear activation functions. By applying the fixed-point theorem and the eigenvalue properties of the strict diagonal dominance matrix, some conditions are derived, which guarantee that such $n$ -neuron neural networks are completely stable. More precisely, the following two important results are obtained: 1) The corresponding neural networks have exactly $5^n$ equilibrium points, among which $3^n$ equilibrium points are locally exponentially stable and the others are unstable; 2) as long as the initial states are not equal to the equilibrium points of the neural networks, the corresponding solution trajectories will converge toward one of the $3^n$ locally stable equilibrium points. A numerical example is provided to illustrate the theoretical findings via computer simulations.

Multistability and Instability of Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions.

In this paper, we discuss the coexistence and dynamical behaviors of multiple equilibrium points for recurrent neural networks with a class of discontinuous nonmonotonic piecewise linear activation functions. It is proved that under some conditions, such n -neuron neural networks can have at least 5(n) equilibrium points, 3(n) of which are locally stable and the others are unstable, based on the contraction mapping theorem and the theory of strict diagonal dominance matrix. The investigation shows that the neural networks with the discontinuous activation functions introduced in this paper can have both more total equilibrium points and more locally stable equilibrium points than the ones with continuous Mexican-hat-type activation function or discontinuous two-level activation functions. An illustrative example with computer simulations is presented to verify the theoretical analysis.

Multistability of memristive Cohen–Grossberg neural networks with non-monotonic piecewise linear activation functions and time-varying delays

The problem of coexistence and dynamical behaviors of multiple equilibrium points is addressed for a class of memristive Cohen–Grossberg neural networks with non-monotonic piecewise linear activation functions and time-varying delays. By virtue of the fixed point theorem, nonsmooth analysis theory and other analytical tools, some sufficient conditions are established to guarantee that such n-dimensional memristive Cohen–Grossberg neural networks can have 5n equilibrium points, among which 3n equilibrium points are locally exponentially stable. It is shown that greater storage capacity can be achieved by neural networks with the non-monotonic activation functions introduced herein than the ones with Mexican-hat-type activation function. In addition, unlike most existing multistability results of neural networks with monotonic activation functions, those obtained 3n locally stable equilibrium points are located both in saturated regions and unsaturated regions. The theoretical findings are verified by an illustrative example with computer simulations.

Dynamical Behaviors of Multiple Equilibria in Competitive Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions.

This paper addresses the problem of coexistence and dynamical behaviors of multiple equilibria for competitive neural networks. First, a general class of discontinuous nonmonotonic piecewise linear activation functions is introduced for competitive neural networks. Then based on the fixed point theorem and theory of strict diagonal dominance matrix, it is shown that under some conditions, such n -neuron competitive neural networks can have 5(n) equilibria, among which 3(n) equilibria are locally stable and the others are unstable. More importantly, it is revealed that the neural networks with the discontinuous activation functions introduced in this paper can have both more total equilibria and locally stable equilibria than the ones with other activation functions, such as the continuous Mexican-hat-type activation function and discontinuous two-level activation function. Furthermore, the 3(n) locally stable equilibria given in this paper are located in not only saturated regions, but also unsaturated regions, which is different from the existing results on multistability of neural networks with multiple level activation functions. A simulation example is provided to illustrate and validate the theoretical findings.

Multistability of neural networks with discontinuous non-monotonic piecewise linear activation functions and time-varying delays

This paper is concerned with the problem of coexistence and dynamical behaviors of multiple equilibrium points for neural networks with discontinuous non-monotonic piecewise linear activation functions and time-varying delays. The fixed point theorem and other analytical tools are used to develop certain sufficient conditions that ensure that the n-dimensional discontinuous neural networks with time-varying delays can have at least 5n equilibrium points, 3n of which are locally stable and the others are unstable. The importance of the derived results is that it reveals that the discontinuous neural networks can have greater storage capacity than the continuous ones. Moreover, different from the existing results on multistability of neural networks with discontinuous activation functions, the 3n locally stable equilibrium points obtained in this paper are located in not only saturated regions, but also unsaturated regions, due to the non-monotonic structure of discontinuous activation functions. A numerical simulation study is conducted to illustrate and support the derived theoretical results.