Power-sigmoid Activation Function Research Articles

ZNNs With a Varying-Parameter Design Formula for Dynamic Sylvester Quaternion Matrix Equation.

This article aims to studying how to solve dynamic Sylvester quaternion matrix equation (DSQME) using the neural dynamic method. In order to solve the DSQME, the complex representation method is first adopted to derive the equivalent dynamic Sylvester complex matrix equation (DSCME) from the DSQME. It is proven that the solution to the DSCME is the same as that of the DSQME in essence. Then, a state-of-the-art neural dynamic method is presented to generate a general dynamic-varying parameter zeroing neural network (DVPZNN) model with its global stability being guaranteed by the Lyapunov theory. Specifically, when the linear activation function is utilized in the DVPZNN model, the corresponding model [termed linear DVPZNN (LDVPZNN)] achieves finite-time convergence, and a time range is theoretically calculated. When the nonlinear power-sigmoid activation function is utilized in the DVPZNN model, the corresponding model [termed power-sigmoid DVPZNN (PSDVPZNN)] achieves the better convergence compared with the LDVPZNN model, which is proven in detail. Finally, three examples are presented to compare the solution performance of different neural models for the DSQME and the equivalent DSCME, and the results verify the correctness of the theories and the superiority of the proposed two DVPZNN models.

Robust tracking control of unknown models for space in-cabin robots with a pneumatic continuum arm

The service robots of space station in-cabin have attracted more and more attention. The space in-cabin robot with a pneumatic continuum arm is studied in this paper. It could be safer, more efficient and more flexible than the space rigid robot. However, the coupling motion of the moving base and the pneumatic continuum continuous arm brings a new challenge for controlling the end-effector to track the desired path. In this paper, a new control method based on the zeroing neural network (ZNN) is developed to solve the high-precision kinematics trajectory tracking control problem of unknown models. The real-time Jacobian matrix of the in-cabin robots with a pneumatic continuum arm is estimated by the input–output information when the parameter and the structure of the kinematic model are unknown. Moreover, this paper also employs a modified activation function power-sigmoid activation function (PSAF) to improve the robustness. In addition, it is proved through the Lyapunov stability theory that the proposed control approach is convergent and stable. Finally, the simulation results are given to show the effectiveness and robustness of the proposed control method for space in-cabin robots with a pneumatic continuum arm.

A weights direct determination neuronet for time‐series with applications in the industrial indices of the Federal Reserve Bank of St. Louis

AbstractThe shortcomings of conventional back‐propagation neuronets, such as slow training speed and local minimum, are known to be addressed by neuronets trained under the weights‐and‐structure‐determination (WASD) algorithm. Derived from power activation feed‐forward neuronets, a multi‐input WASD for time‐series neuronet (MI‐WASDTSN) model is presented in this paper. The MI‐WASDTSN is equipped with a novel WASD for time‐series (WASDTS) algorithm, for handling time‐series modeling and forecasting problems. Employing a power sigmoid activation function, the WASDTS algorithm handles the model fitting and validation by determining the optimal input variables number and the weights of the MI‐WASDTSN. More specifically, the WASDTS algorithm finds and holds only the activation function powers that reduce the model's error during validation. Applications on Federal Reserve Bank of St. Louis (FRED) industrial indices under three different patterns of time‐series validate our MI‐WASDTSN model in order to demonstrate its outstanding learning and forecasting performance. In addition, to support and advance the findings of this work, we created a MATLAB repository for interested users, which is freely available via GitHub.

Feedback Control Systems Stabilization Using a Bio-inspired Neural Network

The proportional–integral–derivative (PID) control systems, which have become a standard for technical and industrial applications, are the fundamental building blocks of classical and modern control systems. In this paper, a three-layer feed-forward neural network (NN) model trained to replicate the behavior of a PID controller is employed to stabilize control systems through a NN feedback controller. A novel bio-inspired weights-and-structure-determination (BIWASD) algorithm, which incorporates a metaheuristic optimization algorithm dubbed beetle antennae search (BAS), is used to train the NN model. More presicely, the BIWASD algorithm identifies the ideal weights and structure of the BIWASD-based NN (BIWASDNN) model utilizing a power sigmoid activation function while handling model fitting and validation. The results of three simulated trials on stabilizing feedback control systems validate and demonstrate the BIWASDNN model’s exceptional learning and prediction capabilities, while achieving similar or better performance than the corresponding PID controller. The BIWASDNN model is compared to three other high-performing NN models, and a MATLAB repository is accessible in public through GitHub to encourage and enhance this work.

Multi-input bio-inspired weights and structure determination neuronet with applications in European Central Bank publications

This paper introduces a 3-layer feed-forward neuronet model, trained by novel beetle antennae search weights-and-structure-determination (BASWASD) algorithm. On the one hand, the beetle antennae search (BAS) algorithm is a memetic meta-heuristic optimization algorithm capable of solving combinatorial optimization problems. On the other hand, neuronets trained by a weights-and-structure-determination (WASD) algorithm are known to resolve the shortcomings of traditional back-propagation neuronets, including slow speed of training and local minimum. Combining the BAS and WASD algorithms, a novel BASWASD algorithm is created for training neuronets, and a multi-input BASWASD neuronet (MI-BASWASDN) model is introduced. Using a power sigmoid activation function and while managing the model fitting and validation, the BASWASD algorithm finds the optimal weights and structure of the MI-BASWASDN. Four financial datasets, taken from the European Central Bank publications, validate and demonstrate the MI-BASWASDN model’s outstanding learning and predicting performance. Also included is a comparison of the MI-BASWASDN model to three other well-performing neural network models, as well as a MATLAB kit that is publicly available on GitHub to promote and support this research.

Nonlinearly Activated Recurrent Neural Network for Computing the Drazin Inverse

Four gradient-based recurrent neural networks for computing the Drazin inverse of a square real matrix are developed. Theoretical analysis shows that any monotonically-increasing odd activation function ensures the global convergence performance of defined neural network models. The computer simulation results further substantiate that the considered neural networks could compute the Drazin inverse with accuracy and effectiveness. Moreover, the presented neural networks show superior convergence in the case when the power-sigmoid activation functions are used compared to linear models.

Improved neural solution for the Lyapunov matrix equation based on gradient search

By using the hierarchical identification principle, based on the conventional gradient search, two neural subsystems are developed and investigated for the online solution of the well-known Lyapunov matrix equation. Theoretical analysis shows that, by using any monotonically-increasing odd activation function, the gradient-based neural networks (GNN) can solve the Lyapunov equation exactly and efficiently. Computer simulation results confirm that the solution of the presented GNN models could globally converge to the solution of the Lyapunov matrix equation. Moreover, when using the power-sigmoid activation functions, the GNN models have superior convergence when compared to linear models.

Improved gradient-based neural networks for online solution of Lyapunov matrix equation

By adding different activation functions, a type of gradient-based neural networks is developed and presented for the online solution of Lyapunov matrix equation. Theoretical analysis shows that any monotonically-increasing odd activation function could be used for the construction of neural networks, and the improved neural models have the global convergence performance. For the convenience of hardware realization, the schematic circuit is given for the improved neural solvers. Computer simulation results further substantiate that the improved neural networks could solve the Lyapunov matrix equation with accuracy and effectiveness. Moreover, when using the power-sigmoid activation functions, the improved neural networks have superior convergence when compared to linear models.

Time-varying square roots finding via Zhang dynamics versus gradient dynamics and the former's link and new explanation to Newton–Raphson iteration

Different from conventional gradient-based neural dynamics, a special class of neural dynamics have been proposed by Zhang et al. since 12 March 2001 for online solution of time-varying and static (or termed, time-invariant) problems (e.g., nonlinear equations). The design of Zhang dynamics (ZD) is based on the elimination of an indefinite error-function, instead of the elimination of a square-based positive or at least lower-bounded energy-function usually associated with gradient dynamics (GD) and/or Hopfield-type neural networks. In this paper, we generalize, develop, investigate and compare the continuous-time ZD (CTZD) and GD models for online solution of time-varying and static square roots. In addition, a simplified continuous-time ZD (S-CTZD) and discrete-time ZD (DTZD) models are generated for static scalar-valued square roots finding. In terms of such scalar square roots finding problem, the Newton iteration (also termed, Newton–Raphson iteration) is found to be a special case of the DTZD models (by focusing on the static-problem solving, utilizing the linear activation function and fixing the step-size to be 1). Computer-simulation results via a power-sigmoid activation function further demonstrate the efficacy of the ZD solvers for online scalar (time-varying and static) square roots finding, in addition to the DTZD's link and new explanation to Newton–Raphson iteration.

Continuous and discrete time Zhang dynamics for time-varying 4th root finding

Recently, a special class of neural dynamics has been proposed by Zhang et al. for online solution of time-varying and/or static nonlinear equations. Different from eliminating a square-based positive error-function associated with gradient-based dynamics (GD), the design method of Zhang dynamics (ZD) is based on the elimination of an indefinite (unbounded) error-function. In this paper, for the purpose of online solution of time-varying 4th root, both continuous-time ZD (CTZD) and discrete-time ZD (DTZD) models are developed and investigated. In addition, power-sigmoid activation function is exploited in Zhang dynamics, which makes ZD models possess the property of superior convergence and better accuracy. To summarize generalization for possible widespread application, such approach is further extended to general time-varying nonlinear equations solving. Computer-simulation results demonstrate the efficacy of the ZD models for finding online time-varying 4th root and solving general time-varying equations.

Robustness analysis of the Zhang neural network for online time-varying quadratic optimization

A general type of recurrent neural network (termed as Zhang neural network, ZNN) has recently been proposed by Zhang et al for the online solution of time-varying quadratic-minimization (QM) and quadratic-programming (QP) problems. Global exponential convergence of the ZNN could be achieved theoretically in an ideal error-free situation. In this paper, with the normal differentiation and dynamics-implementation errors considered, the robustness properties of the ZNN model are investigated for solving these time-varying problems. In addition, linear activation functions and power-sigmoid activation functions could be applied to such a perturbed ZNN model. Both theoretical-analysis and computer-simulation results demonstrate the good ZNN robustness and superior performance for online time-varying QM and QP problem solving, especially when using power-sigmoid activation functions.

Convergence Properties Analysis of Gradient Neural Network for Solving Online Linear Equations

A gradient neural network (GNN) for solving online a set of simultaneous linear equations is generalized and investigated in this paper. Instead of the earlier-presented asymptotical convergence, global exponential convergence could be proved for such a class of neural networks. In addition, superior convergence could be achieved using power-sigmoid activation-functions, compared with using linear activation-functions. Computer-simulation results substantiate further the above analysis and efficacy of such neural networks.

Design and Analysis of a General Recurrent Neural Network Model for Time-Varying Matrix Inversion

Following the idea of using first-order time derivatives, this paper presents a general recurrent neural network (RNN) model for online inversion of time-varying matrices. Different kinds of activation functions are investigated to guarantee the global exponential convergence of the neural model to the exact inverse of a given time-varying matrix. The robustness of the proposed neural model is also studied with respect to different activation functions and various implementation errors. Simulation results, including the application to kinematic control of redundant manipulators, substantiate the theoretical analysis and demonstrate the efficacy of the neural model on time-varying matrix inversion, especially when using a power-sigmoid activation function.