Zhang Neural Network Research Articles

Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises.

Matrix inversion often arises in the fields of science and engineering. Many models for matrix inversion usually assume that the solving process is free of noises or that the denoising has been conducted before the computation. However, time is precious for the real-time-varying matrix inversion in practice, and any preprocessing for noise reduction may consume extra time, possibly violating the requirement of real-time computation. Therefore, a new model for time-varying matrix inversion that is able to handle simultaneously the noises is urgently needed. In this paper, an integration-enhanced Zhang neural network (IEZNN) model is first proposed and investigated for real-time-varying matrix inversion. Then, the conventional ZNN model and the gradient neural network model are presented and employed for comparison. In addition, theoretical analyses show that the proposed IEZNN model has the global exponential convergence property. Moreover, in the presence of various kinds of noises, the proposed IEZNN model is proven to have an improved performance. That is, the proposed IEZNN model converges to the theoretical solution of the time-varying matrix inversion problem no matter how large the matrix-form constant noise is, and the residual errors of the proposed IEZNN model can be arbitrarily small for time-varying noises and random noises. Finally, three illustrative simulation examples, including an application to the inverse kinematic motion planning of a robot manipulator, are provided and analyzed to substantiate the efficacy and superiority of the proposed IEZNN model for real-time-varying matrix inversion.

Taylor O(h³) Discretization of ZNN Models for Dynamic Equality-Constrained Quadratic Programming With Application to Manipulators.

In this paper, a new Taylor-type numerical differentiation formula is first presented to discretize the continuous-time Zhang neural network (ZNN), and obtain higher computational accuracy. Based on the Taylor-type formula, two Taylor-type discrete-time ZNN models (termed Taylor-type discrete-time ZNNK and Taylor-type discrete-time ZNNU models) are then proposed and discussed to perform online dynamic equality-constrained quadratic programming. For comparison, Euler-type discrete-time ZNN models (called Euler-type discrete-time ZNNK and Euler-type discrete-time ZNNU models) and Newton iteration, with interesting links being found, are also presented. It is proved herein that the steady-state residual errors of the proposed Taylor-type discrete-time ZNN models, Euler-type discrete-time ZNN models, and Newton iteration have the patterns of O(h(3)), O(h(2)), and O(h), respectively, with h denoting the sampling gap. Numerical experiments, including the application examples, are carried out, of which the results further substantiate the theoretical findings and the efficacy of Taylor-type discrete-time ZNN models. Finally, the comparisons with Taylor-type discrete-time derivative model and other Lagrange-type discrete-time ZNN models for dynamic equality-constrained quadratic programming substantiate the superiority of the proposed Taylor-type discrete-time ZNN models once again.

ZNN for solving online time-varying linear matrix–vector inequality via equality conversion

In this paper, a special recurrent neural network termed Zhang neural network (ZNN) is proposed and investigated for solving online time-varying linear matrix–vector inequality (LMVI) via equality conversion. That is, by introducing a time-varying vector (of which each element is great than or equal to zero), such a time-varying linear inequality can be converted to a time-varying matrix–vector equation. Then, the ZNN model is developed and investigated for solving online the time-varying matrix–vector equation (as well as the time-varying LMVI) by employing the ZNN design formula. The resultant ZNN model exploits the time-derivative information of time-varying coefficients. Computer-simulation results further demonstrate the efficacy and superiority of the proposed ZNN model for solving online the time-varying LMVI (and the converted time-varying matrix–vector equation).

Solving time-varying quadratic programs based on finite-time Zhang neural networks and their application to robot tracking

In this paper, finite-time Zhang neural networks (ZNNs) are designed to solve time-varying quadratic program (QP) problems and applied to robot tracking. Firstly, finite-time criteria and upper bounds of the convergent time are reviewed. Secondly, finite-time ZNNs with two tunable activation functions are proposed and applied to solve the time-varying QP problems. Finite-time convergent theorems of the proposed neural networks are presented and proved. The upper bounds of the convergent time are estimated less conservatively. The proposed neural networks also have superior robustness performance against perturbation with large implementation errors. Thirdly, feasibility and superiority of our method are shown by numerical simulations. At last, the proposed neural networks are applied to robot tracking. Simulation results also show the effectiveness of the proposed methods.

Different Complex ZFs Leading to Different Complex ZNN Models for Time-Varying Complex Generalized Inverse Matrices

As a special class of recurrent neural network, Zhang neural network (ZNN) has been recently proposed since 2001 for solving various time-varying problems, and has shown high efficiency and excellent performance for solving the problems in the real domain. In this paper, to solve online the time-varying complex generalized inverse (in most cases, the pseudoinverse) problem in the complex domain, a new type of complex-valued ZNN is further proposed and investigated. The design of such a complex ZNN is based on a complex Zhang function (ZF) which is indefinite and quite different from the usual error function (specially, the scalar-valued energy function) in the studies of conventional algorithms. By introducing five different complex ZFs, five different complex ZNN models (termed complex ZNN-I, ZNN-II, ZNN-III, ZNN-IV, and ZNN-V models) are proposed, developed, and investigated for the online solution of the time-varying complex generalized inverse matrices. Theoretical results of convergence analysis are presented to show the desirable properties of complex ZNN models. In addition, we discover the link between the proposed complex ZNN models and the Getz-Marsden dynamic system in the complex domain. Computer-simulation results further demonstrate the effectiveness of complex ZNN models based on different complex ZFs for the time-varying complex generalized inverse matrices.

Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation.

In this brief, a discrete-time Zhang neural network (DTZNN) model is first proposed, developed, and investigated for online time-varying nonlinear optimization (OTVNO). Then, Newton iteration is shown to be derived from the proposed DTZNN model. In addition, to eliminate the explicit matrix-inversion operation, the quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is introduced, which can effectively approximate the inverse of Hessian matrix. A DTZNN-BFGS model is thus proposed and investigated for OTVNO, which is the combination of the DTZNN model and the quasi-Newton BFGS method. In addition, theoretical analyses show that, with step-size h=1 and/or with zero initial error, the maximal residual error of the DTZNN model has an O(τ(2)) pattern, whereas the maximal residual error of the Newton iteration has an O(τ) pattern, with τ denoting the sampling gap. Besides, when h ≠ 1 and h ∈ (0,2) , the maximal steady-state residual error of the DTZNN model has an O(τ(2)) pattern. Finally, an illustrative numerical experiment and an application example to manipulator motion generation are provided and analyzed to substantiate the efficacy of the proposed DTZNN and DTZNN-BFGS models for OTVNO.

Li-function activated ZNN with finite-time convergence applied to redundant-manipulator kinematic control via time-varying Jacobian matrix pseudoinversion

This paper presents and investigates the application of Zhang neural network (ZNN) activated by Li function to kinematic control of redundant robot manipulators via time-varying Jacobian matrix pseudoinversion. That is, by using Li activation function and by computing the time-varying pseudoinverse of the Jacobian matrix (of the robot manipulator), the resultant ZNN model is applied to redundant-manipulator kinematic control. Note that there are nine novelties and differences of ZNN from the conventional gradient neural network in the research methodology. More importantly, such a Li-function activated ZNN (LFAZNN) model has the property of finite-time convergence (showing its feasibility to redundant-manipulator kinematic control). Simulation results based on a four-link planar robot manipulator and a PA10 robot manipulator further demonstrate the effectiveness of the presented LFAZNN model, as well as show the LFAZNN application prospect.

Taylor-type 1-step-ahead numerical differentiation rule for first-order derivative approximation and ZNN discretization

In order to achieve higher computational precision in approximating the first-order derivative and discretize more effectively the continuous-time Zhang neural network (ZNN), a Taylor-type numerical differentiation rule is proposed and investigated in this paper. This rule not only greatly remedies some intrinsic weaknesses of the backward and central numerical differentiation rules, but also overcomes the limitation of the Lagrange-type numerical differentiation rules in ZNN discretization. In addition, a formula is proposed to obtain the optimal step-length of the Taylor-type numerical differentiation rule. Moreover, based on the proposed numerical differentiation rule, the stability, convergence and residual error of the Taylor-type discrete-time ZNN (DTZNN) are analyzed. Numerical experimental results further substantiate the efficacy and advantages of the proposed Taylor-type numerical differentiation rule for first-order derivative approximation and ZNN discretization.

Discrete-time Zhang neural network of O(τ3) pattern for time-varying matrix pseudoinversion with application to manipulator motion generation

In addition to the high-speed parallel-distributed processing property, neural networks can be readily implemented by hardware and thus have been applied widely in various fields. In this paper, via a new numerical-differentiation formula, two Taylor-type discrete-time Zhang neural network (ZNN) models (termed T-ZNN-K and T-ZNN-U models) are first proposed, developed and investigated for online time-varying matrix pseudoinversion. For comparison as well as for illustration, Euler-type discrete-time ZNN models (termed E-ZNN-K and E-ZNN-U models) and Newton iteration are presented. In addition, according to the criterion of whether the time-derivative information of time-varying matrix is explicitly known or not, these discrete-time ZNN models are classified into two categories: (1) models with time-derivative information known (i.e., T-ZNN-K and E-ZNN-K models), and (2) models with time-derivative information unknown (i.e., T-ZNN-U and E-ZNN-U models). Moreover, theoretical analyses show that, the maximal steady-state residual errors (MSSREs) of T-ZNN-K and T-ZNN-U models have an O(τ3) pattern, the MSSREs of E-ZNN-K and E-ZNN-U models have an O(τ2) pattern, whereas the MSSRE of Newton iteration has an O(τ) pattern, with τ denoting the sampling gap. Finally, two illustrative numerical experiments and an application example to manipulator motion generation are provided and analyzed to substantiate the efficacy of the proposed Taylor-type discrete-time ZNN models for online time-varying matrix pseudoinversion.

Zhang neural network for online solution of time-varying linear matrix inequality aided with an equality conversion.

In this paper, for online solution of time-varying linear matrix inequality (LMI), such an LMI is first converted to a time-varying matrix equation by introducing a time-varying matrix, of which each element is greater than or equal to zero. Then, by employing Zhang et al.'s neural dynamic method, a special recurrent neural network termed Zhang neural network (ZNN) is proposed and investigated for solving online the converted time-varying matrix equation as well as the time-varying LMI. Such a ZNN model showed in an explicit dynamics exploits the time-derivative information of time-varying coefficients. In addition, theoretical analysis and results of the proposed ZNN model are discussed and presented to show its excellent performance on solving the time-varying LMI. Computer simulation results further demonstrate the efficacy of the proposed ZNN model for online solution of the time-varying LMI and the converted time-varying matrix equation.

Solving time-varying inverse kinematics problem of wheeled mobile manipulators using Zhang neural network with exponential convergence

A mobile manipulator is a robotic device composed of a mobile platform and a stationary manipulator fixed to the platform. The forward kinematics problem for such mobile manipulators has a mathematical analytic solution; however, the inverse kinematics problem is mathematically intractable (especially for satisfying real-time requirements). To obtain the accurate solution of the time-varying inverse kinematics for mobile manipulators, a special class of recurrent neural network, named Zhang neural network (ZNN), is exploited and investigated in this article. It is theoretically proven that such a ZNN model globally and exponentially converges to the solution of the time-varying inverse kinematics for mobile manipulators. In addition, the kinematics equations of the mobile platform and the manipulator are integrated into one system, and thus the resultant solution can co-ordinate simultaneously the wheels and the manipulator to fulfill the end-effector task. For comparison purposes, a gradient neural network (GNN) is developed for solving time-varying inverse kinematics problem of wheeled mobile manipulators. Finally, we conduct extensive tracking-path simulations performed on a wheeled mobile manipulator using such a ZNN model. The results substantiate the efficacy and high accuracy of the ZNN model for solving time-varying inverse kinematics problem of mobile manipulators. Besides, by comparing the simulation results of the GNN and ZNN models, the superiority of the ZNN model is demonstrated clearly.

From different ZFs to different ZNN models accelerated via Li activation functions to finite-time convergence for time-varying matrix pseudoinversion

In this paper, a special class of recurrent neural network, termed Zhang neural network (ZNN), is investigated for the online solution of the time-varying matrix pseudoinverse. Meanwhile, a novel activation function, named Li activation function, is employed. Then, based on two basic Zhang functions (ZFs) and the intrinsically nonlinear method of ZNN design, two finite-time convergent ZNN models (termed ZNN-1 model and ZNN-2 model) are first proposed and investigated for time-varying matrix pseudoinversion. Such two ZNN models can be accelerated to finite-time convergence to the time-varying theoretical pseudoinverse. The upper bound of the convergence time is also derived analytically via Lyapunov theory. By exploiting the other three simplified ZFs and the extended nonlinearization method, three simplified finite-time convergent ZNN models (termed ZNN-3 model, ZNN-4 model and ZNN-5 model) are sequentially proposed. In addition, the link between the ZNN models and the Getz–Marsden (G–M) dynamic system is discovered and presented in this paper. Computer-simulation results further substantiate the theoretical analysis and demonstrate the effectiveness of ZNN models based on different ZFs for the time-varying matrix pseudoinverse.

Nonlinearly Activated Neural Network for Solving Time-Varying Complex Sylvester Equation

The Sylvester equation is often encountered in mathematics and control theory. For the general time-invariant Sylvester equation problem, which is defined in the domain of complex numbers, the Bartels-Stewart algorithm and its extensions are effective and widely used with an O(n³) time complexity. When applied to solving the time-varying Sylvester equation, the computation burden increases intensively with the decrease of sampling period and cannot satisfy continuous realtime calculation requirements. For the special case of the general Sylvester equation problem defined in the domain of real numbers, gradient-based recurrent neural networks are able to solve the time-varying Sylvester equation in real time, but there always exists an estimation error while a recently proposed recurrent neural network by Zhang et al [this type of neural network is called Zhang neural network (ZNN)] converges to the solution ideally. The advancements in complex-valued neural networks cast light to extend the existing real-valued ZNN for solving the time-varying real-valued Sylvester equation to its counterpart in the domain of complex numbers. In this paper, a complex-valued ZNN for solving the complex-valued Sylvester equation problem is investigated and the global convergence of the neural network is proven with the proposed nonlinear complex-valued activation functions. Moreover, a special type of activation function with a core function, called sign-bi-power function, is proven to enable the ZNN to converge in finite time, which further enhances its advantage in online processing. In this case, the upper bound of the convergence time is also derived analytically. Simulations are performed to evaluate and compare the performance of the neural network with different parameters and activation functions. Both theoretical analysis and numerical simulations validate the effectiveness of the proposed method.

Different Zhang functions resulting in different ZNN models demonstrated via time-varying linear matrix–vector inequalities solving

Our previous work shows that Zhang neural network (ZNN) has the higher efficiency and better performance for solving online time-varying linear matrix–vector inequalities, as compared to the conventional gradient neural network. In this paper, introducing the concept of Zhang function, we further investigate the problem of time-varying linear matrix–vector inequalities solving. Specifically, by defining three different Zhang functions, three types of ZNN models are further elaborately constructed to solve time-varying linear matrix–vector inequalities. The first ZNN model is based on a vector-valued lower-bounded Zhang function and is termed ZNN-1 model. The second one is based on a vector-valued lower-unbounded Zhang function and is termed ZNN-2 model. The third one is based on a transformed lower-unbounded Zhang function and is termed ZNN-3 model. Compared with the ZNN-1 model for solving time-varying linear matrix–vector inequalities, it is surprisedly discovered that the ZNN-2 model incorporates the ZNN-1 model as its special case. Besides, we put research emphasis on the ZNN-3 model for solving time-varying linear matrix–vector inequalities (including its design process, theoretical analysis and simulation verification). When power-sum activation functions are exploited, the ZNN-3 model possesses the property of superior convergence and better accuracy. Computer-simulation results further verify and demonstrate the theoretical analysis and efficacy of the ZNN-3 model for solving time-varying linear matrix–vector inequalities.

From Different Zhang Functions to Various ZNN Models Accelerated to Finite-Time Convergence for Time-Varying Linear Matrix Equation

In addition to the parallel-distributed nature, recurrent neural networks can be implemented physically by designated hardware and thus have been found broad applications in many fields. In this paper, a special class of recurrent neural network named Zhang neural network (ZNN), together with its electronic realization, is investigated and exploited for online solution of time-varying linear matrix equations. By following the idea of Zhang function (i.e., error function), two ZNN models are proposed and studied, which allow us to choose plentiful activation functions (e.g., any monotonically-increasing odd activation function). It is theoretically proved that such two ZNN models globally and exponentially converge to the theoretical solution of time-varying linear matrix equations when using linear activation functions. Besides, the new activation function, named Li activation function, is exploited. It is theoretically proved that, when using Li activation function, such two ZNN models can be further accelerated to finite-time convergence to the time-varying theoretical solution. In addition, the upper bound of the convergence time is derived analytically via Lyapunov theory. Then, we conduct extensive simulations using such two ZNN models. The results substantiate the theoretical analysis and the efficacy of the proposed ZNN models for solving time-varying linear matrix equations.

A new type of recurrent neural networks for real-time solution of Lyapunov equation with time-varying coefficient matrices

A new kind of recurrent neural network is presented for solving the Lyapunov equation with time-varying coefficient matrices. Different from other neural-computation approaches, the neural network is developed by following Zhang et al.'s design method, which is capable of solving the time-varying Lyapunov equation. The resultant Zhang neural network (ZNN) with implicit dynamics could globally exponentially converge to the exact time-varying solution of such a Lyapunov equation. Computer-simulation results substantiate that the proposed recurrent neural network could achieve much superior performance on solving the Lyapunov equation with time-varying coefficient matrices, as compared to conventional gradient-based neural networks (GNN).

Online Solution of Time-Varying Lyapunov Matrix Equation by Zhang Neural Networks

By constructing a matrix-valued unbounded error-function, this paper develops and exploits a new type of recurrent neural networks, named as Zhang neural networks, for the time-varying Lyapunov matrix equation with accuracy and effectiveness. In general, a scalar-valued norm-based energy function is defined for the design and development of the conventional gradient-based neural networks, which could only solve the time-invariant matrix equation exactly. Comparison with some recent patents on the neural networks designed originally for the time-invariant problems solving, the patents relevant to Zhang neural networks is designed for the solution of time-varying problems based on the matrix/ vector-valued error function. An illustrative example substantiates that the presented Zhang neural networks can effectively solve such matrix equation with time-varying coefficients, while the conventional gradient-based neural networks could only approximately approach to the theoretical solution.

Link Between and Comparison and Combination of Zhang Neural Network and Quasi-Newton BFGS Method for Time-Varying Quadratic Minimization

Since 2001, a novel type of recurrent neural network called Zhang neural network (ZNN) has been proposed, investigated, and exploited for solving online time-varying problems in a variety of scientific and engineering fields. In this paper, three discrete-time ZNN models are first proposed to solve the problem of time-varying quadratic minimization (TVQM). Such discrete-time ZNN models exploit methodologically the time derivatives of time-varying coefficients and the inverse of the time-varying coefficient matrix. To eliminate explicit matrix-inversion operation, the quasi-Newton BFGS method is introduced, which approximates effectively the inverse of the Hessian matrix; thus, three discrete-time ZNN models combined with the quasi-Newton BFGS method (named ZNN-BFGS) are proposed and investigated for TVQM. In addition, according to the criterion of whether the time-derivative information of time-varying coefficients is explicitly known/used or not, these proposed discrete-time models are classified into three categories: 1) models with time-derivative information known (i.e., ZNN-K and ZNN-BFGS-K models), 2) models with time-derivative information unknown (i.e., ZNN-U and ZNN-BFGS-U models), and 3) simplified models without using time-derivative information (i.e., ZNN-S and ZNN-BFGS-S models). The well-known gradient-based neural network is also developed to handle TVQM for comparison with the proposed ZNN and ZNN-BFGS models. Illustrative examples are provided and analyzed to substantiate the efficacy of these proposed models for TVQM.

Different Zhang functions leading to different ZNN models illustrated via time-varying matrix square roots finding

In view of the great potential in parallel processing and ready implementation via hardware, neural networks are now often employed to solve online nonlinear matrix equation problems. Recently, a novel class of neural networks, termed Zhang neural network (ZNN), has been formally proposed by Zhang et al. for solving online time-varying problems. Such a neural-dynamic system is elegantly designed by defining an indefinite matrix-valued error-monitoring function, which is called Zhang function (ZF). The dynamical system is then cast in the form of a first-order differential equation by using matrix notation. In this paper, different indefinite ZFs, which lead to different ZNN models, are proposed and developed as the error-monitoring functions for time-varying matrix square roots finding. Towards the final purpose of field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) realization, the MATLAB Simulink modeling and verifications of such ZNN models are further investigated for online solution of time-varying matrix square roots. Both theoretical analysis and modeling results substantiate the efficacy of the proposed ZNN models for time-varying matrix square roots finding.

Design and experimentation of acceleration‐level drift‐free scheme aided by two recurrent neural networks

To solve the joint-angle and joint-velocity drift problems in cyclic motion of redundant robot manipulators, an acceleration-level drift-free (ALDF) scheme subject to a linear equality constraint is proposed, of which the effectiveness is analysed and proved via the theory of second-order system. The scheme is then reformulated into a quadratic program (QP). Furthermore, two recurrent neural networks (RNNs) are developed for solving the resultant QP problem. The first RNN solver is based on Zhang et al 's neural-dynamic method and called Zhang neural network (ZNN), whereas the other is based on the gradient-descent method and called gradient neural network (GNN). Comparison results based on computer simulations between the ZNN and GNN solvers with a circular-path tracking task demonstrate that the ZNN solver has faster convergence and fewer errors. In addition, the hardware experiments of tracking a straight-line path and a rhombic path based on a six degrees of freedom manipulator validate the physical realisability and efficacy of the proposed ALDF scheme and the two RNN QP-solvers. Moreover, the position, velocity and acceleration error analyses indicate the accuracy of the proposed ALDF scheme and the corresponding RNN QP-solvers.