Weight Matrix Of Neural Network Research Articles

DESIGN NEURAL NETWORK-PID CONTROLLER FOR TRAJECTORY TRACKING OF DIFFERENTIAL DRIVE MOBILE ROBOT

This paper proposes the design of a neural network controller based on a sample controller for controlling the trajectory-tracking motion of a differential drive mobile robot (DDMR). Firstly, the trajectory tracking model for DDMR is established based on position error. Next, a perceptron neural network is designed with three hidden layers to control the trajectory tracking of DDMR. The backpropagation algorithm is used to train the neural network with training data obtained from the PID controller with time-varying parameters. The authors have developed this approach and experimentally verified it with minor tracking errors. The neural network's weight matrix (W) and bias vector (b) are updated in real-time, providing an advantage over other methods. The effectiveness of the proposed controller is demonstrated by the DDMR's NURBS trajectory tracking error, which does not exceed 2.17 cm, and the DDMR's motion error, with linear and angular velocities not exceeding 0.004 m/s and 0.0007 rad/s, respectively. The proposed controller can supplement traditional controllers in controlling the trajectory of autonomous mobile robots, thereby improving the ability to generate local trajectories to avoid dynamic obstacles by the neural network

DyLFG: A Dynamic Network Learning Framework Based on Geometry.

Dynamic network representation learning has recently attracted increasing attention because real-world networks evolve over time, that is nodes and edges join or leave the networks over time. Different from static networks, the representation learning of dynamic networks should not only consider how to capture the structural information of network snapshots, but also consider how to capture the temporal dynamic information of network structure evolution from the network snapshot sequence. From the existing work on dynamic network representation, there are two main problems: (1) A significant number of methods target dynamic networks, which only allow nodes to increase over time, not decrease, which reduces the applicability of such methods to real-world networks. (2) At present, most network-embedding methods, especially dynamic network representation learning approaches, use Euclidean embedding space. However, the network itself is geometrically non-Euclidean, which leads to geometric inconsistencies between the embedded space and the underlying space of the network, which can affect the performance of the model. In order to solve the above two problems, we propose a geometry-based dynamic network learning framework, namely DyLFG. Our proposed framework targets dynamic networks, which allow nodes and edges to join or exit the network over time. In order to extract the structural information of network snapshots, we designed a new hyperbolic geometry processing layer, which is different from the previous literature. In order to deal with the temporal dynamics of the network snapshot sequence, we propose a gated recurrent unit (GRU) module based on Ricci curvature, that is the RGRU. In the proposed framework, we used a temporal attention layer and the RGRU to evolve the neural network weight matrix to capture temporal dynamics in the network snapshot sequence. The experimental results showed that our model outperformed the baseline approaches on the baseline datasets.

Consensus Tracking of Singular Multiagent Systems via Sliding-Mode Approach

The admissible consensus tracking control of singular multiagent systems under nonlinear actuator attacks is investigated in this paper. A novel distributed adaptive protocol is designed, which can approximate the actuator attack by adaptive updating the weight matrix of neural network. An integral sliding surface is constructed based on the singularity of agents, and the sliding mode dynamics can reach the sliding mode surface in finite time under the proposed protocols. Sufficient conditions of impulse free and consensus tracking control of the singular multiagent systems are proved, and the attenuation of external disturbances is verified. The effectiveness of the proposed methods is verified through the numerical simulation examples.

Robust Control Based on Adaptive Neural Network for the Process of Steady Formation of Continuous Contact Force in Unmanned Aerial Manipulator

Contact force control for Unmanned Aerial Manipulators (UAMs) is a challenging issue today. This paper designs a new method to stabilize the UAM system during the formation of contact force with the target. Firstly, the dynamic model of the contact process between the UAM and the target is derived. Then, a non-singular global fast terminal sliding mode controller (NGFTSMC) is proposed to guarantee that the contact process is completed within a finite time. Moreover, to compensate for system uncertainties and external disturbances, the equivalent part of the controller is estimated by an adaptive radial basis function neural network (RBFNN). Finally, the Lyapunov theory is applied to validate the global stability of the closed-loop system and derive the adaptive law for the neural network weight matrix online updating. Simulation and experimental results demonstrate that the proposed method can stably form a continuous contact force and reduce the chattering with good robustness.

Sequential Backward Feature Selection Method Based on Stacking Framework

Aiming at the problems of noise and irrelevant feature filtering in data processing, a feature selection method based on the stacking framework is proposed. Use the K-Fold cross-validation method to train and save DNN and SVM-based learners. The prediction results of the base learners are used as the input of the meta-learner, and the logistic regression learning model is trained and saved; comprehensively analyze the correlation coefficients of the fully connected neural network weight matrix and support vector machine, According to the learning results of the meta-learner model, assign different weights to each base learner, calculate the influence factors of each feature, and call the sequence backward search algorithm (SBS) to generate the optimal feature subset. In the experimental stage, a disease diagnosis model was constructed based on the open data set of heart disease research on the Kaggle website, and Stacking-SBS was called to generate the optimal feature subset in the feature space, and the performance comparison experiment of the diagnostic model before and after feature selection was performed, and the method was improved with information. (IG), Chi-square test (Chi) and correlation-based feature selection method (CFS) are compared. The results show that the application of this method can not only reduce model training time, but also significantly improve the model's recall rate and F1 value. In addition, this method is significantly better than the other three feature selection methods in terms of performance improvement. Finally, the open data set of cardiovascular research on the Kaggle website is used to verify the generalization ability of Stacking-SBS. The experimental results show that this method can also significantly improve the performance of the disease diagnosis model.

Adaptive Neural-Network-Based Nonsingular Fast Terminal Sliding Mode Control for a Quadrotor with Dynamic Uncertainty

This paper proposes an adaptive neural-network-based nonsingular fast terminal sliding mode (NN-NFTSMC) approach to address the trajectory tracking control problem of a quadrotor in the presence of model uncertainties and external disturbances. First, the dynamic model of the quadrotor with uncertainty is derived. Then, a control scheme using nonsingular fast terminal sliding mode control (NFTSMC) is proposed to guarantee the finite-time convergence of the quadrotor to its desired trajectory. NFTSMC is firstly formulated for the case that the upper bound of the lumped uncertainty is known in advance. Under this framework, a disturbance observer by using the hyperbolic tangent nonlinear tracking differentiator (TANH-NTD) is designed to estimate the external interference, and a neural network (NN) approximator is used to develop an online estimate of the model uncertainty. Subsequently, adaptive algorithms are designed to compensate the approximation error and update the NN weight matrix. An NN-NFTSMC algorithm is formulated to provide the system with robustness to the model uncertainty and external disturbance. Moreover, Lyapunov-based approach is employed to prove the global stability of the closed-loop system and the finite-time convergence of the trajectory tracking errors. The results of a comparative simulation study with other recent methods illustrate the proposed control method reduces the chattering effectively and has remarkable performance.

Distributed fault‐tolerant parallel formation control of underactuated marine surface vehicles with limited control input

SummaryIn this paper, an adaptive distributed parallel formation control approach is proposed for a group of underactuated marine surface vehicles with actuator fault and limited control input. Firstly, the algebraic topology is employed to model the intervehicle communication network of the multiple vehicles. Both the model uncertainties and environmental disturbance satisfying norm‐bounded conditions are discussed. Secondly, a concise bounded‐feedback adaptive law is developed to compensate the supremum of the neural network weight matrix, rather than each element of the matrix. That could dramatically reduce the computation burden of the algorithm and facilitate its application in practical engineering. Furthermore, a new adaptive parameter is introduced to analyze the input saturation and compensate the energy fading of actuator fault simultaneously. Finally, it is proved that all the signals of the closed‐loop system are semiglobal uniformly ultimately bounded. Numerical examples are presented to demonstrate the performance and superiority of the proposed control strategy in different cases.

Adaptive Boundary Control of Flexible Manipulators with Parameter Uncertainty Based on RBF Neural Network

In this paper, nonlinear dynamical equations of the flexible manipulator with a lumped payload at the free end are derived from Hamilton's principle. The obtained model consists of both distributed parameters and lumped parameters, namely, partial differential equations (PDEs) governing the flexible motion of links and boundary conditions in the form of ordinary differential equations (ODEs). Considering the great nonlinear approximation ability of the radial basis function (RBF) neural network, we propose a combined control algorithm that includes two parts: one is a boundary controller to track the desired joint positions and suppress the vibration of flexible links; another is a RBF neural network designed to compensate for the parametric uncertainties. The iteration criterion of the RBF neural network weight matrix is derived from the extended Lyapunov function. Stabilization analysis is further carried out theoretically via LaSalle’s invariance principle. Finally, the results of the numerical simulation verify that the proposed control law can realize the asymptotic convergence of tracking error and suppression of the elastic vibration as well.

(Invited) Energy Efficient Neural Network Training with Analog Synapses: Challenges and Opportunities

As the exponential performance-per-watt gains from traditional CMOS scaling end, new computing paradigms are being seriously considered. Neural network algorithms such as deep neural networks (DNN) offer compatibility with analog, in-memory computing paradigm with the potential for orders of magnitude improvement over today’s special purpose chips. In this analog paradigm, neural network weights are stored as conductance levels in a physical matrix of tunable nonvolatile resistance memory devices, such as oxide-based resistive memory (ReRAM), phase change memory (PCRAM), three terminal floating gate, and redox memories. Our detailed analysis of a neural processing unit (NPU) core analog block based on these devices can perform individual inference and training operations at energies as low as 10 fJ per operation (100 TOPs/W), achieving greater than a 100x improvement over the best modern digital neural training processors. This will achieve revolutionary performance for modern deep neural network accelerators, which are currently found in computing from smartphones to datacenters. Improvements originate from the efficient mapping of key inference and training logical operations directly to the physical memory array. The multiply-accumulate operation is physically performed by Kirchoff’s Voltage Law (to multiply an input with a weight) and Kirchoff’s current law (to accumulate column weights) (Fig. 1). The enormous energy expense of moving the data from an arithmetic-logic-unit to a register and back several times per training operation is reduced by a factor roughly proportional to the number of rows (or columns) of a neural network weight matrix (typically >100). Analog array benefits have been well established for the inference stage of neural networks and are now under early commercial development. Harnessing analog arrays for neural network training offers to solve a significantly more energy intensive task than inference only – but analog training remains elusive due to device-level challenges. Training places electrical constraints on the analog memory far beyond those for inference-only accelerators. Foremost among these challenges is obtaining sufficient “linearity” and “symmetry” in analog memory devices. The conductance increments during a weight update operation must occur without feedback to verify the correct conductance change, as this iterative process would negate the energy and latency advances over digital CMOS. This open-loop weight update scheme requires that the analog memory changes a predictable conductance value linearly proportional to the number of write pulses, regardless of the initial state. Hence, this is known as “linearity” of an analog device. The linearity often tends to be differ between the positive and negative conductance increments, indicating poor device "symmetry." Furthermore, devices in an array must have significantly overlapping conductance ranges over many cycles. If the analog memory elements do not have sufficient linearity and symmetry, the neural network will suffer unacceptable accuracy loss during the training stage. This is still an area of active research, with several strong two- and three-terminal candidates. Sandia has developed a characterization methodology and modeling code, CrossSim, which allows the training accuracy to be evaluated for experimental devices. Two terminal emerging memories including oxide-based resistive memory (ReRAM) and phase change memory (PCRAM) are very dense and capable of high speed switching. However, typical TaOx ReRAM and GST PCRAM have been evaluated and found to suffer unacceptable accuracy losses, due to the nonlinearity and asymmetry inherent to the devices. In the case of ReRAM this is most likely due to the nonlinear acceleration of the filament growth these devices when switched from the high conductance to low conductance state, or the abrupt amorphization in the low to high conductance change during the PCRAM RESET process. Recently, three-terminal floating gate semiconductor-oxide-nitride-oxide-semiconductor (SONOS) cell, a close cousin of NAND-flash, have been demonstrated to have high training accuracy, low energy, and may be the best near-term weight storage device. However, challenges of SONOS include the relatively long write latency (>1μs), high voltage, limited scalability, and low endurance. The most novel emerging three terminal class of devices known as nonvolatile redox transistors show excellent training behavior with very low write voltages. These three terminal elements, such as Sandia’s Lithium-Ion Synaptic Transistor for Analog Computing (LISTA), utilize the electrochemical mechanism of a lithium-ion battery, where cathode conductivity is modulated by lithium or proton ion transport (charging and discharging the battery). While very promising, these devices face research challenges of increased speed, endurance, and CMOS integration. This presentation will discuss current research directions for key two- and three-terminal analog neural memory devices, with a focus on areas where materials and device innovation can drive significant process in neural training technology. SNL is managed and operated by NTESS under DOE NNSA contract DE-NA0003525. Figure 1

Neural Network-Based Adaptive Consensus Control for a Class of Nonaffine Nonlinear Multiagent Systems With Actuator Faults.

In this paper, the consensus problem is investigated for a class of nonaffine nonlinear multiagent systems (MASs) with actuator faults of partial loss of effectiveness fault and biased fault. To deal with the control difficulty caused by the nonaffine dynamics, a neural network (NN)-based adaptive consensus protocol is developed based on the Lyapunov analysis. The neuron input of the NN uses both the state information and the consensus error information. In addition, the negative feedback term of the NN weight update law is multiplied by an absolute value of the consensus error, which is helpful in improving the consensus accuracy. With the developed adaptive NN consensus protocol, semiglobal consensus with a bounded residual consensus error of the MAS is achieved, and the bounded NN weight matrix is guaranteed. Finally, simulation results show that the developed adaptive NN consensus protocol has advantages of fast convergence rate and good consensus accuracy and has the capability of rapid response with respect to the actuator faults.

Vision-Based Adaptive Neural Positioning Control of Quadrotor Aerial Robot

In this paper, a new vision-based adaptive control algorithm is proposed for the positioning of a quadrotor aerial robot (QAR) with an onboard pin-hole camera. First, the transformation between the position tracking error and image projection error is constructed through the spherical projection method, and then the regulation of the position error is achieved indirectly by stabilizing the image projection error. To overcome the challenge that the dynamics of QAR is physically underactuated, a backstepping-based approach that synthesizes the Lipschitz condition and natural saturation of the inverse tangent function is proposed. In the proposed adaptive controller, an optimized adaptive neural network (NN) means is designed, where only the square of the NN weight matrix’s maximum singular value, not the weight matrix itself, is estimated. Moreover, to facilitate practical application, a novel inertial matrix estimator is introduced in the tuning laws, so that the accurate QAR rotation inertial information is not required. By Lyapunov theory, it is proved that the image projection error converges to an adjustable region of zero asymptotically. The effectiveness of the proposed algorithm has been confirmed by the experimental results.

Neural Networks Based Adaptive Consensus for a Class of Fractional-Order Uncertain Nonlinear Multiagent Systems

Due to the excellent approximation ability, the neural networks based control method is used to achieve adaptive consensus of the fractional-order uncertain nonlinear multiagent systems with external disturbance. The unknown nonlinear term and the external disturbance term in the systems are compensated by using the radial basis function neural networks method, a corresponding fractional-order adaption law is designed to approach the ideal neural network weight matrix of the unknown nonlinear terms, and a control law is designed eventually. According to the designed Lyapunov candidate function and the fractional theory, the systems stability is proved, and the adaptive consensus can be guaranteed by using the designed control law. Finally, two simulations are shown to illustrate the validity of the obtained results.

LTNN: A Layerwise Tensorized Compression of Multilayer Neural Network.

An efficient deep learning requires a memory-efficient construction of a neural network. This paper introduces a layerwise tensorized formulation of a multilayer neural network, called LTNN, such that the weight matrix can be significantly compressed during training. By reshaping the multilayer neural network weight matrix into a high-dimensional tensor with a low-rank approximation, significant network compression can be achieved with maintained accuracy. An according layerwise training is developed by a modified alternating least-squares method with backward propagation for fine-tuning only. LTNN can provide the state-of-the-art results on various benchmarks with significant compression. For MNIST benchmark, LTNN shows 64 × compression rate without accuracy drop. For Imagenet12 benchmark, our proposed LTNN achieves 35.84 × compression of the neural network with around 2% accuracy drop. We have also shown 1.615 × faster on inference speed than the existing works due to the smaller tensor core ranks.

Neural network-based bounded control of robotic exoskeletons without velocity measurements

This paper proposes a novel neural output feedback trajectory tracking controller for robotic exoskeletons. The controller is developed by defining auxiliary dynamics, and utilizing an adaptive feedforward neural network (NN) term to compensate for unknown nonlinear dynamics of the system. The proposed approach only needs position information in both the controller and adaptation rule of the NN weight matrix. In addition, the controller provides an a priori bounded control command. The performance of the controller is validated through simulations and experiments conducted on a lower-limb robotic exoskeleton. It is shown through experiments that the NN term of the controller has assist-as-needed property, such that its contribution in the controller output decreases when the user can follow the desired trajectory in a rehabilitation task.

Neural‐network‐based adaptive leader‐following consensus control for second‐order non‐linear multi‐agent systems

In this study, a novel adaptive neural network (NN)-based leader-following consensus approach is proposed for a class of non-linear second-order multi-agent systems. For the existing NN consensus approaches, to obtain the desired approximation accuracy, the NN-based adaptive consensus algorithms require the number of NN nodes to must be large enough, and thus the online computation burden often are very heavy. However, the proposed adaptive consensus scheme can greatly reduce the online computation burden, because the adaptive adjusting parameters are designed in scalar form, which is the norm of the estimation of the optimal NN weight matrix. According to Lyapunov stability theory, the proposed approach can guarantee the leader-following consensus behaviour of non-linear second-order multi-agent systems to be obtained. Finally, a numerical simulation and a multi-manipulator simulation are carried out to further demonstrate the effectiveness of the proposed consensus approach.

A Classification Model of Identifying Weeds Based on Chaotic Neural Network and the Empirical Mode Decomposition

This paper puts forward a classification method of imitating the human eyes to recognize image as a whole which combined chaotic neural network and the Empirical Mode Decomposition (EMD). The method takes the individual of weeds plant as the research object and utilizes the chaotic neural network, the EMD, Teager energy operator and cluster analysis technology comprehensively. Firstly take the two-dimensional gray image matrix as the chaotic neural network weight matrix directly. Use the chaotic neural network with n neurons expression to iterate and get the one-dimensional output curve. Secondly, make the EMD decomposition for the curve and get the corresponding intrinsic mode function (IMF) curves. Then make the Teager energy transformation for each IMF component and get the average Teager energy. Finally use the fuzzy clustering algorithm to cluster analyze and get the clustering results. It can realize the classification of different categories of weeds through analysis and contrast the results with the original images. The experiment proves the effectiveness of the proposed algorithm for classifying weeds, and it is a universal new method of weeds classification.

Managing spatio-temporal complexity in Hopfield neural network simulations for large-scale static optimization

A simulation methodology, which trades space complexity with time complexity, to create the Hopfield neural network weight matrix, the costliest data structure for simulation of Hopfield neural network algorithm for large-scale optimization problems, is proposed. Modular composition of a weight term of the Hopfield neural network weight matrix for a generic static optimization problem, which facilitates construction and reconstruction of the weights on demand during a simulation, is exposed. Proposed methodology is demonstrated on a static combinatorial optimization problem, namely the Traveling Salesman Problem (TSP), through the algebraic procedure for temporal (versus spatial) weight matrix construction, pseudo code and C/C++ code implementation, and an associated simulation study. The proposed methodology is successfully tested through simulation on a general purpose Windows™-AMD™ platform for up to 1000 city Traveling Salesman Problem instance, which would require approximately no less than 1TB of memory to be allocated simply to instantiate the weight matrix in the memory space of the simulation process.

Calculating Weight Matrix of Neural Network for Resource Leveling

In this paper, a new approach for the computation of the weight matrix of a Hopfield neural network for resource leveling is introduced. The proposed method achieves significantly improved efficiency over the conventional technique of employing the functional expressions of the weights by exploiting the structural properties of the matrices arising in the formulation of the resource leveling problem as a quadratic zero-one optimization. These structural properties are identified and stated in terms of template-matrix contributions of the cost and constraint functions of the quadratic optimization, to the weight matrix of the Hopfield neural network. It is shown that by using these templates, the weight matrix can be filled in directly, based on the early start schedule of a project.

A constrained neural network Kalman filter for price estimation in high frequency financial data.

In this paper we present a neural network extended Kalman filter for modeling noisy financial time series. The neural network is employed to estimate the nonlinear dynamics of the extended Kalman filter. Conditions for the neural network weight matrix are provided to guarantee the stability of the filter. The extended Kalman filter presented is designed to filter three types of noise commonly observed in financial data: process noise, measurement noise, and arrival noise. The erratic arrival of data (arrival noise) results in the neural network predictions being iterated into the future. Constraining the neural network to have a fixed point at the origin produces better iterated predictions and more stable results. The performance of constrained and unconstrained neural networks within the extended Kalman filter is demonstrated on "Quote" tick data from the $/DM exchange rate (1993-1995).