Learning Performance Of Neural Network Research Articles

Solving complex nonlinear problems based on gradient-optimized physics-informed neural networks

In recent years, physics-informed neural networks (PINNs) have attracted more and more attention for their ability to quickly obtain high-precision data-driven solutions with only a small amount of data. However, although this model has good results in some nonlinear problems, it still has some shortcomings. For example, the unbalanced back-propagation gradient calculation results in the intense oscillation of the gradient value during the model training, which is easy to lead to the instability of the prediction accuracy. Based on this, we propose a gradient-optimized physics-informed neural networks (GOPINNs) model in this paper, which proposes a new neural network structure and balances the interaction between different terms in the loss function during model training through gradient statistics, so as to make the new proposed network structure more robust to gradient fluctuations. In this paper, taking Camassa-Holm (CH) equation and DNLS equation as examples, GOPINNs is used to simulate the peakon solution of CH equation, the rational wave solution of DNLS equation and the rogue wave solution of DNLS equation. The numerical results show that the GOPINNs can effectively smooth the gradient of the loss function in the calculation process, and obtain a higher precision solution than the original PINNs. In conclusion, our work provides new insights for optimizing the learning performance of neural networks, and saves more than one third of the time in simulating the complex CH equation and the DNLS equation, and improves the prediction accuracy by nearly ten times.

Impact of augmentation methods in online signature verification

The aim of this paper is to investigate the impact of selected data augmentation techniques on the learning performance of neural networks for dynamic signature verification. The paper investigates selected data augmentation techniques in deep learning for verification purpose of dynamic signature. Two neural networks were used as classifiers: MLP and LSTM-FCN. Investigation of five selected augmentation methods and experiments were performed on the open source signature database SVC2004. The authors tested both classifiers without augmentation and then with data augmentation for three extensions of the learning set and three sizes of the user database. They presented the results of the experiments in tabular form for each augmentation method. The results were compared with the existing dynamic signature verification methods and given in the paper.

Gradient-optimized physics-informed neural networks (GOPINNs): a deep learning method for solving the complex modified KdV equation

Recently, the physics-informed neural networks (PINNs) have received more and more attention because of their ability to solve nonlinear partial differential equations via only a small amount of data to quickly obtain data-driven solutions with high accuracy. However, despite their remarkable promise in the early stage, their unbalanced back-propagation gradient calculation leads to drastic oscillations in the gradient value during model training, which is prone to unstable prediction accuracy. Based on this, we develop a gradient optimization algorithm, which proposes a new neural network structure and balances the interaction between different terms in the loss function during model training by means of gradient statistics, so that the newly proposed network architecture is more robust to gradient fluctuations. In this paper, we take the complex modified KdV equation as an example and use the gradient-optimized PINNs (GOPINNs) deep learning method to obtain data-driven rational wave solution and soliton molecules solution. Numerical results show that the GOPINNs method effectively smooths the gradient fluctuations and reproduces the dynamic behavior of these data-driven solutions better than the original PINNs method. In summary, our work provides new insights for optimizing the learning performance of neural networks and improves the prediction accuracy by a factor of 10 to 30 when solving the complex modified KdV equation.

Input Space Partitioning for Neural Network Learning

To improve the learning performance of neural network (NN), this paper introduces an input attribute grouping based NN ensemble method. All of the input attributes are partitioned into exclusive groups according to the degree of inter-attribute promotion or correlation that quantifies the supportive interactions between attributes. After partitioning, multiple NNs are trained by taking each group of attributes as their respective inputs. The final classification result is obtained by integrating the results from each NN. Experimental results on several UCI datasets demonstrate the effectiveness of the proposed method.

A novel data selection method based on shadowed sets

One of the main factors that affect the supervised learning performance of neural network (NN) is training data set. However, for real world problems, it is not always easy to obtain high quality training data sets. In this paper, a novel training data selection method based on shadowed sets is proposed that can select an informative and representative subset of training data to ameliorate the supervised learning performance of NN. The main goal of this work is to improve the generalization ability and diminish misclassification errors of classifier of NN. This paper firstly introduces central idea of shadowed sets. Then the algorithm of proposed data selection method is described in detail. Finally, taking LVQ model as an example, some experiments are made to test validity of this method. The experiment results indicate that using selected sample data to train NN can lessen computation consumption, save training time, guarantee generalization ability, which verify the effectiveness and applicability of proposed data selection method.