Generalization Ability Of Network Research Articles

Torque Ripple Control Strategy of Switched Reluctance Motor Based on BP Neural Network

Switched reluctance motor (SRM) has many advantages, but when the motor is running, the fixed turn-on angle will cause the torque ripple of the motor at different speeds and loads. Therefore, an torque ripple control strategy of switched reluctance motor based on BP neural network is proposed. Firstly, the nonlinear relationship among speed, load torque and opening angle is established by using the fitting generalization ability of BP neural network. In this step, the optimal angle data under the minimum pulsation needs to be obtained by simulation. After collecting data, select and classify the data, and then train and improve the neural network, That nonlinear relationship is introduced into the motor control strategy, so that the motor can automatically adjust its opening angle according to different rotational speeds and load torques and achieve the purpose of reducing torque ripple under different working conditions. Finally, the motor simulation model is built in Matlab/Simulink, and the results are analyzed. This control strategy can control and reduce torque ripple.

Generative Adversarial Training for Supervised and Semi-supervised Learning.

Neural networks have played critical roles in many research fields. The recently proposed adversarial training (AT) can improve the generalization ability of neural networks by adding intentional perturbations in the training process, but sometimes still fail to generate worst-case perturbations, thus resulting in limited improvement. Instead of designing a specific smoothness function and seeking an approximate solution used in existing AT methods, we propose a new training methodology, named Generative AT (GAT) in this article, for supervised and semi-supervised learning. The key idea of GAT is to formulate the learning task as a minimax game, in which the perturbation generator aims to yield the worst-case perturbations that maximize the deviation of output distribution, while the target classifier is to minimize the impact of this perturbation and prediction error. To solve this minimax optimization problem, a new adversarial loss function is constructed based on the cross-entropy measure. As a result, the smoothness and confidence of the model are both greatly improved. Moreover, we develop a trajectory-preserving-based alternating update strategy to enable the stable training of GAT. Numerous experiments conducted on benchmark datasets clearly demonstrate that the proposed GAT significantly outperforms the state-of-the-art AT methods in terms of supervised and semi-supervised learning tasks, especially when the number of labeled examples is rather small in semi-supervised learning.

Computer 3D Vision-Aided Full-3D Optimization of a Centrifugal Impeller

Abstract A computer three-dimensional (3D) vision-aided performance prediction framework for turbomachinery is established in this paper, to improve the accuracy and generalization ability of the artificial neural network (ANN) model under inputs of more than 90 control parameters. In this framework, a RandLA-encoder is built to extract the flow information related to performance and geometric parameters from point cloud data of flow fields inside impellers. By implicitly learning this kind of flow information, the prediction error of the ANN model is reduced by 20–30% compared with the traditional one. Based on this, a full-3D optimization with 91 variables, including arbitrary blade surface and non-axisymmetric (but periodic) hub surface, is conducted on Krain low-speed impeller, aiming at a comprehensive performance improvement. After the optimization, compared to the baseline, the maximum isentropic efficiency of the compressor is increased by 1.6%, the isentropic efficiency at design point is increased by 1%, and the flow range is increased by 5%, with a slight increase in pressure ratio.

Application of Convolutional Neural Network in Emotion Recognition of Ideological and Political Teachers in Colleges and Universities

With the update of Internet technology and the development of we-media, ideological and political education in colleges and universities has been greatly impacted. Higher requirements are put forward for ideological and political teachers in colleges and universities, whose emotions seriously affect the quality and effect of teaching. Aiming at the problems of poor network generalization ability and large computation amount caused by many network parameters in the existing emotion recognition methods, a face emotion recognition method based on convolutional neural network is proposed. The network structure of nested Maxout multilayer perceptron layer is constructed by optimizing the convolutional neural model. Maxout can enhance the feature extraction capability of the convolutional layer of convolutional neural network. Meanwhile, Maxout performs linear combination of target features to select the most effective feature information. Then, the pretraining model is used for emotion recognition training. The strong perception ability of the model for facial features is retained by changing the important parameters. Simulation results demonstrate that this method has a higher recognition rate of face emotion and can effectively achieve accurate face emotion classification.

A new method to improve the generalization ability of neural networks: A case study of nuclear mass training

As one of the fundamental input quantities in nuclear physics, an accurate calculation of the nucleus mass is necessary. Currently, neural network methods improve the accuracy of mass models. However, the reliability for extrapolation cannot be predicted. In this paper, a training method of neural network method is proposed. A fitting method similar to the macroscopic microscopic mass model is used. After fitting the liquid drop model the neural network method is used for training, and then afterwards the parameters are refitted and the neural network model is retrained, and so on several times. Taking the Weizsäcker-Skyrme (WS) model as an example, for known nuclei, it can improve the calculation results by about 20% over the original results. The extrapolation results for unknown neutron-deficient nuclei are also significantly improved. We believe that this method can calculate the masses of partially unknown nuclei more accurately. Further, it may be possible to improve the extrapolation capability of the neural network method using this approach. It is hoped that the present work can be useful for nuclear physics experiments and applications of neural network methods.

An adaptive self-guided wavelet convolutional neural network with compound loss for low-dose CT denoising

Computed Tomography (CT) is an imaging method widely used in clinical, industrial, and other applications. Furthermore, it is one of the common methods of modern clinical medical imaging diagnosis. However, excessive radiation doses in CT scans can cause harm to the human body. Reducing the radiation dose will cause serious degradation of the reconstructed image quality, produce speckle noise and streak artifacts, and affect the accuracy of clinical medical diagnosis. In order to obtain high-quality images while reducing CT radiation dose, we propose an adaptive self-guided wavelet convolutional neural network (ASWCNN) to convert low-dose CT (LDCT) images into normal-dose CT (NDCT) images as true as possible. In our ASWCNN, combining wavelet transform and sub-pixel convolution, a top-down self-guiding structure is proposed as the overall architecture of the network. An adjustable pyramid residual block (APRB) is proposed to self-adaption extract multi-scale and diversity information features as image resolution decreases along with the network. The adjacent scale information fusion block (ASIFB) is proposed to fuse the information features between adjacent scales step by step to improve the stability of network training. At the same time, we use the cross-latitude mixed attention block (CMAB) as the feature enhancement block of our network to enhance the fused feature information, enhance the effective information and suppress the useless information. In addition, we proposed the correction reconstruction block (CRB) to reduce the gap between the obtained denoised CT image and the given NDCT image. Finally, we take the compound loss combining the pixel-level loss, structure-perceived loss, and gradient loss as our loss function. Experimental results show that our proposed method can effectively retain the structure and texture information of CT images while removing noise and artifacts. At the same time, compared with other methods, better subjective visual evaluation and objective quantitative evaluation are obtained, and the network generalization ability is good.

Absorption in the wavemaker by Artificial neural networks (ANNs)

Wave-making theories are becoming available, but their applicability is limited to specific ranges of waves and wavemaker types. Machine learning can also be used to discover nonlinear functional relationships. As a result, based on machine learning, this paper proposes a simple and universal framework for generating and absorbing waves.This framework trains neural networks to determine the transfer function between the wavemaker's free-surface elevation and velocity. To increase the generalization ability of neural networks, penalty term and data augmentation techniques based on wave-making mechanisms are introduced, rather than pure data-driven.As a result, once the wavemaker has the target wave profiles in front of it, it can generate waves while also absorbing reflected waves. Analytical solutions are used to validate the simulated wave profiles and wave orbital velocities, demonstrating that the proposed framework is effective at eliminating the re-reflection wave.The validation for generating the solitary wave and the New Year's wave is then performed, indicating that the generated waves agree very well with the desired wave elevation. The proposed framework can help with wavemaker design in the future, and it does not require any complex theoretical derivation.

A Data Augmentation Method to Optimize Neural Networks for Predicting SOH of Lithium Batteries

Neural Network is an excellent methodology for predicting lithium battery state of health (SOH). However, if the data amount is insufficient, the neural network will be overfitted, which decreass the prediction accuracy of SOH. To solve this issue, a data augmentation method based on random noise superposition is proposed. The original dataset is expanded in this approach, which enhances the neural network’s generalization ability. Moreover, random noises simulate capacity regeneration, capacity dips and sensor errors during the actual operation of lithium batteries, which also improves the adaptive and robustness of the SOH prediction method. The proposed method is validated on mainstream neural networks, including long short-term memory (LSTM) and gated recurrent unit (GRU) neural networks. In terms of the results, the proposed data augmentation method effectively improves the neural network generalization ability and lithium battery SOH prediction accuracy.

Short-term memory neural network-based cognitive computing in sports training complexity pattern recognition.

With the development of Chinese sports, many sports training researchers try to use artificial intelligence technology to study the training methods and training elements of athletes. However, in reality, these methods are often based on different basic training principles, resulting in the reduction in the generalization ability of artificial intelligence networks. This paper studies the complexity of sports training principles by using an artificial intelligence network model. Based on the improved model of dropout optimization algorithm, this paper proposes an artificial intelligence sports training node prediction method based on the combination of dropout optimization algorithm and short-term memory neural network (LSTM), which avoids the establishment of complex sports training models. Based on artificial intelligence operation and maintenance records and sports training core capacity experimental data, the maximum node static estimation of artificial intelligence sports training is realized. The research shows that the node prediction model is established by using the method described in this paper. Through experimental comparison and analysis, the model has high prediction accuracy. Due to the state memory function of LSTM, it has advantages in the prediction of 2000 data on a long time scale. The mean absolute error percentage of the prediction results is less than 3.4%, and the maximum absolute error percentage is less than 5.2%. The artificial intelligence network model in this paper has good generalization ability. Compared with other models, the model proposed in this paper can get more accurate prediction results in sports training of different groups and effectively alleviate the problem of overfitting. Therefore, traditional stadiums and gymnasiums should actively introduce artificial intelligence technology with a more positive attitude, to realize the development and innovation in technology application, service innovation, management efficiency, and function integration.

Research on Sand-Dust Storm Forecasting Based on Deep Neural Network With Stacking Ensemble Learning

Weather forecasting is an important factor affecting production and life. With the development of technology, weather forecasting methods such as weather map forecasting, numerical weather forecasting, and quantitative forecasting methods have emerged. However, these traditional data analysis methods have shortcomings such as incomplete analysis, insufficient objectivity, inability to quantitatively predict the weather, and low prediction accuracy. The import of neural networks into the field of weather forecasting helps to alleviate the above shortcomings and improve the accuracy of weather forecasting. In this paper, the LSTM network and the CNN network are used to predict the sand-dust storm weather. In order to improve the prediction performance, we use the Stacking integration algorithm to fuse the LSTM and CNN models. To improve the experimental scientific and comprehensive, using fully connected network and support vector machine as meta-classifiers, two LSTM-CNN integrated sand-dust storm prediction models are established. At last, the above integrated model is used in the prediction of sand-dust storms in Inner Mongolia. The experimental results show that compared with a single LSTM or CNN model, the Stacking ensemble model has different degrees of improvement in model evaluation indicators such as accuracy, precision, recall, and f1-score. The Stacking ensemble model uses the fully connected network model as the meta-classifier is even better. These prove that the Stacking ensemble algorithm improves the sand-dust storm classification effect and generalization ability of a single neural network to a certain extent.

Skipout: An Adaptive Layer-Level Regularization Framework for Deep Neural Networks

Regularization methods can surprisingly improve the generalization ability of deep neural networks. Among numerous methods, the branch of Dropout regularization is very popular in practice. However, Dropout-like regularization variants have some deficiencies: blindness of regularization, dependency of regularization on layer types, and how to determine the parameters of regularization. In this paper, we propose Skipout framework to tackle these drawbacks by breaking the chain of co-adapted units in consecutive layers. Skipout is a layer-level regularization method that categorizes the layers of a deep network adaptively during training. It divides them into robust and critical layers based on the network architecture and the given task. Instead of turning off some units as in Dropout methods, Skipout recognizes the robust layers and skips them out untrained while sustaining to train the critical layers. The trick of Skipout is in backpropagating the cumulated residual errors via the robust layers, where the activation functions of their units are temporarily set to identity. The units of robust layers are dually activated: by their own functions in the forward pass and by identity in the backward pass. Therefore, Skipout takes advantage of vanishing gradient solutions via robust layers. Moreover, its implementation is simple and applicable to both convolutional and fully-connected layers. Experiments on diverse benchmark datasets and different deep models confirm that Skipout framework can improve the generalization performance of deep neural networks.

Generalization Error Analysis of Neural Networks with Gradient Based Regularization

We study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Applying these methods is equivalent to using neural networks to solve some partial differential equations, mostly in high dimensions in practical applications. In this work, we introduce a general framework to analyze the generalization error of regularized networks. The error estimate relies on two assumptions on the approximation error and the quadrature error. Moreover, we conduct some experiments on the image classification tasks to show that gradient-based methods can significantly improve the generalization ability and adversarial robustness of neural networks. A graphical extension of the gradient-based methods are also considered in the experiments.

Reflection Coefficients Inversion Based on the Bidirectional Long Short-Term Memory Network

Improving the vertical resolution of seismic data to satisfy the demands for detailed characterization of reservoirs is an important part in seismic data processing. The sparse spike inversion(SSI) technique greatly enhances the seismic resolution by assuming that the reflection coefficients follow the sparse distributions. However, its characterization for the spatial structure of thin and thin interbed reservoirs is still limited. In this letter, we propose a novel approach for enhancing seismic resolution using a bidirectional long short-term memory(BiLSTM) neural network by extracting reflection coefficients directly from seismic data. By designing the network architectures, multiple seismic samples map to the specific reflection coefficient. Compared with the SSI method, the BiLSTM neural network provides higher resolution inversion results on model data, which demonstrates the effectiveness of the proposed approach for thin structure characterizations. The convergence speed of the proposed method is fast and the training process is stable. In addition, we conduct the high-resolution seismic data processing on the field data based on the transfer learning technology. High-resolution processing results illustrate the generalization ability and adaptability of the BiLSTM neural network.

Fault Prediction of Elevator Door Lock Based on MPGA-BP Algorithm

The door lock is one of the elevator fault prone parts, and the fault may further lead to personal injury accidents. In the process of elevator operation, the door lock acts frequently, resulting in a large amount of operation data. Using big data method, the fault prediction of elevator door lock could be realized, so as to prevent and reduce accidents. BP (Back Propagation) neural network algorithm and GA-BP (Genetic Algorithm) were used as door lock fault prediction algorithms, but there were some defects in performance. BP algorithm was easy to fall into local minima and the convergence speed was uncertain, while GA-BP algorithm would reduce the genetic diversity of the population. In this paper, we use MPGA (Multiple Population Genetic Algorithm) to improve the initial weight threshold of BP algorithm, adjust the neural network, and establish a door lock fault prediction model. The simulation results show that MPGA-BP model greatly improves the generalization ability and prediction accuracy of BP neural network, compared with traditional BP and GA-BP model.

Multilevel Scattering Center and Deep Feature Fusion Learning Framework for SAR Target Recognition

In synthetic aperture radar (SAR) automatic target recognition (ATR), there are mainly two types of methods: physics-driven model and data-driven network. The physics-driven model can exploit electromagnetic theory to obtain physical properties, while the data-driven network will extract deep discriminant feature of targets. These two types of features represent the target characteristics in scattering domain and image domain, respectively. However, the representation discrepancy caused by the different modalities between them hinders the further comprehensive utilization and fusion of both features. In order to take full advantage of physical knowledge and deep discriminant feature for SAR ATR, we propose a new feature fusion learning framework SDF-Net to combine scattering and deep image features. In this work, we treat the attributed scattering centers (ASC) as set-data instead of multiple individual points, which can well mine the topological interaction among scatterers. Then multi-region multi-scale sub-sets are constructed at both component and target levels. To be specific, the most significant scattering intensity and overall representation in these sub-sets are exploited successively to learn permutation-invariant scattering features according to a set-oriented deep network. The scattering representations can provide mid-level semantic and structural features that are subsequently fused with the complementary deep image features to yield an end-to-end high-level feature learning framework, which helps enhance the generalization ability of networks especially under complex observation conditions. Extensive experiments on Moving and Stationary Target Acquisition and Recognition database verify the effectiveness and robustness of the SDF-Net compared against both typical SAR ATR networks and ASC-based models.

Hcdg: A Hierarchical Consistency Framework for Domain Generalization on Medical Image Segmentation

Modern deep neural networks struggle to transfer knowledge and generalize across domains when deploying to real-world applications. Domain generalization (DG) aims to learn a universal representation from multiple source domains to improve the network generalization ability on unseen target domains. Previous DG methods mostly focus on the data-level consistency scheme to advance the generalization capability of deep networks, without considering the synergistic regularization of different consistency schemes. In this paper, we present a novel Hierarchical Consistency framework for Domain Generalization (HCDG) by ensembling Extrinsic Consistency and Intrinsic Consistency. Particularly, for Extrinsic Consistency, we leverage the knowledge across multiple source domains to enforce data-level consistency. Also, we design a novel Amplitude Gaussian-mixing strategy for Fourier-based data augmentation to enhance such consistency. For Intrinsic Consistency, we perform task-level consistency for the same instance under the dual-task form. We evaluate the proposed HCDG framework on two medical image segmentation tasks, i.e., optic cup/disc segmentation on fundus images and prostate MRI segmentation. Extensive experimental results manifest the effectiveness and versatility of our HCDG framework. Code will be available once accept.

Robust Feature Matching via Hierarchical Local Structure Visualization

Feature matching, which refers to seeking good correspondence between two feature point sets, is a critical prerequisite in many applications of remote sensing and photogrammetry. This work can be viewed as an extension of LSV-ANet. Traditional local structure visualization (LSV) descriptor is very sensitive to outliers existing in the small region around a feature point, which limits the ability of LSV-ANet to recognize fine-grained patterns and its generalizability for complex matching scenes. Thus, in this letter, we propose a hierarchical local structure visualization (HLSV) to solve this issue. Specifically, local structures of feature points are first decoupled by hierarchical tensor and then reconstructed in a learning fashion, which explicitly permits us to learn a nonmutually exclusive relationship for enhancement structure manipulation. In addition, in order to further improve the network generalization ability, we design a permutation-invariant network layer to ensure that the output of model is invariant to the input order. To demonstrate the robustness of the HLSV, extensive experiments on various real remote sensing image pairs for feature matching are conducted. The experiment results reveal that HLSV is superior to the current eight state-of-the-art alternatives.

Gas path parameter prediction of aero-engine based on an autoregressive discrete convolution sum process neural network

In order to improve the approximation ability of neural network to functional and improve the prediction accuracy of time series data, this paper proposes an autoregressive discrete convolution sum process neural network prediction model and applies it to the prediction of aero-engine gas path performance parameters. Coupling threshold wavelet de-noising method is applied to the preprocessing of engine gas path parameters, which can effectively remove the noise in the time series data. Process neurons are applied to artificial neural networks to expand the computational functions of artificial neurons. The process neuron can not only realize the spatial aggregation operation of discrete input data and the connection weight, but also realize the time aggregation operation of the product integral of the continuous input function and the connection weight function. The output of discrete time series is affected by the current input/output, and also by the historical input/output value. Therefore, an autoregressive feedback link is added to the network topology, and the value of the network's output node is used as the input to adjust the network connection weight. The convolution sum operation of discrete time series data can realize the time accumulation effect, so the discrete convolution sum operator is used to replace the integral operator to realize the time aggregation function of the process neuron. Since the integral operation of the continuous function is avoided, the weight training process of the process neural network is effectively simplified, and the accuracy loss in the continuous-time function fitting process of discrete input samples is effectively avoided. Bayesian regularization network weight learning algorithm is used to solve the problems of slow convergence speed and easy to fall into local optimum of back propagation learning algorithm, and improve the generalization ability of neural network. It can be seen from the prediction simulation results of the two engines that the average relative error and root mean square error of the engine gas path parameter prediction based on the autoregressive discrete convolution sum process neural network model can be reduced to 0.67% and 0.35 respectively. The stable and high-precision prediction results show that the network model and weight learning algorithm proposed in this paper have better robustness in functional approximation, and have higher accuracy in time series data prediction.

Image Reconstruction in Electrical Capacitance Tomography Based on Deep Neural Networks

Electrical Capacitance Tomography (ECT) image reconstruction has been largely applied for industrial applications. However, there is still a crucial need to develop a new framework to enhance the quality of reconstructed images and make it faster. Deep learning has recently boomed and applied in many fields since it is good at mapping complicated nonlinear functions based on series of artificial neural networks. In this paper, a novel image reconstruction method based on a deep neural network is proposed. The proposed image reconstruction algorithm mainly uses Long Short-Term Memory (LSTM) deep neural network, which is abbreviated as LSTM-IR algorithm. A big simulation dataset containing 160k pairs of instances is created to train and test the performance of the proposed LSTM-IR algorithm. Each pair of the sample has a predefined permittivity distribution vector and corresponding capacitance vector. The generalization ability and feasibility of the LSTM-IR network are measured using contaminated data, data not included in the training dataset, and experimental data. The preliminary results show that the proposed LSTM-IR method can create fast and more accurate ECT images than traditional and deep learning image reconstruction algorithms.

Construction of a machine-learning-based prediction model for mechanical properties of ultra-fine-grained Fe–C alloy

In recent years, ultra-fine-grained Fe–C alloy have received widespread attention because of their high levels of strength, hardness, and wear resistance. However, these alloy have strict requirements on control of the process parameters during production to meet product specifications. This study was aimed at designing a prediction model for their mechanical properties, which could ultimately promote the industrial development of ultra-fine-grained Fe–C alloy. The effect of process parameters on the mechanical properties of ultra-fine-grained Fe–C alloy was analyzed by electron backscatter diffraction (EBSD) and scanning electron microscope (SEM). By fusing experimental combinations of alloy composition, rolling process, annealing process, and resultant mechanical properties into a dataset for training and verification, a machine-learning-based prediction model of mechanical properties of ultra-fine-grained Fe–C alloy was constructed and tested. When choosing the machine learning algorithm, the best network model was obtained for this application by optimizing a conventional backpropagation (BP) network structure: a three-layer network structure, with a learning rate of 0.5, 18 hidden layer neurons, and activation functions of the hidden and output layers, namely Tansig and Purelin (MATLAB functions), respectively. Finally, the sample data was divided into training set and verification set, and the prediction accuracy, efficiency and generalization ability of BP neural network (BPNN), genetic neural network (GA-BP) and support vector machine (SVM) were compared and analyzed. The superiority of BPNN in the prediction of mechanical properties of ultrafine grained Fe–C alloy was determined, and the mean absolute percentage error (MAPE) of the BPNN for tensile strength, yield strength and uniform elongation are 3.02%, 4.47% and 6.47% respectively, and the number of training steps is 412, which are less than the other two models. The result lays a foundation for the production of high stability and high performance steel products.