Kind Of Neural Network Research Articles

Automatic defect depth estimation for ultrasonic testing in carbon fiber reinforced composites using deep learning

Ultrasonic testing (UT) is commonly used to inspect the geometric shape of internal damage in composite materials and the test results need to be interpreted by trained experts. In this work, an automatic signal classification method based on deep learning is proposed for depth estimation of the detects introduced by low-velocity impact (LVI) in carbon fiber reinforced plastics (CFRPs). Three kinds of neural networks, LSTM, CNN, and CNN-LSTM are used to analyze the attributes with different depths. Then, trained models are applied to identify the depth information of impact damage. The results show that the CNN-LSTM model is a more accurate in-depth classification for LVI defects in CFRP based on A-scan signals than the other two structures.

Multi-factor settlement prediction around foundation pit based on SSA-gradient descent model

With the rise of machine learning, a lot of excellent algorithms are used for settlement prediction. Backpropagation (BP) and Elman are two typical algorithms based on gradient descent, but their performance is greatly affected by the random selection of initial weights and thresholds, so this paper chooses Sparrow Search Algorithm (SSA) to build joint model. Then, two sets of land subsidence monitoring data generated during the excavation of a foundation pit in South China are used for analysis and verification. The results show that the optimization effect of SSA on the gradient descent model is remarkable and the stability of the model is improved to a certain extent. After that, SSA is compared with GA and PSO algorithms, and the comparison shows that SSA has higher optimization efficiency. Finally, select SSA-KELM, SSA-LSSVM and SSA-BP for further comparison and it proves that the optimization efficiency of SSA for BP is higher than other kind of neural network. At the same time, it also shows that the seven influencing factors selected in this paper are feasible as the input variables of the model, which is consistent with the conclusion drawn by the grey relational analysis.

Sex Detection of Chicks Based on Audio Technology and Deep Learning Methods.

The sex detection of chicks is an important work in poultry breeding. Separating chicks of different sexes early can effectively improve production efficiency and commercial benefits. In this paper, based on the difference in calls among one-day-old chicks of different sexes, a sex detection method based on chick calls is designed. Deep learning methods were used to classify the calls of chicks and detect their sex. This experiment studies three different varieties of chicks. The short-time zero-crossing rate was used to automatically detect the endpoints of chick calls in audio. Three kinds of audio features were compared: Spectrogram, Cepstrogram and MFCC+Logfbank. The features were used as the input in neural networks, and there were five kinds of neural networks: CNN, GRU, CRNN, TwoStream and ResNet-50. After the cross-comparison experiment of different varieties of chicks, audio features and neural networks, the ResNet-50 neural network trained with the MFCC+Logfbank audio features of three yellow chick calls had the highest test accuracy of 83% when testing Three-yellow chicks' calls. The GRU neural network trained with the Spectrogram audio features of native chick calls had the highest test accuracy of 76.8% when testing Native chicks' calls. The ResNet-50 neural network trained with Spectrogram audio features of flaxen-yellow chick calls had the highest test accuracy of 66.56%when testing flaxen-yellow chick calls. Multiple calls of each chick were detected, and the majority voting method was used to detect the sex of the chicks. The ResNet-50 neural network trained with the Spectrogram of three yellow chick calls had the highest sex detection accuracy of 95% when detecting the three yellow chicks' sex. The GRU neural network trained with the Spectrogram and cepstrogram of native chick calls and the CRNN network trained with the Spectrogram of native chick calls had the highest sex detection accuracy of 90% when detecting the native chicks' sex. The Twostream neural network trained with MFCC+Logfbank of flaxen-yellow chick calls and the ResNet-50 network trained with the Spectrogram of flaxen-yellow chick calls had the highest sex detection accuracy of 80% when detecting the flaxen-yellow chicks' sex. The results of the cross-comparison experiment show that there is a large diversity between the sex differences in chick calls of different breeds. The method is more applicable to chick sex detection in three yellow chicks and less so in native chicks and flaxen-yellow chicks. Additionally, when detecting the sex of chicks of a similar breed to the training chicks, the method obtained better results, while detecting the sex of chicks of other breeds, the detection accuracy was significantly reduced. This paper provides further perspectives on the sex detection method of chicks based on their calls and help and guidance for future research.

Multi‐event‐triggered adaptive critic control with guaranteed cost for discrete‐time nonlinear nonzero‐sum games

AbstractIn this article, a new event‐based adaptive critic algorithm with multiple triggering conditions is investigated to address multi‐player nonzero‐sum game problems for discrete‐time nonlinear dynamics. In order to improve resource utilization while ensure mutual independence among players, the corresponding novel triggering conditions are designed for each player. The corresponding control input is updated only when the relevant triggering condition is violated. It is emphasized that these triggering conditions are established based on the iteration of the time‐triggered mechanism. Then, according to the setting triggering conditions, we prove that the real cost function possesses a predetermined upper bound, which realizes the cost guarantee of the controlled system. Additionally, the multi‐player closed‐loop system is proved to be asymptotically stable and the multi‐event‐triggered control method is implemented by constructing three kinds of neural networks. Finally, the effectiveness of the developed multi‐event‐triggered control approach is verified through conducting two simulation examples.

Nonlinear inflation forecasting with recurrent neural networks

AbstractMotivated by the recent literature that finds that artificial neural networks (NN) can efficiently predict economic time‐series in general and inflation in particular, we investigate if the forecasting performance can be improved even further by using a particular kind of NN—a recurrent neural network. We use a long short‐term memory recurrent neural network (LSTM) that was proven to be highly efficient for sequential data and computed univariate forecasts of monthly US CPI inflation. We show that even though LSTM slightly outperforms autoregressive model (AR), NN, and Markov‐switching models, its performance is on par with the seasonal autoregressive model SARIMA. Additionally, we conduct a sensitivity analysis with respect to hyperparameters and provide a qualitative interpretation of what the networks learn by applying a novel layer‐wise relevance propagation technique.

Detection and Analysis of Human Cells Based on Artificial Neural Network.

The detection and classification of histopathological cell images is a hot topic in current research. Medical images are an important research direction and are widely used in computer-aided diagnosis, biological research, and other fields. A neural network model based on deep learning is also common in medical image analysis and automatic detection and classification of tissue and cell images. Current medical cell detection methods generally do not consider that the yield is affected by other factors in the topological region, which leads to inevitable errors in the accuracy and generalization of the algorithm; at the same time, the current medical cell imaging methods are too simple to predict the classification markers, which affect the accuracy of cell image classification. This study introduces the concepts of two kinds of neural networks and then constructs a cell recognition model based on the convolution neural network principle and staining principle. In the experimental part, we developed three groups of experiments using the same equation as the experiment and tested the best cell recognition model proposed in this study.

Discontinuous neural networks and discontinuity learning

In the framework of discontinuous function approximation and discontinuity interface detection, we consider an approach involving Neural Networks. In particular, we define a novel typology of Neural Network layers endowed with new learnable parameters and discontinuities in the space of the activations. These layers allow to create a new kind of Neural Networks, whose main property is to be discontinuous, able not only to approximate discontinuous functions but also to learn and detect the discontinuity interfaces. A sound theoretical analysis concerning the properties of the new discontinuous layers is performed, and some tests on discontinuous functions are proposed, in order to assess the potential of such instruments.

Research for Pedestrian Steel Bridge Design of Neural Network in Structural Model Updating

The application of the neural network method in health monitoring and structural system identification has received extensive attention. A reasonable neural network structure is very important for its performance. This paper takes the pedestrian bridge of the Xingfu intersection in Urumqi, China, as the research object and uses MIDAS/Civil to establish a finite element analysis model. Taking the natural vibration frequency obtained from the dynamic test of the actual bridge as the target, two kinds of neural networks are used to predict the structural material parameters. An appropriate bridge model correction method is selected by comparing the prediction results of the BP neural network and the GRNN. The test results show that the pedestrian bridge model based on MIDAS/Civil has a high accuracy, but it still does not meet the actual needs. The modified model based on the BP neural network is close to the actual measured results, and a more accurate finite element analysis model can be established by this method, which makes the modified model closer to the real stress state of the structure.

Compute-in-Memory for Numerical Computations.

In recent years, compute-in-memory (CIM) has been extensively studied to improve the energy efficiency of computing by reducing data movement. At present, CIM is frequently used in data-intensive computing. Data-intensive computing applications, such as all kinds of neural networks (NNs) in machine learning (ML), are regarded as ‘soft’ computing tasks. The ‘soft’ computing tasks are computations that can tolerate low computing precision with little accuracy degradation. However, ‘hard’ tasks aimed at numerical computations require high-precision computing and are also accompanied by energy efficiency problems. Numerical computations exist in lots of applications, including partial differential equations (PDEs) and large-scale matrix multiplication. Therefore, it is necessary to study CIM for numerical computations. This article reviews the recent developments of CIM for numerical computations. The different kinds of numerical methods solving partial differential equations and the transformation of matrixes are deduced in detail. This paper also discusses the iterative computation of a large-scale matrix, which tremendously affects the efficiency of numerical computations. The working procedure of the ReRAM-based partial differential equation solver is emphatically introduced. Moreover, other PDEs solvers, and other research about CIM for numerical computations, are also summarized. Finally, prospects and the future of CIM for numerical computations with high accuracy are discussed.

Partition harvesting of a column-comb litchi harvester based on 3D clustering

Accurate recognition of litchi fruits in orchard environments and acquisition of their coordinate position information are key for realizing successful harvesting using litchi harvesters. However, the existing detection methods are often aimed at large and relatively sparse fruits and thus are inappropriate for small and densely distributed litchi fruits. Therefore, at present, litchi fruit are typically manually harvested, resulting in a low harvesting efficiency that cannot meet the needs of growers. To improve the efficiency of litchi harvesting, this study proposes a column-comb litchi harvesting method based on K-means 3D clustering partitioning, which includes four main steps: (1) Litchi image acquisition and labeling methods are developed. (2) An improved version of the present YOLOv3-tiny network model structure is developed named the YOLOv3-tiny-Litchi network model, and the litchi fruit detection results of five kinds of neural networks, namely, YOLOv3-tiny, YOLOv3-tiny-Litchi, YOLOv4, YOLOv5x and Faster R-CNN, are compared. (3) A depth camera is used to obtain the 3D coordinates of litchi fruits, and the K-means clustering algorithm is used to divide the litchi harvesting area to obtain the optimal partitioning results. (4) Field experiments on litchi harvesting are reported. The experimental results show that the improved YOLOv3-tiny-Litchi model can recognize litchi fruits more accurately; the recall rate is 78.99%, the precision rate is 87.43%, and the F1 score is 0.83. The results of 3D clustering partitioning show that when K is equal to 6, the optimal harvesting rate is 90.03%, which satisfies the theoretical requirements of litchi harvesters. The field experiments show that when the theoretical number of partitions is 6, the average harvesting rate of the litchi harvester is 91.15% and the recognition rate is 88.39%, and the harvesting efficiency of mechanical partition harvesting is 1.4 times that of mechanical harvesting without partitioning and 2 times that of manual harvesting, essentially meeting the operating requirements. These research results can serve as a reference for the mechanization of litchi harvesting.

A Stacking Ensemble Deep Learning Model for Bitcoin Price Prediction Using Twitter Comments on Bitcoin

Cryptocurrencies can be considered as mathematical money. As the most famous cryptocurrency, the Bitcoin price forecasting model is one of the popular mathematical models in financial technology because of its large price fluctuations and complexity. This paper proposes a novel ensemble deep learning model to predict Bitcoin’s next 30 min prices by using price data, technical indicators and sentiment indexes, which integrates two kinds of neural networks, long short-term memory (LSTM) and gate recurrent unit (GRU), with stacking ensemble technique to improve the accuracy of decision. Because of the real-time updates of comments on social media, this paper uses social media texts instead of news websites as the source data of public opinion. It is processed by linguistic statistical method to form the sentiment indexes. Meanwhile, as a financial market forecasting model, the model selects the technical indicators as input as well. Real data from September 2017 to January 2021 is used to train and evaluate the model. The experimental results show that the near-real time prediction has a better performance, with a mean absolute error (MAE) 88.74% better than the daily prediction. The purpose of this work is to explain our solution and show that the ensemble method has better performance and can better help investors in making the right investment decision than other traditional models.

Design of Convolutional Neural Network Based on FPGA

Recently with the rapid development of artificial intelligence AI, various deep learning algorithms represented by Convolutional Neural Networks (CNN) have been widely utilized in various fields, showing their unique advantages; especially in Skin Cancer (SC) imaging Neural networks (NN) are methods for performing machine learning (ML) and reside in what's called deep learning (DL). DL refers to the utilization of multiple layers during a neural network to perform the training and classification of data. The Convolutional Neural Networks (CNNs), a kind of neural network and a prominent machine learning algorithm go through multiple phases before they get implemented in hardware to perform particular tasks for a specific application. State-of-the-art CNNs are computationally intensive, yet their parallel and modular nature make platforms like Field Programmable Gate Arrays (FPGAs) compatible with the acceleration process. The objective of this paper is to implement a hardware architecture capable of running on an FPGA platform of a convolutional neural network CNN, for that, a study was made by describing the operation of the concerned modules, we detail them then we propose a hardware architecture with RTL scheme for each of these modules using the software ISE (Xilinx). The main objective is to show the efficiency of such a realization compared to a GPU based execution. An experimental study is accomplished for the PH2 database set of benchmark images. The proposed FPGA-based CNN design gives competitive results and shows well its efficiency.

Neural Network-Based Approach for Evaluating College English Teaching Methodology

A fair and scientific method of evaluating collegiate English instruction is currently lacking. Traditional methods are commonly used to evaluate the quality of college English education in terms of the standard statistical analysis evaluation model. As the evaluation of English teaching is a nonlinear issue, the above strategies have achieved some good results but have certain limitations. They are not scientific and objective in the selection of evaluation indicators, or in the setting of evaluation index weights, and there is a certain degree of subjectivity. Artificial neural network (ANN) is widely used in massive fields due to its characteristics of nonlinear processing, adaptive learning, and high fault tolerance. As a kind of neural network, BP neural network has strong nonlinear mapping ability, so it is feasible and scientific to solve the nonlinear relationship of college English teaching evaluation (ETE). Therefore, in this work, we first designed an ETE system index. Then, a strategy of college ETE with BP network is designed, which can carry out high-performance modeling for the designed teaching evaluation index. In order to alleviate the issue of slow convergence speed for BP network and fall into local optimum, this work also combines particle swarm algorithm with BP network to further improve network performance. Massive experiments have proved the reliability and effectiveness of this work.

Analysis and prediction of the mechanical behavior of corrugated plate as primary support in tunnels with elastoplastic constitution

Compared to the traditional primary support mainly made of shot concrete, new-type corrugated-plate support, which has been maturely used in pipes and culverts, has many advantages like good flexibility and high efficiency. To study its mechanical characteristics in tunnels, several finite-element models with different constitutions, sizes, loads and yield stresses were built and simulated. And for its correctness and utility, a verification experiment and 2 kinds of neural networks were also carried out. The result shows that: firstly, 90% of the result in the simulation is in accord with the experiment. Then, in the elastoplastic model, for vertical displacement, the maximum is at the vault. For Von-Mises stress, the stress is higher at the vault and invert (241 MPa as an example), while it is relatively lower at part of the haunch (7 MPa as an example). Next, in terms of the increase of d (maximum vertical displacement), there are 2 phases: elastic phase and plastic phase. In the former phase, d increases steadily, while in the latter phase it surges rapidly. In terms of s (maximum Von-Mises stress), there are 3 phases: elastic phase, transition phase and plastic phase. In elastic and plastic phase, s ascends normally. But in the transition phase, s increases slowly until all the structure turns plastic. Moreover, as for the bearing capacity, the influence of t is the biggest, followed by h and w. Finally, for prediction, neural networks are a good choice. Meanwhile, different settings are needed for predicting different values. For predicting d, LM (Levenberg-Marquardt) algorithm with appropriate number of neurons is efficient (13 s for each training, MSE (Mean Squared Error) = 2.24). For predicting s, BR (Bayesian Regularization) algorithm is better with enough neurons (120 s for each training, MSE = 0.7). Since the conditions are finite, within an applicable scope, this research provides some innovative suggestions for applying new-type corrugated-plate support and neural networks in tunnelling field.

Error Compensation Enhanced Day-Ahead Electricity Price Forecasting

The evolution of electricity markets has led to increasingly complex energy trading dynamics and the integration of renewable energy sources as well as the influence of several external market factors contributed towards price volatility. Therefore, day-ahead electricity price forecasting models, typically using some kind of neural network, play a crucial role in the optimal behavior of market agents. The most prominent models and benchmarks rely on improving the accuracy of predictions and the time for convergence by some sort of a priori processing of the dataset that is used for the training of the neural network, such as hyperparameter tuning and feature selection techniques. What has been overlooked so far is the possible benefit of a posteriori processing, which would consider the effects of parameters that could refine the predictions once they have been made. Such a parameter is the estimation of the residual training error. In this study, we investigate the effect of residual training error estimation for the day-ahead price forecasting task and propose an error compensation deep neural network model (ERC–DNN) that focuses on the minimization of prediction error, while reinforcing error stability through the integration of an autoregression module. The experiments on the Nord Pool power market indicated that this approach yields improved error metrics when compared to the baseline deep learning structure in different training scenarios, and the refined predictions for each hourly sequence shared a more stable error profile. The proposed method contributes towards the development of more flexible hybrid neural network models and the potential integration of the error estimation module in future benchmarks, given a small and interpretable set of hyperparameters.

Performance Analysis of Three kinds of Neural Networks in the Classification of Mask Images

The global spread of COVID-19 has led to a massive increase in public demand for effective measures to slow the spread of the virus. In response, the World Health Organization has advised people to wear face masks every day to stop the spread of the virus. In facial mask recognition, traditional manual feature extraction methods are cumbersome and inaccurate, considering the various types of masks and other practical problems. This paper considers various neural network models comprehensively and selects three classical CNN networks, including VGG, ResNet, and DenseNet. After using the data expansion and optimization methods, good data results are obtained for the three network models. Remarkably, the improved DenseNet accuracy can reach 99.55%.

Cross Validation Voting for Improving CNN Classification in Grocery Products

The development of deep neural networks that has been carried out in recent years allows solving highly complex computer vision classification problems. Often, although the results obtained with these classifiers are high, there are certain sectors that seek greater accuracy from these systems. Increasing the accuracy of neural networks can be achieved through ensemble learning, which combines different classifiers with the aim of selecting a winner based on different criteria about them. These techniques have traditionally shown good results although they involve training models of different nature and can even produce an overfitting with respect to the training data, so datasets must be chosen to correctly evaluate the result. In this paper, a Cross-Validation-Voting (CVV) technique for grocery product classification is presented. This technique improves several single state-of-the-art classifiers without combining different ones and avoids the problems of overfitting with respect to the training set. The single classifiers are trained multiple times against distributed sets to show how the results obtained to date from the classification of a well-known dataset are improved. In this dataset, an extensive test set was previously selected by the authors to show comparable results with other papers in the literature. The technique is valid not only for vision nets and can be used to solve numerous problems with different kinds of neural networks and classifiers.

Piston detection in segmented telescopes via multiple neural networks coordination of feature-enhanced images

High-precision piston detection within a large capture range is a key for segmented telescopes. In this paper, we propose a simple and efficient piston detection method based on multiple neural networks coordination. By setting a mask with a sparse multi-subpupil configuration at conjugate plane of the segmented mirror, a new dataset that is extremely sensitive to the piston is created. And two kinds of neural networks are built for different stages of detection, which ensures the method is of both large-scale and high-precision. Simulation shows that the piston can be detected in the range of the coherence length of the operating light with a sub-nanometer scale precision by this method. This method is robust and does not require complex hardware. It can be widely applied in segmented and deployable primary mirror telescopes.

Estimated Pao2: A Continuous and Noninvasive Method to Estimate Pao2 and Oxygenation Index.

To validate a continuous and noninvasive method to estimate hypoxemia severity for all Spo2 values. All patients who had an arterial blood gas and simultaneous continuous noninvasive monitoring from 2011 to 2019 at Boston Children's Hospital (Boston, MA) PICU. External cohort at Sainte-Justine Hospital PICU (Montreal, QC, Canada) from 2017 to 2020. We estimated the Pao2 using three kinds of neural networks and an empirically optimized mathematical model derived from known physiologic equations. We included 52,879 Pao2 (3,252 patients) in the derivation dataset and 12,047 Pao2 (926 patients) in the validation dataset. The mean function on the last minute before the arterial blood gas had the lowest bias (bias -0.1% validation cohort). A difference greater than or equal to 3% between pulse rate and electrical heart rate decreased the intraclass correlation coefficients (0.75 vs 0.44; p < 0.001) implying measurement noise. Our estimated Pao2 equation had the highest intraclass correlation coefficient (0.38; 95% CI, 0.36-0.39; validation cohort) and outperformed neural networks and existing equations. Using the estimated Pao2 to estimate the oxygenation index showed a significantly better hypoxemia classification (kappa) than oxygenation saturation index for both Spo2 less than or equal to 97% (0.79 vs 0.60; p < 0.001) and Spo2 greater than 97% (0.58 vs 0.52; p < 0.001). The estimated Pao2 using pulse rate and electrical heart rate Spo2 validation allows a continuous and noninvasive estimation of the oxygenation index that is valid for Spo2 less than or equal to 97% and for Spo2 greater than 97%. Display of continuous analysis of estimated Pao2 and estimated oxygenation index may provide decision support to assist with hypoxemia diagnosis and oxygen titration in critically ill patients.

Revisiting the dynamics of Bose-Einstein condensates in a double well by deep learning with a hybrid network

Deep learning, accounting for the use of an elaborate neural network, has recently been developed as an efficient and powerful tool to solve diverse problems in physics and other sciences. In the present work, we propose a novel learning method based on a hybrid network integrating two different kinds of neural networks: Long Short-Term Memory(LSTM) and Deep Residual Network(ResNet), in order to overcome the difficulty met in numerically simulating strongly-oscillating dynamical evolutions of physical systems. By taking the dynamics of Bose-Einstein condensates in a double-well potential as an example, we show that our new method makes a high efficient pre-learning and a high-fidelity prediction about the whole dynamics. This benefits from the advantage of the combination of the LSTM and the ResNet and is impossibly achieved by a single network in the case of direct learning. Our method can be applied for simulating complex cooperative dynamics in a system with fast multiple-frequency oscillations with the aid of auxiliary spectrum analysis.