Three-layer Neural Network Research Articles

Roughness prediction of end milling surface for behavior mapping of digital twined machine tools

Background: The quality of machined parts is considered as a relevant factor to evaluate the production performance of machine tools. For mapping the production performance into a digital twin machine tool, a virtual metrology model for surface roughness prediction, which affects products' mechanical capacity and aesthetic performance, is proposed in this paper. Methods: The proposed model applies a three-layer backpropagation neural network by using real-time vibration, force, and current sensor data collected during the end milling machining process. A grid search plan is used to settle down the number of neurons in the middle layer of the backpropagation neural network. Results: The experimental results indicate that the model with multiple signals as input performs better than it with a single signal. In detail, when the model input is the combination of force, vibration, and current sensor data, the prediction accuracy reaches the optimum with the mean absolute percentage error of 1.01%. Conclusions: Compared with the state-of-the-art convolutional neural network method with automatic feature extraction ability and other commonly used traditional machine learning methods, the proposed data preprocessing procedure integrated with a three-layer backpropagation neural network has a minimum prediction error.

Prediction and evaluation the moisture damage resistance of rejuvenated asphalt mixtures based on neural network

The aim of this paper is to establish a model for predicting the moisture damage resistance of the rejuvenated asphalt mixture. Based on the surface free energy theory, the contact angels between asphalt and aggregate were measured with known reagents using the sessile drop method. The adhesion work, spalling work and moisture damage resistance index of the asphalt-aggregate interface system were then calculated. Secondly, the tensile strength ratio of the rejuvenated asphalt mixture was determined, and the influence of six indices, including RAP content, rejuvenator content, adhesion work, spalling work, per cent air voids in bituminous mixtures (VV) and freeze–thaw cycle on the performance of the moisture damage resistance of rejuvenated asphalt mixture was analyzed using the gray correlation entropy method. Finally, a three-layer neural network was used to predict the moisture damage resistance of the rejuvenated asphalt mixture. The results show that as the content of aged asphalt increases, the adhesion work of the asphalt- aggregate interface system decreases. Furthermore, for the system of the same rejuvenated asphalt and different aggregates, the rank of adhesion and water resistance from strong to weak is asphalt-limestone＞asphalt- granite, and the rank of the gray entropy correlation between six indices and moisture damage resistance from strong to weak is adhesion work＞spalling work＞VV＞rejuvenator content＞RAP content＞freeze-thaw cycles. Finally, the accuracy of the neural network predicting the moisture damage resistance of rejuvenated asphalt mix reaches 96.188%, which provides a new method to predict the moisture damage resistance of rejuvenated asphalt mixture.

2D image edge detection enhancement using convolutional neural network

Traditional edge detection operators are usually applied on edge detection in 2D image processing. However, the edge detection system equipped with simple operators has many disadvantages, such as high sensitivity to noise and neglect of significant edge details. This work proposed a method to enhance edge detection with convolutional neural net-work. To overcome the shortcomings of the system using simple edge detection operators in 2D image processing, an edge detection system using convolutional neural network was developed with Python language. In the convolutional neural network, two convolutional layers were designed to extract 2D image features that were relative to edge information. Then a normalization layer was applied to normalize the convoluted output. After that, pre-processing was utilized to denoise and smooth the image input. The final step was edge detection using traditional operators. Experiments were also implemented to verify the improvement of the plugin of the three-layer convolutional neural network in the de-signed edge detection system. Relative frequencies were utilized to quantify the edge de-tection performance. Results showed that the involvement of convolutional neural net-work could strengthen edge detection operators performance obviously in computer vi-sion.

Ballasted Track Behaviour Induced by Absent Sleeper Support and its Detection Based on a Convolutional Neural Network Using Track Data

With development of the heavy-haul railway, the increased axle load and traction weight bring a significant challenge for the service performance and safety maintenance of the railway track. Conducting defect recognition on concrete sleepers and ballast using big data is vital. This paper focused on the detection of absent sleeper support in a ballasted track with an emphasis on the integration of model-based and data-driven methods. To this end, a mathematical model consisting of the wagon, track and wheel–rail contact subsystems was first established to acquire the necessary raw data for the data-driven method, in which the wagon was regarded as a 47-degree-of-freedom multi-body subsystem, and the track was treated as a multi-layer discrete-elastic support beam subsystem with absent sleeper support. Then, an architectural hierarchy of a three-layer convolutional neural network (TLCNN) was developed, which includes three convolutional layers and two pooling layers, and a method for reconstructing one-dimensional sleeper vertical displacement to a two-dimensional time–space matrix was also proposed. Thirdly, verification was carried out by comparing the simulation and experimental results to illustrate the accuracy and reliability of the mathematical model, and the dynamic behaviour of the track with absent sleeper support was investigated. Lastly, the established TLCNN was used to train the raw data of the sleeper vertical displacement and detect the existence of absent sleeper support. Results show that the integration of model-based and data-driven methods was a reliable and effective approach for the detection of absent sleeper support. The proposed TLCNN can acquire and extract robust characteristics in a noisy environment. To handle more complex recognition tasks and further improve performance, deeper CNN models and larger sample sizes should be preferentially considered in practical applications.

Adaptive Neural Network Control Using Nonlinear Information Gain for Permanent Magnet Synchronous Motors.

In this study, an adaptive neural network (NN) control using nonlinear information (NI) gain for permanent magnet synchronous motors (PMSMs) is proposed to improve control and estimation performance. The proposed method consists of a nonlinear controller, a three-layer NN approximator, and NI gain. The nonlinear controller is designed via a backstepping procedure for position tracking. The commutation scheme is designed to implement the PMSM control without the direct-quadrature (DQ) transform. The three-layer NN approximator is designed to estimate the unknown complex function generated by the recursive backstepping process. The NI gains are designed to enhance the control and estimation performance according to the increased tracking errors owing to the load torque and the desired position variations. All of signals in the closed-loop system guarantee the semiglobal uniformly ultimately boundness (UUB) using the Lyapunov stability theorem and the input-to-state stability (ISS) property. The performance of the proposed method was validated by experiments performed using a PMSM testbed.

Feasibility of measuring moisture content of green sand by a low frequency multiprobe detector based on dielectric characteristics

Green sand is a mixture of silica sand, bentonite, water and coal powder, and other additives. Moisture content is an important index to characterize the properties of green sand. Based on the dielectric characteristics of green sand and transmission line theory, a method for rapidly measuring the moisture content of green sand by means of a low frequency multiprobe detector was proposed. A system was constructed, where six detectors with different arrangements and probes were designed. The experimental results showed that the voltage difference of transmission line increases with the increasing frequency before 29 MHz while decreases after 35 MHz. A voltage difference platform occurs in the range of 29–35 MHz, which is suitable for measuring the moisture content due to its insensitivity to frequency. The electric field intensity gradually decreases with the increase of the probe depth, and the intensity of central probe is always greater than that of the edge probe. When the distance of the probe away from the sand sample surface is 80 mm, the electric field intensity of the edge probe is found to be very weak. The optimal excitation frequency for measuring the moisture content of green sand is 29–33 MHz. The optimal detector is the one with one center probe and three edge probes, and their lengths are 80 mm and 60 mm, respectively. The distance between the center and edge probes is 25 mm, and the diameter of probes is 5 mm. Taking the voltage difference of transmission line, bentonite content, coal powder content and compactability as parameters of the input layer, and the moisture content as a parameter of the output layer, a three-layer BP artificial neural network model for predicting the moisture content of green sand was constructed according to the experimental results at 33 MHz. The prediction error of the model is not higher than 3.3% when the moisture content of green sand is within the range of 3wt.%–7wt.%.

Research on Lower Limb Step Speed Recognition Method Based on Electromyography.

Wearable exoskeletons play an important role in people's lives, such as helping stroke and amputation patients to carry out rehabilitation training and so on. How to make the exoskeleton accurately judge the human action intention is the basic requirement to ensure that it can complete the corresponding task. Traditional exoskeleton control signals include pressure values, joint angles and acceleration values, which can only reflect the current motion information of the human lower limbs and cannot be used to predict motion. The electromyography (EMG) signal always occurs before a certain movement; it can be used to predict the target's gait speed and movement as the input signal. In this study, the generalization ability of a BP neural network and the timing property of a hidden Markov chain are used to properly fuse the two, and are finally used in the research of this paper. Experiments show that, using the same training samples, the recognition accuracy of the three-layer BP neural network is only 91%, while the recognition accuracy of the fusion discriminant model proposed in this paper can reach 95.1%. The results show that the fusion of BP neural network and hidden Markov chain has a strong solving ability for the task of wearable exoskeleton recognition of target step speed.

Neural network estimation of kinetic parameters in distributed activation energy model (DAEM) without a priori assumptions for parallel reaction system

In this study, a new estimation method for the kinetic parameters in a distributed activation energy model (DAEM) was designed and developed. In the proposed method, the conversion estimation by the DAEM is regarded as a feedforward computation of a three-layer neural network, and the kinetic parameters of the DAEM are estimated by optimization of the neural network. The proposed method does not require an a priori assumption of the kinetic parameters or mechanism of a parallel reaction system. First, we created reaction data using numerical simulations, and a kinetic analysis using the neural network was performed. The neural network predicted conversion X very accurately; however, reactions with low contributions to the parallel reaction system also appeared. Next, we carried out a kinetic analysis using the neural network with the lower limit on the contribution of the ith reaction Vi*/V*. The lower limit on Vi*/V* did not influence the prediction accuracy of X and had a significant effect on reducing the reactions with low Vi*/V* and the estimation accuracies of the kinetic parameters in the DAEM. The optimal value of the lower limit on Vi*/V* was determined to be 1.0×10−5–1.0×10−4 when using the neural network with 64 hidden layer nodes. Moreover, the prediction accuracy of the proposed method was compared with those of conventional methods — the single-Gaussian method (1-DAEM), double-Gaussian method (2-DAEM), and Miura and Maki method. The kinetic parameters estimated using the proposed method were closer to the true values than those obtained using conventional methods. Moreover, the X value predicted by the proposed method was more accurate than that predicted by conventional methods. The influence of approximation on training data creation was also examined. The estimation accuracy of the neural network was still high but slightly deteriorated when the neural network was optimized using the reaction data created without the approximation.

Organic Memristor with Synaptic Plasticity for Neuromorphic Computing Applications

Memristors have been considered to be more efficient than traditional Complementary Metal Oxide Semiconductor (CMOS) devices in implementing artificial synapses, which are fundamental yet very critical components of neurons as well as neural networks. Compared with inorganic counterparts, organic memristors have many advantages, including low-cost, easy manufacture, high mechanical flexibility, and biocompatibility, making them applicable in more scenarios. Here, we present an organic memristor based on an ethyl viologen diperchlorate [EV(ClO4)]2/triphenylamine-containing polymer (BTPA-F) redox system. The device with bilayer structure organic materials as the resistive switching layer (RSL) exhibits memristive behaviors and excellent long-term synaptic plasticity. Additionally, the device's conductance states can be precisely modulated by consecutively applying voltage pulses between the top and bottom electrodes. A three-layer perception neural network with in situ computing enabled was then constructed utilizing the proposed memristor and trained on the basis of the device's synaptic plasticity characteristics and conductance modulation rules. Recognition accuracies of 97.3% and 90% were achieved, respectively, for the raw and 20% noisy handwritten digits images from the Modified National Institute of Standards and Technology (MNIST) dataset, demonstrating the feasibility and applicability of implementing neuromorphic computing applications utilizing the proposed organic memristor.

Accurate tumor segmentation and treatment outcome prediction with DeepTOP

BackgroundAccurate outcome prediction prior to treatment can facilitate trial design and clinical decision making to achieve better treatment outcome. MethodWe developed the DeepTOP tool with deep learning approach for region-of-interest segmentation and clinical outcome prediction using magnetic resonance imaging (MRI). DeepTOP was constructed with an automatic pipeline from tumor segmentation to outcome prediction. In DeepTOP, the segmentation model used U-Net with a codec structure, and the prediction model was built with a three-layer convolutional neural network. In addition, the weight distribution algorithm was developed and applied in the prediction model to optimize the performance of DeepTOP. ResultsA total of 1889 MRI slices from 99 patients in the phase III multicenter randomized clinical trial (NCT01211210) on neoadjuvant treatment for rectal cancer was used to train and validate DeepTOP. We systematically optimized and validated DeepTOP with multiple devised pipelines in the clinical trial, demonstrating a better performance than other competitive algorithms in accurate tumor segmentation (Dice coefficient: 0.79; IoU: 0.75; slice-specific sensitivity: 0.98) and predicting pathological complete response to chemo/radiotherapy (accuracy: 0.789; specificity: 0.725; and sensitivity: 0.812). DeepTOP is a deep learning tool that could avoid manual labeling and feature extraction and realize automatic tumor segmentation and treatment outcome prediction by using the original MRI images. ConclusionDeepTOP is open to provide a tractable framework for the development of other segmentation and predicting tools in clinical settings. DeepTOP-based tumor assessment can provide a reference for clinical decision making and facilitate imaging marker-driven trial design.

A three layer neural network can represent any multivariate function

In 1987, Hecht-Nielsen showed that any continuous multivariate function can be implemented by a certain type three-layer neural network. This result was very much discussed in neural network literature. In this paper we prove that not only continuous functions but also all discontinuous functions can be implemented by such neural networks.

A Novel Temporal Network-Embedding Algorithm for Link Prediction in Dynamic Networks.

Understanding the evolutionary patterns of real-world complex systems such as human interactions, biological interactions, transport networks, and computer networks is important for our daily lives. Predicting future links among the nodes in these dynamic networks has many practical implications. This research aims to enhance our understanding of the evolution of networks by formulating and solving the link-prediction problem for temporal networks using graph representation learning as an advanced machine learning approach. Learning useful representations of nodes in these networks provides greater predictive power with less computational complexity and facilitates the use of machine learning methods. Considering that existing models fail to consider the temporal dimensions of the networks, this research proposes a novel temporal network-embedding algorithm for graph representation learning. This algorithm generates low-dimensional features from large, high-dimensional networks to predict temporal patterns in dynamic networks. The proposed algorithm includes a new dynamic node-embedding algorithm that exploits the evolving nature of the networks by considering a simple three-layer graph neural network at each time step and extracting node orientation by using Given's angle method. Our proposed temporal network-embedding algorithm, TempNodeEmb, is validated by comparing it to seven state-of-the-art benchmark network-embedding models. These models are applied to eight dynamic protein-protein interaction networks and three other real-world networks, including dynamic email networks, online college text message networks, and human real contact datasets. To improve our model, we have considered time encoding and proposed another extension to our model, TempNodeEmb++. The results show that our proposed models outperform the state-of-the-art models in most cases based on two evaluation metrics.

A Novel BSO Algorithm for Three-Layer Neural Network Optimization Applied to UAV Edge Control

A Novel BSO Algorithm for Three-Layer Neural Network Optimization Applied to UAV Edge Control

An Enhancement Method Based on Long Short-Term Memory Neural Network for Short-Term Natural Gas Consumption Forecasting

Artificial intelligence models have been widely applied for natural gas consumption forecasting over the past decades, especially for short-term consumption forecasting. This paper proposes a three-layer neural network forecasting model that can extract key information from input factors and improve the weight optimization mechanism of long short-term memory (LSTM) neural network to effectively forecast short-term consumption. In the proposed model, a convolutional neural network (CNN) layer is adopted to extract the features among various factors affecting natural gas consumption and improve computing efficiency. The LSTM layer is able to learn and save the long-distance state through the gating mechanism and overcomes the defects of gradient disappearance and explosion in the recurrent neural network. To solve the problem of encoding input sequences as fixed-length vectors, the layer of attention (ATT) is used to optimize the assignment of weights and highlight the key sequences. Apart from the comparisons with other popular forecasting models, the performance and robustness of the proposed model are validated on datasets with different fluctuations and complexities. Compared with traditional two-layer models (CNN-LSTM and LSTM-ATT), the mean absolute range normalized errors (MARNE) of the proposed model in Athens and Spata are improved by more than 16% and 11%, respectively. In comparison with single LSTM, back propagation neural network, support vector regression, and multiple linear regression methods, the improvement in MARNE exceeds 42% in Athens. The coefficient of determination is improved by more than 25%, even in the high-complexity dataset, Spata.

Grape Pseudocercospora Leaf Specked Area Estimation Using Hybrid Genetic Algorithm and Recurrent Neural Network

Grapes are prone to Pseudocercospora vitis fungus which causes Isariopsis leaf speck disease to the crop’s leaves, flower, and most importantly the fruit. Typical manual inspection of vineyard farmers is normally ineffective, destructive, and laborious. To address this, the use of integrated computer vision, machine learning, and computational intelligence techniques were realized to sort out healthy grape leaf image from a fungus-specked leaf image and to estimate the specked area percentage (SAP). A dataset made up of 343 normally healthy and 200 fungus-specked grape leaf images were initially pre-processed and segmented via graph cut prior to feature extraction and selection. Significant features were identified using classification tree (CTree). A multigene genetic programming tool called GPTIPSv2 was utilized to generate the fitness function needed for the optimization process done via genetic algorithm (GA). An optimal hidden neuron counts of 110, 50, and 10 were selected for a three-layered GA-optimized recurrent neural network (GA-RNN). Linear discriminant analysis (LDA) topped other disease recognition models with an accuracy of 99.99%. The developed GA-RNN model outperformed Gaussian process regression (GPR), regression tree (RTree), regression support vector machine (RSVM), and linear regression (RLinear) in performing leaf specked area estimation with an R2 value of 0.822. The developed CTree-LDA2-GA-RNN2 hybrid model has been proven to be the most viable model for this task.

Strain-Mediated Magnetization Switching Behavior in a Bicomponent Nanomagnet

The strain-mediated method is considered to be a promising solution for implementing energy-efficient magnetization rotation. However, current methods strictly require a voltage clock. Here, a multiferroic nanomagnet composed of bicomponent magnetic materials (Terfenol-D:$\mathrm{Ni}$ = 1:2) is developed to study strain-mediated magnetization switching behavior. With micromagnetic simulations, what is demonstrated is that the strict requirements for a precise applied voltage period can be overcome in such a bicomponent nanomagnet, where the threshold of the square-wave voltage pulse width required for complete and repeated magnetization reversal is only 0.42 ns if the amplitude of the voltage is 1 V. Besides deterministic magnetization switching, further study shows that the unique strain-mediated stochastic magnetization reversal behavior of the designed device can be used to mimic the biological neuron. With the application of the derived neural device parameters, a three-layer artificial neural network is further constructed to recognize a handwritten dataset, based on which, an accuracy of more than 98% can be achieved. Overall, these results open an intriguing way toward straintronic memory and neuromorphic systems.

Adaptive Dynamic Programming-Based Method for Signal Evaluation of Energy Transportation System

When a leak event happens in energy transportation system, emergency procedures would be taken by staff according to the abnormal signal evaluation result. The wrong evaluation result leads to inadequate treatment and preparation, and then the leak event would evolve to fire and explosion hazards. To solve it, a data-driven signal evaluation method based on adaptive dynamic programming is proposed in this paper. First, a signal estimation model based on three-layer neural networks is proposed to describe the pressure signal change along pipeline. Then, based on the estimation result of pressure change, a flow signal evaluation method based on value iteration scheme is proposed to obtain the leak flow rate. Moreover, the abnormal signal analysis principles are added into the iterative solving process to close to the actual leak event situation. For different leak scenarios, the proposed method could give the reasonable and reliable result through the designed neural network structures. Finally, different types of case results demonstrate that the proposed method could be applied to the signal evaluation problem.

Research on the impact of artificial intelligence-based e-commerce personalization on traditional accounting methods

With the development of artificial intelligence technology in various fields, the traditional accounting method is no longer applicable to the personalized development of e-commerce industry; Therefore, it is essential to improve the accounting method and construct a personalized recommendation model for e-commerce. Based on this background, this study firstly reconstructs the steps of accounting element recognition in the traditional accounting system and constructs an automated accounting recognition mechanism using BP neural network algorithm, aiming to improve the accuracy and efficiency of accounting element recognition; Secondly, a personalized e-commerce recommendation model based on multiple intelligence is built, which uses intelligent Q-learning algorithm to optimize the recommendation module, aiming to improve the accuracy of personalized recommendation. By comparing the performance of different accounting models under different personalized e-commerce systems, the accounting model proposed in this paper can predict the accounting entries well under the three-layer BP neural network, and the error between the maximum predicted value and the actual value is 0.23%. The recommendation model proposed in the study outperforms the traditional recommendation model and the recommendation model under collaborative filtering algorithm in predicting customers' personal preferences, whose predicted value is closer to the real situation. In summary, both the accounting method and the personalized recommendation model for e-commerce proposed in this study can achieve better application results, thus providing a new idea for the development of the e-commerce industry.

Continuous–Discrete Time Neural Network Observer for Nonlinear Dynamic Systems Application to Vehicle Systems

This paper proposes a novel continuous-discrete (sampled data) time neural network (NSNN) observer for nonlinear systems. It can therefore be applied to systems with a high degree of non-linearity with no prior knowledge of the system dynamics. The proposed observer is a three-layer feedforward neural network that has been intensively trained using the error backpropagation learning algorithm, which includes an e-modification term to ensure robustness of the observer. A structure of the output predictor with a corrective term is added in the structure of the NN observer to overcome the problem of discrete time measurement. Simulations using MATLAB and CarSim are illustrated to demonstrate the performance of the proposed state observer strategy to reconstruct the state variables and parameters of a vehicle system.

Performance Simulation of Multiferroic Neuron Device Driven by an Inclined Monopulse Clock

Multiferroic nanomagnet neuron devices have the advantages of ultra-low power consumption and high integration, which give them promising applications in neuromorphic computing. In this letter, a multiferroic nanomagnet neuron device driven by an inclined monopulse clock is proposed. The strain field direction of the device is at an angle to the nanomagnet's long axis, and the nanomagnet's magnetic moment can be driven to switch randomly 0°/180° by applying a pulse voltage of 0.1 ns pulse width only, thus realizing artificial neuron functions. The numerical model of the neuron device is established based on the Landau–Lifshitz–Gilbert equation. The numerical simulation results indicate that the neuron device can complete high-speed neuromorphic computation with tiny energy use(̴2.65aJ). Additionally, a three-layer artificial neural network based on neuron devices is built. The simulation results demonstrate that the network can recognize handwritten digits in the MNIST dataset at a rate of more than 98% and has a high tolerance for process error. The device has significant advantages over conventional spin neuron devices, including a simple structure, ultra-low energy consumption, fast computation capabilities, and a wide fabrication process error tolerance range. The study results in this letter can offer crucial theoretical recommendations for applying strain magnetoelectronic devices in neuromorphic computing.