Double-layer Neural Network Research Articles

A double‐layer neural network wind speed prediction framework based on training set segmentation and error correction

AbstractHighly accurate wind speed prediction is one of the effective ways to improve wind energy utilization. Therefore, this study proposes a novel double‐layer neural network (D‐NN) framework based on feedforward neural networks. First, a feedforward neural network with an extremely short training time is used as the basic learner to ensure that the D‐NN framework has a short training and optimization time. Then, the double‐layer neural network automatically segments and adjusts the length of the training set using an improved error correction method, thus improving the model's ability to capture sequence features and enhancing the model's generalization performance in a non‐directly‐connected superposition manner. Finally, a genetic algorithm is used to automatically optimize the hyperparameters of each sub‐neural network in the D‐NN and the optimal training set length of each sub‐neural network, resulting in the final optimized prediction model. The accuracy and reliability of the prediction models designed based on the D‐NN framework are verified by comparing several different models. The results of the five different prediction evaluation metrics show that the D‐NN framework can greatly improve the accuracy of wind speed prediction.

Some new nonlinear wave solutions and dynamical behavior of the (3+1)-dimensional Ito equation

Some new nonlinear wave solutions of the (3+1)-dimensional Ito equation, such as three-wave type solutions, lump wave solutions, rogue wave solutions and hybrid solutions of different forms, are studied by the bilinear neural network method. Besides, three-wave solutions are investigated via the single-layer neural network ‘4-3-1’ model, and hybrid soliton solutions between lump-type and strip-like soliton in ‘4-5-1’ model are discussed. Furthermore, rogue wave solutions are studied by the double-layer neural network the ‘4-2-3-1’ and the ‘4-3-3-1’ models, and the interference phenomena between them are analyzed using visual images. We also employed the extended parameter limit method to investigate the collapse degradation behavior of both solutions. Lump solutions and Y-type semi-rational solutions are derived separately.

Performance simulation and diagnosis of faulty states in air-cycle refrigeration systems in civil aircrafts

In the present study, performance simulation models and diagnosis method of faulty states in air-cycle refrigeration systems in civil aircrafts are developed, and the effects of the training data set on the diagnosis accuracy were analyzed. A base model for normal state simulation of air-cycle refrigeration system (ACS) was established based on the existing method with several modifications to improve accuracy and performance. Dedicated models for faulty states were developed according to the causes of fault. Based on the established models, a data set was generated. Preliminary training of multiple two-layer neural network models is performed with the existing data set, and the trained model achieved a validation accuracy of 99.00 % which confirms the feasibility of using double-layer neural network for fault identification and classification based on simulation data. The effects of the training data set to the model are analyzed, and it was concluded that: a data set of about 50,000 entries can be used to train a satisfactory diagnostic model; the proportion of normal state data needs to be reduced appropriately to improve the sensitivity of the model to faulty states; it is possible to largely reduce the number of sensors with smaller effect to the diagnosis accuracy.

Adaptive neural network output feedback robust control of electromechanical servo system with backlash compensation and disturbance rejection

In this paper, the high-performance tracking control of electromechanical servo systems is concerned. A novel neural network state observer is designed to observe the unknown states. Compared with existing neural network observers, the proposed observer has higher observation accuracy and better robustness. The addition of a fixed-weight single-node neural network can effectively improve the approximation ability of the double-layer neural network without adding a huge amount of calculation. With the addition of the new gain adjustment terms, the observer can still achieve high observation accuracy when the neural network approximation performance is poor, and the observation error can be kept arbitrarily small. To cope with the inherent explosion of the complexity problem in the classical backstepping method in controller design, a command filter is utilized. Compared with other results, the command filtering error has also been considered, and compensating signals are designed to eliminate it. The Lyapunov function is used to show the stability of the controller. Extensive comparative simulations and experimental results verify the effectiveness and advancement of the proposed control strategy compared with other controllers.

Reinforcement Learning-Based Optimal Tracking Control for Levitation System of Maglev Vehicle With Input Time Delay

The running environment of a magnetic levitation (maglev) vehicle is complex, with problems such as track irregularity, external disturbance, varying system parameters, and time delays, which bring great challenges to the design of a high-performance levitation controller. In this article, an adaptive neural network controller with input delay compensation and a control parameter optimization scheme is proposed for the electromagnetic levitation system of a maglev vehicle, which can solve the key engineering problems of external disturbance, input time delay, and time-varying mass. Aiming at the problem of input time delay, a sliding-mode surface with time-delay compensation is constructed, and a double-layer neural network and adaptive law are utilized to approximate the uncertain dynamics; thus, a finite time adaptive tracking control law is proposed. Based on the Lyapunov method, the stability of the proposed controller in finite time is analyzed. The proposed method does not only update the input and output weights of the neural network online, but also introduces reinforcement learning (Actor–Critic network) to optimize the key controller parameter in real time and enhance system robustness. Simulation and experimental results show that the proposed controller can effectively suppress the air gap vibration with time delay and uncertain dynamics, and significantly improve the performance of levitation control.

An Artificial Bee Colony-Based Double Layered Neural Network Approach for Solving Quadratic Bi-Level Programming Problems

In the current work, we devised a hybrid method involving a Double-Layer Neural Network (DLNN) for solving a quadratic Bi-Level Programming Problem (BLPP). For an efficient and effective solution of such problems, the proposed potential methodology includes an improved Artificial Bee Colony (ABC) algorithm, a Hopfield Network (HN), and a Boltzmann Machine (BM). The improved ABC algorithm accommodates upper-level decision problems by selecting a set of potential solutions from all combinations of solutions. However, for lower-level decision problem, HN and BM are amalgamated to manifest a DLNN that initially generates its structure by choosing a limited number of units, and will subsequently converge to an optimal solution/unit among those units and hence, constitutes an effective, efficient solution technique.We compared the accuracy, computational time and effectiveness (ability to find the true optimum) of the proposed DLNN with improved-ABC, DLNN with PSO (where PSO replaces the improved-ABC in the upper-level problem of the proposed DLNN with improved-ABC), DLNN with GA (where GAreplaces the improved-ABC in the upper-level of the proposed algorithm) and other conventional approaches and found the proposed DLNN with improved-ABC can yield high quality global optimal solutions with higher accuracy in relatively smaller time.

Spin-qubit noise spectroscopy from randomized benchmarking by supervised learning

We demonstrate a method to obtain the spectra of $1/f$ noises in spin-qubit devices from randomized benchmarking, assisted by supervised learning. The noise exponent, which indicates the correlation within the noise, is determined by training a double-layer neural network with the ratio between the randomized benchmarking results of pulse sequences that correct noise and not. After the training is completed, the neural network is able to predict the exponent within an absolute error of about 0.05, comparable with existing methods. The noise amplitude is then evaluated by training another neural network with the decaying fidelity of randomized benchmarking results from the uncorrected sequences. The relative error for the prediction of the noise amplitude is as low as 5\% provided that the noise exponent is known. Our results suggest that the neural network is capable of predicting noise spectra from randomized benchmarking, which can be an alternative method to measure noise spectra in spin-qubit devices.

DEVELOPMENT OF ALGORITHMIC SUPPORT AND A DEDICATED DEVICE FOR IMAGING THE INTERNAL ORGANS OF THE PATIENT ACCORDING TO MRI AND CT SCANNERS

In work the algorithm of restoration of the images damaged as a result of influence of noise of various nature is considered. The advantages and disadvantages of the existing approaches, as well as the prospects of using artificial neural networks, are noted. A double-layer neural network is used as an image restoration tool, and it is assumed that the location of the damaged pixels is known. A neuron is represented as a 3x3 array, where each element of the array has a pixel color value that corresponds to the value of that color in the palette. The neural network is trained on intact images, while the color difference of pixels acts as a learning criterion. For a more accurate restoration, it is recommended at the training stage to select images similar in color to damaged ones. At the recovery stage, neurons (3x3) are formed around the damaged pixels, so that the damaged pixel is located in the middle of the neuron data array. The damaged pixel is assigned a neuron value depending on the average value of the weights matrix. An algorithm for the restoration of pixels, as well as its software implementation. The simulation was carried out in the RGB palette separately for each channel. To assess the quality of the recovery were selected groups of images with varying degrees of damage. Unlike existing solutions, the algorithm has the simplicity of implementation. The research results show that regardless of the degree of damage (within 50%), about 70% of damaged pixels are restored. Further studies suggest a modification of the algorithm to restore images with enlarged areas of damage, as well as adapting it to restore three-dimensional images.

AN ALGORITHM FOR THE RESTORATION OF PIXELS IMAGE BASED ON THE NEURAL NETWORK

In work the algorithm of restoration of the images damaged as a result of influence of noise of various nature is considered. The advantages and disadvantages of the existing approaches, as well as the prospects of using artificial neural networks, are noted. A double-layer neural network is used as an image restoration tool, and it is assumed that the location of the damaged pixels is known. A neuron is represented as a 3x3 array, where each element of the array has a pixel color value that corresponds to the value of that color in the palette. The neural network is trained on intact images, while the color difference of pixels acts as a learning criterion. For a more accurate restoration, it is ecommended at the training stage to select images similar in color to damaged ones. At the recovery stage, neurons (3x3) are formed around the damaged pixels, so that the damaged pixel is located in the middle of the neuron data array. The damaged pixel is assigned a neuron value depending on the average value of the weights matrix. An algorithm for the restoration of pixels, as well as its software implementation. The simulation was carried out in the RGB palette separately for each channel. To assess the quality of the recovery were selected groups of images with varying degrees of damage. Unlike existing solutions, the algorithm has the simplicity of implementation. The research results show that regardless of the degree of damage (within 50%), about 70% of damaged pixels are restored. Further studies suggest a modification of the algorithm to restore images with enlarged areas of damage, as well as adapting it to restore three-dimensional images.

Motor Fault Detection and Feature Extraction Using RNN-Based Variational Autoencoder

In most of the fault detection methods, the time domain signals collected from the mechanical equipment usually need to be transformed into frequency domain or other high-level data, highly relying on professional knowledge such as signal processing and fault pattern recognition. Contrary to those existing approaches, we proposed a two-stage machine learning analysis architecture which can accurately predict the motor fault modes only by using motor vibration time-domain signals without any complicated preprocessing. In the first stage, the method RNN-based VAE was proposed which is highly suitable for dimension reduction of time series data. In addition to reducing the dimension of sequential data from 150*3 to 25 dimensions, our method furthermore improves the prediction accuracy evaluated by several classification algorithms. While other dimension reduction methods such as Autoencoder and Variational Autoencoder cannot improve the classification accuracy effectively or even decreased. It indicates that the sequential data after dimension reduction via the RNN-based VAE still can maintain the high-dimensional data information. Furthermore, the experimental results demonstrate that it can be well applied to time series data dimension reduction and shows a significant improvement of the prediction performance, even with a simple double-layer Neural Network can reach over 99% of accuracy. In the second stage, Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) are used to further perform the second dimension reduction, such that the different or unknown fault modes can be clearly visualized and detected.

Neural-network-designed pulse sequences for robust control of singlet-triplet qubits

Composite pulses are essential for universal manipulation of singlet-triplet spin qubits. In the absence of noise, they are required to perform arbitrary single-qubit operations due to the special control constraint of a singlet-triplet qubits; while in a noisy environment, more complicated sequences have been developed to dynamically correct the error. Tailoring these sequences typically requires numerically solving a set of nonlinear equations. Here we demonstrate that these pulse sequences can be generated by a well-trained, double-layer neural network. For sequences designed for the noise-free case, the trained neural network is capable of producing almost exactly the same pulses known in the literature. For more complicated noise-correcting sequences, the neural network produces pulses with slightly different line-shapes, but the robustness against noises remains comparable. These results indicate that the neural network can be a judicious and powerful alternative to existing techniques, in developing pulse sequences for universal fault-tolerant quantum computation.

A Double-Layer Neural Network Framework for High-Frequency Forecasting

Nowadays, machine trading contributes significantly to activities in the equity market, and forecasting market movement under high-frequency scenario has become an important topic in finance. A key challenge in high-frequency market forecasting is modeling the dependency structure among stocks and business sectors, with their high dimensionality and the requirement of computational efficiency. As a group of powerful models, neural networks (NNs) have been used to capture the complex structure in many studies. However, most existing applications of NNs only focus on forecasting with daily or monthly data, not with minute-level data that usually contains more noises. In this article, we propose a novel double-layer neural (DNN) network for high-frequency forecasting, with links specially designed to capture dependence structures among stock returns within different business sectors. Various important technical indicators are also included at different layers of the DNN framework. Our model framework allows update over time to achieve the best goodness-of-fit with the most recent data. The model performance is tested based on 100 stocks with the largest capitals from the S8P 500. The results show that the proposed framework outperforms benchmark methods in terms of the prediction accuracy and returns. Our method will help in financial analysis and trading strategy designs.

Groundwater qualitative prediction using artificial neural networks and support vector machine model case Study: Sirjan plain

Groundwater resources are one considered as one of the most common and important resources of drinking, agriculture and industry water. Due to the lowering of groundwater levels and its volatility, groundwater quality is of utmost importance. The aim of this study is to identify the predictive ability of artificial neural network of Multi-Layer Perceptron (MLP) and Support Vector Machine model and adaptive neuro-fuzzy inference system in which the quality of groundwater in Sirjan Plain has been predicted. A case study was conducted on the Sirjan Plain located in the city of Sirjan in Kerman province. For this purpose, the data of rainfall, the water level in wells and UTM coordinates of intended wells have been used as input combinations and qualitative parameters of the water of wells as output parameters. After initial processes such as normalization, for double-layer neural network, 85% of data were used for training and 15% for validation, and the same ration were applied to ANFIS and SVM. After reviewing the fitness statistical criteria such as correlation coefficient (R), and Root Mean Square Error (RMSE), it was observed that neural network presented an acceptable result compared to SVM and ANFIS models.

A collaborative decision-making model for orientation detection

Orientation detection is a fundamental task for biological vision and machine vision. Hubel and Wiesel discovered the selectivity in a simple cell to stimulus of specific orientation, and proposed the famous feedforward model. The Hubel–Wiesel hypothesis attributes the orientation selectivity in a simple cell to the overlapping receptive field centers of its afferent LGN cells along a line, and therefore has several difficulties in the implementation. This paper proposes a collaborative decision-making approach of orientation detection using a double-layer neural network. The single estimation layer estimates the relative position of the contrast edge according to each bottom neuron's response to the contrast stimulus; and the collaborative-decision making layer determines the orientation by optimizing a least square with a unimodular constraint. This computational model cannot just account for orientation selectivity in a flexible way, but be applied to image processing. The statistical experiments found a satisfactory model configuration that balances the computational cost, effectiveness, and efficiency. The simulation experiments yield accurate results invariant to the contrast, and reasonably explain several visual illusions. Moreover, the proposed algorithm outperforms the related image processing algorithms on challenging natural images. The underlying neural mechanism of this model is compatible with the neurobiological findings, and is therefore appropriate for approaches of accomplishing higher level visual tasks.

A neural network-based approach for calculating dissolved oxygen profiles in reservoirs

A Neural Network (NN) modelling approach has been shown to be successful in calculating pseudo steady state time and space dependent Dissolved Oxygen (DO) concentrations in three separate reservoirs with different characteristics using limited number of input variables. The Levenberg–Marquardt algorithm was adopted during training. Pre-processing before training and post processing after simulation steps were the treatments applied to raw data and predictions respectively. Generalisation was improved and over-fitting problems were eliminated: Early stopping method was applied for improving generalisation. The correlation coefficients between neural network estimates and field measurements were as high as 0.98 for two of the reservoirs with experiments that involve double layer neural network structure with 30 neurons within each hidden layer. A simple one layer neural network structure with 11 neurons has yielded comparable and satisfactorily high correlation coefficients for complete data set, and training, validation and test sets of the third reservoir.

Neural network mosaic model for pupillary responses to spatial stimuli.

A neural network mosaic model was developed to investigate the spatial-temporal properties of the human pupillary control system. It was based on the double-layer neural network model developed by Cannon and Robinson and the pupillary dual-path model developed by Sun and Stark. The neural network portion of the model received its input from a sensor array and consisted of a retina-like two-dimensional neuronal layer. The dual-path portion of the model was composed of interconnections of the neurons that formed a mosaic of AC transient and DC sustained paths. The spatial aggregates of the AC and DC signals were input to the AC and DC summing neurons, respectively. Finally, the weighted sum of the aggregate AC and DC signals provided the output for driving the pupillary response. An important property of the model was that it could adaptively learn from training samples by adjustment of the weights. The neural network mosaic model showed excellent performance in simulating both the traditional pupillary phenomena and the new spatial stimulation findings such as responses to change in stimulus pattern and shift of light spot. Moreover, the model could also be used for the diagnosis of clinical deficits and image processing in machine vision.

Double layer associative memories

A Double Layer Neural network which better utilises the error-correcting capability of orthogonal-coded memories is proposed. Computer simulations have shown that the maximum number of stored memories the network is capable of handling before serious errors occur is greatly enhanced. Possible optical implementation schemes are also discussed.