Neural Network Data Processing Research Articles

Survey of Optimization Algorithms in Modern Neural Networks

The main goal of machine learning is the creation of self-learning algorithms in many areas of human activity. It allows a replacement of a person with artificial intelligence in seeking to expand production. The theory of artificial neural networks, which have already replaced humans in many problems, remains the most well-utilized branch of machine learning. Thus, one must select appropriate neural network architectures, data processing, and advanced applied mathematics tools. A common challenge for these networks is achieving the highest accuracy in a short time. This problem is solved by modifying networks and improving data pre-processing, where accuracy increases along with training time. Bt using optimization methods, one can improve the accuracy without increasing the time. In this review, we consider all existing optimization algorithms that meet in neural networks. We present modifications of optimization algorithms of the first, second, and information-geometric order, which are related to information geometry for Fisher–Rao and Bregman metrics. These optimizers have significantly influenced the development of neural networks through geometric and probabilistic tools. We present applications of all the given optimization algorithms, considering the types of neural networks. After that, we show ways to develop optimization algorithms in further research using modern neural networks. Fractional order, bilevel, and gradient-free optimizers can replace classical gradient-based optimizers. Such approaches are induced in graph, spiking, complex-valued, quantum, and wavelet neural networks. Besides pattern recognition, time series prediction, and object detection, there are many other applications in machine learning: quantum computations, partial differential, and integrodifferential equations, and stochastic processes.

Level of fire danger of the local territory

The object of the study is the fire risk of the local area. The problem to be solved is to take into account most of the significant parameters in the territorial placement of fire-rescue units of different functional capacities. As part of the solution to this problem, a technique for assessing the fire risk of a large-scale local area has been developed. The methodology is focused on local territories of a large area with a low population density. A special feature of the proposed method is the differentiated fire risk assessment of each point of the surface plane. For such an assessment, the parameters that are decisive from the point of view of impact on the fire hazard are analyzed and structured. The specified factors include the spatial distribution of population density and buildings, the transport and communication network, the spatial distribution of the density and type of vegetation, and statistical data on landscape fires. The use of existing geo-informational resources in real time is foreseen. A new approach of ranking the fire risk of the elementary plane of the territory in accordance with the necessary number of resources of rescue units to ensure the appropriate level of safety is proposed. Neural network data processing methods were used to compare local area parameters with fire risk ranks. A neural network capable of comparing the fire risk of the territory with its parameters was obtained. The functionality of the developed methodology was tested and the fire risk levels of an arbitrary area were graded with an average degree of correlation of 0.97. The proposed method allows for assessment and correction of the state of provision of local territories with civil protection resources. The developed methodology is especially relevant when creating new fire and rescue units of territorial communities.

Generalized inverse matrix-recurrent neural network data processing algorithm for multiwavelength pyrometer

A generalized inverse matrix-recurrent neural network (GIM-RNN) data processing algorithm for unknown emissivity was proposed to measure high-temperature by multiwavelength pyrometer (MWP). First, emissivity classification was realized quickly according to a solution from generalized inverse algorithm to underdetermined multiwavelength equation group. According to the relationship between the emissivity of 2 adjacent channels of the 6 channel thermometer used in this paper, 243 emissivity models (1 * 3 * 3 * 3 * 3 * 3) were designed for classification. Twelve of them were shown on the schematic diagram in the text. To make the figure clear and easy to observe, the prediction results of six common emissivity models were listed as the experimental results. Then, it was input into the corresponding RNN subnetwork to inverse temperature precisely. Simulation results showed that in the range of 1500 to 3000 K temperature, the relative error of the test set was within 1.0% for the network trained after classification by GIM, whereas the relative error of the test set was within 1.2% for the network trained without GIM classification. After 5.0% random noise was added to the inputting data, the relative error still was controlled within 1.5%, which reflected the good antinoise performance of the algorithm. Multispectral measurement data of rocket engine plumes is processed by the proposed algorithm in this manuscript. The inversion results are consistent with the theoretical results. It is indicated that the proposed algorithm has good adaptability to different materials. It is expected to become a general data processing algorithm for MWP.

Generalized inverse matrix - long short-term memory neural network data processing algorithm for multi-wavelength pyrometry.

The data processing of multi-wavelength pyrometry (MWP) is faced with the problem of solving N equations and N+1 unknown underdetermined equations. The traditional iterative optimization methods are difficult to meet the actual measurement requirements in terms of accuracy and efficiency. With the development of artificial intelligence technology in the field of data processing, it is expected to solve this problem. A generalized inverse matrix (GIM) is combined with a long short-term memory (LSTM) neural network algorithm for data processing of MWP is proposed, which emissivity influence is dispensed completely. Firstly, GIM is used for classification of the emissivity. Furthermore, inputting to the LSTM network not only ensures the accuracy of temperature measurement but also greatly improves the efficiency. The simulation results demonstrated that the accuracy of the GIM-LSTM algorithm was superior to that of the GIM-EPF and BP methods. After random noise was added, the relative error was still less than that for the GIM-EPF and BP methods, and the algorithm exhibited excellent anti-noise performance. Publicly available temperature data for the exhaust plume of a rocket engine were processed by the GIM-LSTM method, and the average relative error was less than the traditional method. Especially, in terms of inversion speed, the operational time of the GIM-LSTM algorithm was at the millisecond level, which is of great significance for the real-time monitoring of rocket exhaust plumes. The proposed GIM-LSTM data processing algorithm affords high accuracy and speed and is suitable for practical measurement of high-emissivity objects in real-time via MWP.

RBF neural network modeling approach using PCA based LM–GA optimization for coke furnace system

Neural network prediction and data processing have been widely used in chemical industry, however, there exist many disturbance variables that will affect the system output, and traditional neural network prediction model has poor accuracy. In this paper, the dimensionality reduction of normalized input variables that affect the outputs is first implemented by using principal component analysis (PCA). Then, radial basis function (RBF) neural network model is established. Levenberg–Marquardt (LM) algorithm is used to initialize the weights of RBF neural network, which overcomes the influence of initial weights during the training process. Genetic algorithm (GA) is further introduced to train the centers, widths and weights to improve the modeling accuracy. Finally, the root mean square error (RMSE) is used to evaluate the prediction performances. Compared with two RBF neural network modeling methods, the proposed method can improve the prediction accuracy greatly.

Scientific and Technological Issues of Using Artificial Intelligence and Neural Network Data Processing Technologies in the Automated Systems of the Strategic Missile Forces

Scientific and Technological Issues of Using Artificial Intelligence and Neural Network Data Processing Technologies in the Automated Systems of the Strategic Missile Forces Military Thought No. 2, Vol. 30

Spiking Neural Membrane Computing Models

As third-generation neural network models, spiking neural P systems (SNP) have distributed parallel computing capabilities with good performance. In recent years, artificial neural networks have received widespread attention due to their powerful information processing capabilities, which is an effective combination of a class of biological neural networks and mathematical models. However, SNP systems have some shortcomings in numerical calculations. In order to improve the incompletion of current SNP systems in dealing with certain real data technology in this paper, we use neural network structure and data processing methods for reference. Combining them with membrane computing, spiking neural membrane computing models (SNMC models) are proposed. In SNMC models, the state of each neuron is a real number, and the neuron contains the input unit and the threshold unit. Additionally, there is a new style of rules for neurons with time delay. The way of consuming spikes is controlled by a nonlinear production function, and the produced spike is determined based on a comparison between the value calculated by the production function and the critical value. In addition, the Turing universality of the SNMC model as a number generator and acceptor is proved.

Neural network data processing for financial insolvency fore-casting of the water supply enterprises

This work is devoted to creation of neural network methods with the increased stability and reliability for bankruptcy forecasting. In this paper, we investigate the performance of four different neural network (NN) methods for bankruptcy prediction of the resource provisioning system enterprises. Two methods based on NN-ensembles of classifiers (boosting-ensembles and bagging-ensembles) and another two tested methods are stand-alone classifiers (multilayer perceptron (MLP) and network of radial basic functions). We defined that bagging-ensembles of neural networks have advantage in comparison with classical - stand-alone classifiers. Developed bagging-ensemble of neural networks is currently used to the sphere of financial monitoring for identification of signs of financial insolvency of the water supply enterprises and assessment of risk of social tension in regions.

Dual-emission CdTe/AgInS2 photoluminescence probe coupled to neural network data processing for the simultaneous determination of folic acid and iron (II)

This work focused on the combination of CdTe and AgInS2 quantum dots in a dual-emission nanoprobe for the simultaneous determination of folic acid and Fe(II) in pharmaceutical formulations. The surface chemistry of the used QDs was amended with suitable capping ligands to obtain appropriate reactivity in terms of selectivity and sensitivity towards the target analytes. The implementation of PL-based sensing schemes combining multiple QDs of different nature, excited at the same wavelength and emitting at different ones, allowed to obtain a specific analyte-response profile. The first-order fluorescence data obtained from the whole emission spectra of the CdTe/AgInS2 combined nanoprobe upon interaction with folic acid and Fe(II) were processed by using chemometric tools, namely partial least-squares (PLS) and artificial neural network (ANN). This enabled to circumvent the selectivity issues commonly associated with the use of QDs prone to indiscriminate interaction with multiple species, which impair reliable and accurate quantification in complex matrices samples.ANN demonstrated to be the most efficient chemometric model for the simultaneous determination of both analytes in binary mixtures and pharmaceutical formulations due to the non-linear relationship between analyte concentration and fluorescence data that it could handle. The R2P and SEP% obtained for both analytes quantification in pharmaceutical formulations through ANN modelling ranged from 0.92 to 0.99 and 5.7–9.1%, respectively. The obtained results revealed that the developed approach is able to quantify, with high reliability and accuracy, more than one analyte in complex mixtures and real samples with pharmaceutical interest.

Neural network for structural health monitoring with combined direct and indirect methods

Advancement in wireless communication as well as recording and transferring data over the internet provides a lot of possibilities for smart inspection and monitoring for machines and structures. The big data recorded and transferred through such a system must be analyzed efficiently on the go to provide accurate feedback to the system. Neural network (NN) data processing techniques are an effective methodology for fast and accurate analyses of the data and provide feedback to the system. An NN methodology is proposed for structural health monitoring of bridge structures. The proposed platform uses the direct and indirect sensors mounted on the bridge structure and on the passing vehicle, respectively. This proposed approach will decrease the cost and the potential damages to the sensors in direct methods, and will increase the accuracy and reliability of monitoring in indirect techniques. The methodology and data processing techniques have been validated using a lab-scaled test bed.

Automation of recognition of chemicals using electronic sensor technology based on neural network data processing

The composition of the initial substance was determined using an electronic sensor “electronic nose”, consisting of 8 sensors, to which air was supplied with a syringe with alcohol vapor containing various kinds of impurities. The signal from the sensors was recorded with a sampling frequency of 1 s for 120 s. The output of the device was presented in two different interpretations - in the form of curves obtained from each sensor, or the areas under the curves. The purpose of the work is to build a recognition system for 11 impurities and water in the starting material. The composition of the initial substance was determined using an “electronic nose”, which allows one to obtain 120 values from each of 8 sensors in the form of curves or the values of the areas under the curves. A large number of classes (12), the dynamic presentation of the source data information make it advisable to build a pattern recognition system based on a neural network - a multilayer perceptron trained on the basis of the error back propagation algorithm. When training the network, existing samples are used, indicating which class they belong to. The properties of each analyte are represented as a vector of 120 values of 8 attributes defining one of 12 classes. To reduce the dimensionality of the input data of the neural network, the authors proposed the use of convolution of the available information without significant loss of information capacity of signs by constructing 8 polynomial regressions of the 19th order that describe the curves from each of the 8 sensors of the “electronic nose”. The input matrix obtained as a result of convolution consisted of 20 polynomial regression coefficients of each of 8 curves for 12 classes under consideration. A two-layer neural network with 43 neurons and a sigmoidal activation function in the hidden layer and 12 neurons and a linear activation function in the output layer was constructed. As a result of network training, 2 classification errors were obtained, which allows us to use the approach proposed by the authors to build a recognition system based on preliminary convolution of data dynamically obtained from the “electronic nose”

Detection and on-line prediction of leak magnitude in a gas pipeline using an acoustic method and neural network data processing

Considering the importance of monitoring pipeline systems, this work presents the development of a technique to detect gas leakage in pipelines, based on an acoustic method, and on-line prediction of leak magnitude using artificial neural networks. On-line audible noises generated by leakage were obtained with a microphone installed in a 60 m long pipeline. The sound noises were decomposed into sounds of different frequencies: 1 kHz, 5 kHz and 9 kHz. The dynamics of these noises in time were used as input to the neural model in order to determine the occurrence and the leak magnitude. The results indicated the great potential of the technique and of the developed neural network models. For all on-line tests, the models showed 100% accuracy in leak detection, except for a small orifice (1 mm) under 4 kgf/cm² of nominal pressure. Similarly, the neural network models could adequately predict the magnitude of the leakages.

Residual Stress Assessment in Inconel 718 Machining Through Wavelet Sensor Signal Analysis and Sensor Fusion Pattern Recognition

On-line residual stress assessment in turning of Inconel 718 was carried out through multiple sensor monitoring based on cutting force, acoustic emission and vibration signals acquisition and analysis. The detected sensor signals were processed by the wavelet packet transform technique to extract statistical features from the packet coefficients for the construction of wavelet feature vectors. The latter were used for sensor fusion pattern recognition through neural network data processing grounded on X-ray diffraction residual stress measurements on the turned part surface. The scope of the sensory data fusion approach was to achieve a robust scheme for multi-sensor monitoring decision making on machined surface integrity in terms of residual stress level acceptability.

Based on Corrected Model of General Regression Neural Network for Forecasting Water Consumption of Populus (P.×euramericana cv.“74/76<i>”</i>) Seedlings

Water consumption of plants is a key parameter for formulating irrigation system, and the precise prediction play a important role in improving the use efficiency of limited water resources. In this experiment, by using the method of General Regression Neural Network (GRNN) and MATLAB DATA PROCESSING SYSTEM combined with the meteorological data of air temperature, relative air humidity, solar radiation, wind speed, soil water content and dew point temperature as the input variable, the author established the artificial neural network system to forecast the seedling water consumption of P.×euramericana cv.“74/76”, and made the amendment to the Neural Network Model, by comparison between original model and corrected model, we draw the conclusion that all parameters, except solar radiation, had better correlation between corrected values and measured values: relative air humidity (0.99>0.98), air temperature (0.62>0.61), dew point temperature (0.96>0.82), soil water content (0.56>0.54), wind speed (0.89>0.86), and the corrected model had higher forecasting accuracy. In addition, the maximum relative error of GRNN corrected model was 0.104, the minimum relative error was 0.010, the average relative error was 0.04. The corrected model is superior to the original model that the former performs a higher forecasting accuracy with relatively shorter time consumption and faster speed in training. Therefore, we propose that GRNN model and its corrected model can be used in prediction of seedling water consumption.

Instrumentation for the monitoring of toxic pollutants in water resources by means of neural network analysis of absorption and fluorescence spectra

The concentration of several pollutants, usually present in industrial waste waters, is predicted by the neural network data processing of absorption and fluorescence measurements in the visible spectral range. Proper network training provides quantitative analysis of many pollutants with sub-ppm resolution. Compact optical fibre instrumentation for absorption spectroscopy and an innovative flowcell for fluorescence measurements enable cost-effective, in situ, nonstop monitoring of waste waters.

Study on the Optimal Distance between the Sand Surface and the Sensors Array for the Measurement of Sand Moisture

A method of discriminating the moisture content of sand by neural-network data processing using the ultrasonic pulse-echo signals received by receivers in a parabolic transmitter-receiver array was proposed in our previous work. The stability of the signal received was not satisfactory, however, and the final discrimination ratio of sand moisture was low at some points. To achieve a reflected signal with higher signal to noise ratio, this study examined the optimal distance between a newly arranged transmitter-receiver array and the surface of the moist sand. The theoretical estimation and the experimental results of pressure of received sound versus distance show a similar tendency. The optimal distance is determined by analyzing the theoretical and experimental results. The comparison of the distribution of received data and the discrimination ratio verify that the stability of the discrimination in the new optimal arrangement is improved.

A disposable multielectrode biosensor for rapid simultaneous detection of the insecticides paraoxon and carbofuran at high resolution

A sensitive screen-printed amperometric multielectrode biosensor for the rapid discrimination of the insecticides paraoxon and carbofuran in mixtures was developed. It was based on the principle of acetylcholinesterase (AChE) inhibition and chemometric data analysis using artificial neural networks (ANNs). Four types of native or recombinant AChEs (electric eel, bovine erythrocytes, rat brain, Drosophila melanogaster) were immobilised by screen printing on four-electrode thickfilm sensors in sets containing each AChE. The multisensors were used for quantitative inhibition analysis of binary reagent mixtures combining the AChE-multisensor with feed-forward ANNs. The sensors registered a detection range for both analytes of 0.2–20 μg/l with an overall assay time of less than 60 min. The individual inhibition pattern of each AChE–analyte combination enabled the discrimination of both analytes by the use of neural network data processing. Thus, paraoxon and carbofuran in mixtures displaying a concentration range of 0–20 μg/l for each analyte could be analysed with prediction errors of 0.9 μg/l for paraoxon and 1.4 μg/l for carbofuran.

Analysis and Measurement of Reflectance of Moist Sand Surface Using a Four Transmitter Array

A one-transmitter eight-receiver (1T8R) array pattern was employed in our previous work to measure the moisture content of sand by neural-network data processing of ultrasonic pulse-echo signals. In order to enhance the power of the incident wave to improve the stability of measurement of sand moisture, this paper proposes a four-transmitter five-receiver (4T5R) array pattern. The results of theoretical estimation and experimental measurement show a similar tendency, that the reflectance of 4T5R is markedly larger than that of 1T8R. Thus it is verified that the new transmitter-receiver arrangement improves the measuring stability.

Experiments on Discrimination between the Same Materials of Different Grain Size

The authors devised a method to discriminate water bottom materials using the pulse echo method and multilayered neural network data processing. Using this method, experiments to investigate the discrimination ratio were carried out using crushed stones and sea sand in a water bath. In the case of crushed stones, the change in discrimination ratio due to the change in size of crushed stones was studied. For sea sand with small grain size relative to the wavelength, the change in the discrimination ratio relative to specific acoustic impedance was studied. As the result of the experiments, a discrimination ratio of 70–80% was achieved when the size of crushed stones was different by approximately one wavelength, and there was a difference of 12000 ( g/cm2·s) in impedance for sea sand.

JETNET 3.0—A versatile artificial neural network package

An F77 package for feed-forward artificial neural network data processing, JETNET 3.0, is presented. It represents a substantial extension and generalization of an earlier release, JETNET 2.0. The package, which consists of a set of subroutines, is focused on multilayer perceptron architectures. As compared to earlier versions it contains a variety of minimization options, measures for monitoring the learning process, limited precision emulation, etc. Also, the reader is provided with a set of guidelines for when to use the different options.