Single-output Neural Network Research Articles

Design, Assessment, and Modeling of Multi-Input Single-Output Neural Network Types for the Output Power Estimation in Wind Turbine Farms

Design, Assessment, and Modeling of Multi-Input Single-Output Neural Network Types for the Output Power Estimation in Wind Turbine Farms

An end-to-end neural network for UUV autonomous collision avoidance

This paper proposes a 3D autonomous collision avoidance method based on convolutional gated recurrent units to improve the autonomy of Unmanned Underwater Vehicle (UUV). The state equations of the UUV autonomous collision avoidance system are constructed by studying its mechanism and integrating dynamic/static obstacle recognition, dynamic obstacle motion prediction, collision risk assessment, and collision avoidance. Then a multi-input single-output neural network architecture that integrates static feature extraction, dynamic time sequence modeling, and feature integration is proposed based on Convolutional Neural Networks (CNNs) and Gated Recurrent Units (GRUs) to describe the state space. CNNs extract features from sonar observation data to improve the accuracy of obstacle recognition. GRUs are combined with CNNs to capture the correlation of long-distance features and extract dynamic features. The spatial and temporal invariance of the neural network architecture enhances the fault tolerance of the UUV collision avoidance system for inputs and adaptability to observation noise and environments. Finally, simulation results show that this method is adaptable to sonar observation noise and unknown environments to solve the problem of forward-looking sonar-based UUV collision avoidance in unknown complex ocean environments.

Deep learning models for assisted decision-making in performance optimization of thin film nanocomposite membranes

The performance of thin film nanocomposite nanofiltration membranes (TFNNMs) is frequently constrained by the trade-off between the permeability and rejection/selectivity of the membrane. In this study, to provide assisted decision-making in performance optimization of TFNNMs, we utilized a dataset comprising four categories of impact parameters, including membrane manufacturing conditions, membrane performance, nanoparticle material performance, and organic solvent nanofiltration conditions, and trained a dual output neural network (D-ANN) to simulate the relative permeability (RP) and relative rejection/selectivity (RR) of TFNNMs. The D-ANN model outperformed conventional machine learning models and single-output neural networks in predicting both RP and RR, indicating its potential in uncovering correlations between output variables. The Shapley additive explanation method was employed to elucidate the contribution and marginal effects of each influencing parameter on RP and RR, among them, NP size, NP loading, water contact angle, and chloride concentration were influential parameters that are important for both RP and RR. Based on the D-ANN model, we extrapolated two sets of unknown data and achieved good predictive performance (RP-error <15%, RS-error <7% for most sample points). Furthermore, we observed that the predicted performance of the data points rarely exceeded the actual performance, indicating a low probability of overestimation. Based on these premises, the influential parameters can be optimized using the D-ANN model, which ultimately led to exceptional performance for a total of 22 datasets in four pairs of organic solvent and solute purifications guided by the D-ANN model. This study demonstrates that the D-ANN model can provide decisions for the performance optimization of TFNNMs while highlighting the enhancement of nanofiltration performance by thin film nanocomposites, significantly facilitating the design of high-performance TFNNMs.

Signal Property Information-Based Target Detection with Dual-Output Neural Network in Complex Environments

The performance of traditional model-based constant false-alarm ratio (CFAR) detection algorithms can suffer in complex environments, particularly in scenarios involving multiple targets (MT) and clutter edges (CE) due to an imprecise estimation of background noise power level. Furthermore, the fixed threshold mechanism that is commonly used in the single-input single-output neural network can result in performance degradation due to changes in the scene. To overcome these challenges and limitations, this paper proposes a novel approach, a single-input dual-output network detector (SIDOND) using data-driven deep neural networks (DNN). One output is used for signal property information (SPI)-based estimation of the detection sufficient statistic, while the other is utilized to establish a dynamic-intelligent threshold mechanism based on the threshold impact factor (TIF), where the TIF is a simplified description of the target and background environment information. Experimental results demonstrate that SIDOND is more robust and performs better than model-based and single-output network detectors. Moreover, the visual explanation technique is employed to explain the working of SIDOND.

Multi-output physics-informed neural network for one- and two-dimensional nonlinear time distributed-order models

&lt;abstract&gt;&lt;p&gt;In this article, a physics-informed neural network based on the time difference method is developed to solve one-dimensional (1D) and two-dimensional (2D) nonlinear time distributed-order models. The FBN-$ \theta $, which is constructed by combining the fractional second order backward difference formula (BDF2) with the fractional Newton-Gregory formula, where a second-order composite numerical integral formula is used to approximate the distributed-order derivative, and the time direction at time $ t_{n+\frac{1}{2}} $ is approximated by making use of the Crank-Nicolson scheme. Selecting the hyperbolic tangent function as the activation function, we construct a multi-output neural network to obtain the numerical solution, which is constrained by the time discrete formula and boundary conditions. Automatic differentiation technology is developed to calculate the spatial partial derivatives. Numerical results are provided to confirm the effectiveness and feasibility of the proposed method and illustrate that compared with the single output neural network, using the multi-output neural network can effectively improve the accuracy of the predicted solution and save a lot of computing time.&lt;/p&gt;&lt;/abstract&gt;

The use of dual machine learning in industrial electrical tomography

Machine learning techniques are playing a key role in tomography. Process tomography, also known as industrial tomography, uses a variety of physical phenomena. Contrary to the commonly used computed tomography in medicine, electrical, ultrasound, radio and even optical tomography are used in industry. In electrical tomography we distinguish between impedance and capacitance tomography. This manuscript presents an algorithmic method to allow accurate measurements of reactors and industrial vessels using electrical impedance tomography. Reactors may contain liquids which undergo phase changes resulting in crystallization or gassing. The tomograph can detect gas crystals or bubbles. The innovative contribution of the authors is the development of an original algorithm that allows the conversion of input measurements to 2D images. First, the algorithm trains multiple single-output neural networks, each of which generates a single image pixel. Secondly, two models were used (support vector machines and artificial neural networks), which were assigned to individual pixels of the image. The image was reconstructed using two methods, not one, so the new method was called dual machine learning (DML). In order to assess the effectiveness of the new approach, both homogeneous methods (SVM and ANN) were compared with the new DML method. The results confirmed the higher effectiveness of the new approach.

Low-Complexity Multi-Task Learning Aided Neural Networks for Equalization in Short-Reach Optical Interconnects

With the rapid development of machine learning technologies in recent years, different types of neural network (NN)-based equalizers have been proposed and proved to be efficient digital signal processing tools to deal with the nonlinear impairments in short-reach direct detection optical interconnects. However, one major concern for these NN-based equalizers is their computational complexity (CC), since only a few tens of multiplications per symbol can be practically handled considering real-time implementation. In this paper, we propose an NN-based multi-symbol equalization scheme inspired by multi-task learning. Compared with traditional single-output NN-based equalizers, the CC can be significantly reduced with the help of the proposed scheme. A 50-Gb/s 25-km pulse amplitude modulation (PAM)-4 direct detection optical link is experimentally carried out and 3 different types of NNs, i.e., feedforward NN (FNN), cascade FNN (C-FNN), and recurrent NN (RNN), are employed for the proposed multi-symbol equalization scheme. Experimental results show that a maximum CC reduction of 43.2%/41.1%/44.0% can be achieved with FNN/C-FNN/RNN-based multi-symbol equalization to achieve their best performance, compared with the corresponding single-output NNs. The best performance of the proposed and the traditional FNN/C-FNN-based schemes are the same, while the RNN-based scheme sacrifices a slight system performance. Aided by multi-symbol equalization, the CC needed to recover 1 symbol for various NNs can all be reduced to about a few tens of multiplications, indicating the feasibility of real-time implementation of these NN-based equalizers.

Application of neural networks in predictions of brake wear particulate matter emission

According to the World Health Organization, air pollution with PM10 and PM2.5 (PM-particulate matter) is a significant problem that can have serious consequences for human health. Vehicles, as one of the main sources of PM10 and PM2.5 emissions, pollute the air and the environment both by creating particles by burning fuel in the engine, and by wearing of various elements in some vehicle systems. In this paper, the authors conducted the prediction of the formation of PM10 and PM2.5 particles generated by the wear of the braking system using a neural network (Artificial Neural Networks (ANN)). In this case, the neural network model was created based on the generated particles that were measured experimentally, while the validity of the created neural network was checked by means of a comparative analysis of the experimentally measured amount of particles and the prediction results. The experimental results were obtained by testing on an inertial braking dynamometer, where braking was performed in several modes, that is under different braking parameters (simulated vehicle speed, brake system pressure, temperature, braking time, braking torque). During braking, the concentration of PM10 and PM2.5 particles was measured simultaneously. The total of 196 measurements were performed and these data were used for training, validation, and verification of the neural network. When it comes to simulation, a comparison of two types of neural networks was performed with one output and with two outputs. For each type, network training was conducted using three different algorithms of backpropagation methods. For each neural network, a comparison of the obtained experimental and simulation results was performed. More accurate prediction results were obtained by the single-output neural network for both particulate sizes, while the smallest error was found in the case of a trained neural network using the Levenberg-Marquardt backward propagation algorithm. The aim of creating such a prediction model is to prove that by using neural networks it is possible to predict the emission of particles generated by brake wear, which can be further used for modern traffic systems such as traffic control. In addition, this wear algorithm could be applied on other vehicle systems, such as a clutch or tires.

Application of regression neural network and MIV algorithm in visual communication design

Regression neural network algorithm can continuously learn the characteristics of training samples, and it has good data tolerance, robustness and generalization ability, and has been widely used in all kinds of nonlinear regression problems. In this paper, a multi-input and single output neural network regression model is constructed. Through the weight training of a large number of experimental sample data characteristics, the nonlinear regression mapping relationship of multi parameter input is established, so as to realize the accurate prediction of the basic situation of cultural and creative enterprises, then to analyze the influence of cultural and creative enterprises on the visual communication design course. The result of the correlation coefficient of the two is 0.79. The average relative errors between the predicted results and the actual results are 9.167% and 9.63%, respectively, both of which are less than 12%. It shows that the regression neural network model constructed in this paper can accurately analyse and predict the data, and the error between the predicted value and the target output value of the network model meets the actual requirements. So it has good prediction accuracy and generalization performance.

Special acoustical role of pinna simplifying spatial target localization by the brown long-eared bat Plecotus auritus.

Echolocating bats locate a target by sonar. The performance of this system is related to the shape of the binaural conformation in bats. From numerical predictions, it was found that in a central frequency band, the orientation of a strong sidelobe is shifted nearly linearly in the vertical direction. Inspired by this, the authors built an accurate wide-scope elevation estimation system by constructing a pair of erect artificial pinnae and realized simultaneous elevation and azimuth estimation by constructing a pair of orthogonal pinnae. By demonstrating the simplicity of spatial target echolocation, the authors showed that only two independent single-output neural networks were needed for either elevation or azimuth estimation. This method could be applied to imitate any other mammal species with similar pinna directivity patterns to facilitate and improve an artificial echolocation system.

A neural network ensemble approach with jittered basin characteristics for regionalized low flow frequency analysis

Previous studies have shown that the predictive ability of artificial neural networks can be improved by combining multiple models. This study presents a jittering-based neural network ensemble approach to obtain improved regional low flow estimates at ungauged sites. The fundamental idea of the jittered ensemble is to add noises to the explanatory variables and thereby augments the training data sets to develop the network models based on different but associated training data. To evaluate the performance of the jittered ensemble, this study employs two different neural network architectures, a single-output neural network model and a multi-output neural network model. In addition, a jittered ensemble approach is coupled with the variable importance measuring algorithm to infer the relationship between basin characteristics and predicted low flow quantiles. Effectiveness of the proposed methods is demonstrated using selected basins in the northeastern United States. Results suggest that the jittering-based ensemble model is able to consistently outperform a single modeling approach. Also, improvement is achieved even using small sizes of ensembles although a sufficient sample size offers more reliable predictions. Finally, this study recognizes that the effects of the basin characteristics varied or remained constant based on the low flow quantiles considered.

Predicting the ingredients of self compacting concrete using artificial neural network

Self compacting concrete (SCC) is a highly flowable type of concrete that spreads into form without the need of mechanical vibration. This paper presents a comparative study between two methodologies which have been applied on two different data sets of SCC mixtures, which were gathered from the literature, using artificial neural network (ANN). The two methodologies aim to get the best prediction accuracy for the SCC ingredients using the 28-day compressive strength and slump flow diameters as inputs of the ANN. In the first methodology, the ANN model is constructed as a multi input – multi output neural network with the six ingredients as outputs. In the second methodology, the ANN model is constructed as a multi input – single output neural network where the six ingredient outputs are predicted separately from six different neural networks of multi input – single output type. Also, the influence of the mixes homogeneity on the prediction accuracy is investigated through the second data set. The results demonstrate the superiority of the second methodology in terms of accuracy of the predicted outputs. Moreover, the uniformity of the training data assures the accuracy of the predicted ingredients.

Network Modeling of Fin-and-Tube Evaporator Performance Under Dry and Wet Conditions

A new neural network modeling approach to the evaporator performance under dry and wet conditions has been developed. Not only the total cooling capacity but also the sensible heat ratio and pressure drops on both air and refrigerant sides are modeled. Since the evaporator performance under dry and wet conditions is, respectively, dominated by the dry-bulb temperature and the web-bulb temperature, two neural networks are used together for capturing the characteristics. Training of a multi-input multi-output neural network is separated into training of multi-input single-output neural networks for improving the modeling flexibility and training efficiency. Compared with a well-developed physics-based model, the standard deviations of trained neural networks under dry and wet conditions are less than 1% and 2%, respectively. Compared with the experimental data, errors fall into ±5%.

Inductive transfer with context-sensitive neural networks

Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, csMTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the csMTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the csMTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from csMTL encoded data provides initial evidence that this improvement is not shared across all machine learning models.

Classification and identification of mosquito species using artificial neural networks

An artificial neural network method is presented for classification and identification of Anopheles mosquito species based on the internal transcribed spacer2 (ITS2) data of ribosomal DNA string. The method is implemented in two different multi-layered feed-forward neural network model forms, namely, multi-input single-output neural network (MISONN) and multi-input multi-output neural network (MIMONN). A number of data sequences in varying sizes of different Anopheline malarial vectors and their corresponding species coding are employed to develop the neural network models. The classification efficiency of the network models for untrained data sequences is evaluated in terms of quantitative performance criteria. The results demonstrate the efficiency of the neural network models to extract the genetic information in ITS2 sequences and to adapt to new data. The method of MISONN is found to exhibit superior performance over MIMONN in distinguishing and identification of the mosquito vectors.

Multi-rate model predictive control of a chemical reactor based on three neural models

This paper describes a multi-rate model predictive control for a chemical reactor rig. Three multi-input single-output neural network models are developed with different sampling times; each for one reactor output. A multi-rate control strategy is proposed based on the multi-rate models, which enables reduced dimension optimisations to be used in majority of searching for optimal controls. In addition, some practical control techniques are also applied. Effectiveness of the proposed method is evaluated by both simulations and on-line control. A comparison with three decentralised PID controllers is presented.

Uncertainty Estimation Using Fuzzy Measures for Multiclass Classification

Uncertainty arises in classification problems when the input pattern is not perfect or measurement error is unavoidable. In many applications, it would be beneficial to obtain an estimate of the uncertainty associated with a new observation and its membership within a particular class. Although statistical classification techniques base decision boundaries according to the probability distributions of the patterns belonging to each class, they are poor at supplying uncertainty information for new observations. Previous research has documented a multiarchitecture, monotonic function neural network model for the representation of uncertainty associated with a new observation for two-class classification. This paper proposes a modification to the monotonic function model to estimate the uncertainty associated with a new observation for multiclass classification. The model, therefore, overcomes a limitation of traditional classifiers that base decisions on sharp classification boundaries. As such, it is believed that this method will have advantages for applications such as biometric recognition in which the estimation of classification uncertainty is an important issue. This approach is based on the transformation of the input pattern vector relative to each classification class. Separate, monotonic, single-output neural networks are then used to represent the "degree-of-similarity" between each input pattern vector and each class. An algorithm for the implementation of this approach is proposed and tested with publicly available face-recognition data sets. The results indicate that the suggested approach provides similar classification performance to conventional principle component analysis (PCA) and linear discriminant analysis (LDA) techniques for multiclass pattern recognition problems as well as providing uncertainty information caused by misclassification.

Modeling Selected Properties of Extruded Waxy Maize Cross‐Linked Starches with Neural Networks

ABSTRACT: Extrusion process is described as a complex multiple input, multiple output system. Various cross‐linked waxy maize starches were extruded at initial moisture contents of 20, 24, and 28% (dry basis) and screw speeds of 110, 150, and 190 rpm. Multiple Input, Single Output neural network models were developed for predicting individual product properties, and Multiple Input, Multiple Output models were developed for predicting selected product properties at 1 time from selected input process parameters. Models varied only in the number of hidden neurons, ranging between 8 and 14. Models had R2 values of 0.89 and greater. Models explained the process and the influence of selected process parameters on selected product properties.

Neural network-based predictive control for multivariable processes

In this paper, the systematic derivations of setting up a nonlinear model predictive control based on the neural network are presented. This extends our previous work (Chen, 1998) into a multivariable system to explore the characteristics of the design. There are two stages for the development of nonlinear neural network predictive control: a neural network model and a control design. In the neural network model design, a parallel multiple-input, single-output neural network autoregressive with a model of exogenous inputs (NNARX) is proposed for multistep ahead predictions. In control design, the controller with extended control horizon is developed. The Levenberg-Marquardt algorithm is applied to training the NNARX model. Also, the sequential quadratic programming is used to search for the optimal manipulated inputs. The gradient of the objective function and constraints that require computation of Jacobian matrices are completely derived for optimization calculation. To demonstrate the control ability of MIMO cases, the proposed method is applied through two nonlinear simulation problems.

Quick fuzzy backpropagation algorithm

A modification of the fuzzy backpropagation (FBP) algorithm called QuickFBP algorithm is proposed, where the computation of the net function is significantly quicker. It is proved that the FBP algorithm is of exponential time complexity, while the QuickFBP algorithm is of polynomial time complexity. Convergence conditions of the QuickFBP, resp. the FBP algorithm are defined and proved for: (1) single output neural networks in case of training patterns with different targets; and (2) multiple output neural networks in case of training patterns with equivalued target vector. They support the automation of the weights training process (quasi-unsupervised learning) establishing the target value(s) depending on the network's input values. In these cases the simulation results confirm the convergence of both algorithms. An example with a large-sized neural network illustrates the significantly greater training speed of the QuickFBP rather than the FBP algorithm. The adaptation of an interactive web system to users on the basis of the QuickFBP algorithm is presented. Since the QuickFBP algorithm ensures quasi-unsupervised learning, this implies its broad applicability in areas of adaptive and adaptable interactive systems, data mining, etc. applications.