Backpropagation Learning Algorithm Research Articles

Estimation of bubble point pressure and solution gas oil ratio using artificial neural network

Bubble point pressure and gas solubility are considered vital pressure-volume-temperature properties of crude oil samples as both play a significant role in reservoir and production engineering calculations. These two fluid oil properties can be measured experimentally or using mathematical correlations, the experimental methods are time consuming and expensive, although they give a reliable result. Therefore, using mathematical correlations can be as an alternative way, however, existing correlation having number of limitations. Hence, this study presents a novel numerical model based on artificial neural network (ANN) model and universal mathematical correlation to forecast bubble point pressure and gas solubility. The artificial neural network model based on back-propagation learning algorithm and sigmoid function with numerous inputs include API gravity, reservoir temperature, and gas specific gravity. The ANN model is developed and validated by utilizing more than 450 data points collected from numerous fields located worldwide with various reservoir characterization. In this study, the bubble point pressure is estimated first and then used to forecast gas solubility based on the calculated correlation coefficient. The results of this study concluded that in order to optimize the ANN model, collected data must to be distributed as follows; 65% for training, 20% for testing and 15% for the validation processes with R2 of 0.988 and average absolute percent relative error (AAPRE) =5.56% for the bubble point pressure and 4.5% for the gas solubility. A comparison study is performed between the new ANN extracted correlation and other well-known models (Standing, Petrosky and Frashad, McCain, Vasquez and Beggs, Marhoun, Glaso, Dokla, and Lasater correlations) to show the robustness of the presented correlation. The results show that the presented correlation has the ability to predict bubble point pressure and gas solubility precisely with lowest AAPRE compared to other models.

Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks.

Spiking neural networks (SNNs) use spatiotemporal spike patterns to represent and transmit information, which are not only biologically realistic but also suitable for ultralow-power event-driven neuromorphic implementation. Just like other deep learning techniques, deep SNNs (DeepSNNs) benefit from the deep architecture. However, the training of DeepSNNs is not straightforward because the well-studied error backpropagation (BP) algorithm is not directly applicable. In this article, we first establish an understanding as to why error BP does not work well in DeepSNNs. We then propose a simple yet efficient rectified linear postsynaptic potential function (ReL-PSP) for spiking neurons and a spike-timing-dependent BP (STDBP) learning algorithm for DeepSNNs where the timing of individual spikes is used to convey information (temporal coding), and learning (BP) is performed based on spike timing in an event-driven manner. We show that DeepSNNs trained with the proposed single spike time-based learning algorithm can achieve the state-of-the-art classification accuracy. Furthermore, by utilizing the trained model parameters obtained from the proposed STDBP learning algorithm, we demonstrate ultralow-power inference operations on a recently proposed neuromorphic inference accelerator. The experimental results also show that the neuromorphic hardware consumes 0.751 mW of the total power consumption and achieves a low latency of 47.71 ms to classify an image from the Modified National Institute of Standards and Technology (MNIST) dataset. Overall, this work investigates the contribution of spike timing dynamics for information encoding, synaptic plasticity, and decision-making, providing a new perspective to the design of future DeepSNNs and neuromorphic hardware.

A Secure Method for Authenticity Verification of Handwritten Signatures Through Digital Image Processing and Artificial Neural Networks

This paper aims to propose a method to do authenticity verification of handwritten signatures based on the use of digital image processing and artificial neural networks techniques through the backpropagation learning algorithm with 500 and 901 approaches, in order to optimize this verification process and act as a decision support tool, in an automated way. The results showed an average percentage error of 20% in the first and of 5.83% in the second, while the performance of a trained professional for that has an average error of 6.67%. Thus, we could observe the efficiency of the proposed method, as well as the difference and evolution of approaches through the relevance of the results.

On the antiderivatives of xp/(1 \u2212 x) with an application to optimize loss functions for classification with neural networks

Supervised learning in neural nets means optimizing synaptic weights W such that outputs y(x;W) for inputs x match as closely as possible the corresponding targets t from the training data set. This optimization means minimizing a loss function {mathscr{L}}(mathbf {W}) that usually motivates from maximum-likelihood principles, silently making some prior assumptions on the distribution of output errors y −t. While classical crossentropy loss assumes triangular error distributions, it has recently been shown that generalized power error loss functions can be adapted to more realistic error distributions by fitting the exponent q of a power function used for initializing the backpropagation learning algorithm. This approach can significantly improve performance, but computing the loss function requires the antiderivative of the function f(y) := yq− 1/(1 − y) that has previously been determined only for natural qin mathbb {N}. In this work I extend this approach for rational q = n/2m where the denominator is a power of 2. I give closed-form expressions for the antiderivative {int limits } f(y) dy and the corresponding loss function. The benefits of such an approach are demonstrated by experiments showing that optimal exponents q are often non-natural, and that error exponents q best fitting output error distributions vary continuously during learning, typically decreasing from large q > 1 to small q < 1 during convergence of learning. These results suggest new adaptive learning methods where loss functions could be continuously adapted to output error distributions during learning.

Satellite Multi-Sensor Data Fusion for Soil Clay Mapping Based on the Spectral Index and Spectral Bands Approaches

Integrating satellite data at different resolutions (i.e., spatial, spectral, and temporal) can be a helpful technique for acquiring soil information from a synoptic point of view. This study aimed to evaluate the advantage of using satellite mono- and multi-sensor image fusion based on either spectral indices or entire spectra to predict the topsoil clay content. To this end, multispectral satellite images acquired by various sensors (i.e., Landsat-5 Thematic Mapper (TM), Landsat-8 Operational Land Imager (OLI), Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), and Sentinel2-MultiSpectral Instrument (S2-MSI)) have been used to assess their potential in identifying bare soil pixels over an area in northeastern Tunisia, the Lebna and Chiba catchments. A spectral index image and a spectral bands image are generated for each satellite sensor (i.e., TM, OLI, ASTER, and S2-MSI). Then, two multi-sensor satellite image fusions are generated, one from the spectral index images and the other from spectral bands. The resulting spectral index and spectral band images based on mono-and multi-sensor satellites are compared through their spectral patterns and ability to predict the topsoil clay content using the Multilayer Perceptron with backpropagation learning algorithm (MLP-BP) method. The results suggest that for clay content prediction: (i) the spectral bands’ images outperformed the spectral index images regardless of the used satellite sensor; (ii) the fused images derived from the spectral index or bands provided the best performances, with a 10% increase in the prediction accuracy; and (iii) the bare soil images obtained by the fusion of many multispectral sensor satellite images can be more beneficial than using mono-sensor images. Soil maps elaborated via satellite multi-sensor data fusion might become a valuable tool for soil survey, land planning, management, and precision agriculture.

Meningkatkan Kinerja Backpropagation Neural Network Menggunakan Algoritma Adaptif

The application of Artificial Neural Networks in various fields of human life is getting wider, especially in the industrial sector. One of the artificial neural network structures that are quite often used is the Feedforward Neural Network with its well-known learning algorithm, namely Backpropagation. However, as reported by several researchers, Backpropagation has several weaknesses such as it takes a long time to converge in the training process, it is quite sensitive to initial weight conditions and is relatively often trapped in a local minima which can thwart the training process. In this study, the Adaptive algorithm is proposed as an alternative to the Backpropagation learning algorithm. The proposed algorithm provides hope in overcoming the weaknesses faced by Bakpropagation. As reported in the test results, compared to Backpropagation, the Adaptive algorithm is much stronger in dealing with variations in the initial weight conditions. From 100 tests in this study for each Backpropagation and Adaptive algorithm, with random variations for the initial weight value, the success rate of the Adaptive algorithm training process reaches 100% compared to Backpropagation which is at the level of 77%. In terms of speed, the Adaptive algorithm has successfully carried out the training process with an average number of iterations of 37 times compared to Backpropagation which requires an average of 162 iterations.

AI@nti-Malware: An intelligent framework for defending against malware attacks

Distinguishing among types of malware is important to understanding how they infect computing systems, the level of threat that they pose, and means of protecting against them. This study develops an intelligent framework, AI@nti-Malware, that combines artificial intelligence learning, data imbalance, and feature evaluation mechanisms to establish a malware classification model that is effective for defending against malware attacks. The SMOTEENN algorithm is used to generate training data for a minority of categories to solve the problem of model offset and to improve the effectiveness of the model. The results of an analysis using the CTU-13 open dataset show that the intelligent framework with the machine learning algorithm XGBoost can reach an accuracy of 99.98%, while that with the deep learning backpropagation algorithm has an accuracy of 98.88%.

Computational-Based Approaches for Predicting Biochemical Oxygen Demand (BOD) Removal in Adsorption Process

Predicting the adsorption performance to remove organic pollutants from wastewater is an essential environmental-related topic, requiring knowledge of various statistical tools and artificial intelligence techniques. Hence, this study is the first to develop a quadratic regression model and artificial neural network (ANN) for predicting biochemical oxygen demand (BOD) removal under different adsorption conditions. Nanozero-valent iron encapsulated into cellulose acetate (CA/nZVI) was synthesized, characterized by XRD, SEM, and EDS, and used as an efficient adsorbent for BOD reduction. Results indicated that the medium pH and adsorption time should be adjusted around 7 and 30 min, respectively, to maintain the highest BOD removal efficiency of 96.4% at initial BOD [Formula: see text] mg/L, mixing [Formula: see text] rpm, and adsorbent dosage of 3 g/L. An optimized ANN structure of 5–10–1, with the “ trainlm” back-propagation learning algorithm, achieved the highest predictive performance for BOD removal ([Formula: see text]: 0.972, Adj-[Formula: see text]: 0.971, RMSE: 1.449, and SSE: 56.680). Based on the ANN sensitivity analysis, the relative importance of the adsorption factors could be arranged as [Formula: see text]. A quadratic regression model was developed to visualize the impacts of adsorption factors on the BOD removal efficiency, optimizing pH at 7.3 and time at 46.2 min. The accuracy of the quadratic regression and ANN models in predicting BOD removal was approximately comparable. Hence, these computational-based methods could further maximize the performance of CA/nZVI material for removing BOD from wastewater under different adsorption conditions. The applicability of these modeling techniques would guide the stakeholders and industrial sector to overcome the nonlinearity and complexity issues related to the adsorption process.

Neural Network Parameter Identification Based Constitutive Modeling of Superelastic Shape Memory Alloys

AbstractSuperelastic shape memory alloys (SMAs) are two‐phase polycrystal materials with hysteretic energy dissipation and deformation recovery capabilities. This feature opens numerous application possibilities, particularly using SMA wires in structural vibration control. However, vibrations, such as wind and earthquake induced, may occur in high rate regimes with alternating amplitudes. Experiments show that the thermodynamically coupled SMA behavior is highly sensitive to strain rate and amplitude. Accordingly, the prediction accuracy of the constitutive models depends notably on their thermodynamic parameters, such as latent heat, heat transfer coefficient and specific heat. The identification of these parameters requires extensive experiments. To circumvent this challenge, this study proposes for SMA wires a machine learning based parameter identification procedure using artificial neural networks (ANNs). In this approach, a recently improved uniaxial macroscopic constitutive SMA model is utilized as a forward physics based model to compute all possible responses in an expected parameter space. With the generated data, a multilayer ANN is trained as a data based inverse model. The Latin hypercube sampling and the backpropagation learning algorithms are applied. After training, the data based model is able to identify suitable thermodynamic parameters from SMA stress responses. With the identified parameters, the constitutive model replicates the strain rate and amplitude dependent SMA response effects. The experimental results are matched by the model with high accuracy.

Gas path parameter prediction of aero-engine based on an autoregressive discrete convolution sum process neural network

In order to improve the approximation ability of neural network to functional and improve the prediction accuracy of time series data, this paper proposes an autoregressive discrete convolution sum process neural network prediction model and applies it to the prediction of aero-engine gas path performance parameters. Coupling threshold wavelet de-noising method is applied to the preprocessing of engine gas path parameters, which can effectively remove the noise in the time series data. Process neurons are applied to artificial neural networks to expand the computational functions of artificial neurons. The process neuron can not only realize the spatial aggregation operation of discrete input data and the connection weight, but also realize the time aggregation operation of the product integral of the continuous input function and the connection weight function. The output of discrete time series is affected by the current input/output, and also by the historical input/output value. Therefore, an autoregressive feedback link is added to the network topology, and the value of the network's output node is used as the input to adjust the network connection weight. The convolution sum operation of discrete time series data can realize the time accumulation effect, so the discrete convolution sum operator is used to replace the integral operator to realize the time aggregation function of the process neuron. Since the integral operation of the continuous function is avoided, the weight training process of the process neural network is effectively simplified, and the accuracy loss in the continuous-time function fitting process of discrete input samples is effectively avoided. Bayesian regularization network weight learning algorithm is used to solve the problems of slow convergence speed and easy to fall into local optimum of back propagation learning algorithm, and improve the generalization ability of neural network. It can be seen from the prediction simulation results of the two engines that the average relative error and root mean square error of the engine gas path parameter prediction based on the autoregressive discrete convolution sum process neural network model can be reduced to 0.67% and 0.35 respectively. The stable and high-precision prediction results show that the network model and weight learning algorithm proposed in this paper have better robustness in functional approximation, and have higher accuracy in time series data prediction.

Artificial Neural Network Modeling and Numerical Simulation of Syngas Fuel and Injection Timing Effects on the Performance and Emissions of a Heavy-Duty Compression Ignition Engine.

The use of syngas as an alternative fuel in compression ignition engines is a potential way to curb the emission of pollutants and optimize the performance of these engines. The present research studied the effect of changing fuel injection timing and adding syngas on the output of a heavy-duty diesel engine. The optimal mode of the turbulence model, fuel spray model, combustion model, and pollutant emission model was used to solve computational fluid dynamics. Changing fuel injection timing from 70 to 10° before top dead center (BTDC) and diesel variants, including conventional diesel, diesel + 20% syngas, and diesel + 40% syngas, is the strategy studied here. It was found that the use of syngas could reduce emissions significantly. The in-cylinder mean effective pressure was the highest for diesel + 20% syngas. On the other hand, increasing the rate of syngas and retarding injection timing reduced the ignition period and in-cylinder temperature. The lowest rate of CO emission was obtained from diesel + 40% syngas at a fuel injection timing of 70° BTDC, whereas the lowest particulate matter emission was related to diesel + 40% syngas at 40° BTDC injection timing. The lowest rate of CO2 emission was related to diesel + 40% syngas at a fuel injection timing of 10° BTDC, while the same timing for conventional diesel exhibited the lowest NOX emission rate. Finally, the performance parameters of the engine including indicated power, indicated fuel consumption, and indicated thermal efficiency were decreased with the increase in syngas fraction, so that their highest values were obtained from the conventional diesel at the injection timing of 40° BTDC. In addition, an artificial neural network (ANN) model based on a standard back-propagation learning algorithm was developed for modeling the performance and emissions of the engine. The results for the optimum ANN model showed that the optimal ANN has two hidden layers with 20–25 neurons and the transfer function of logsig–logsig for hidden layers 1 and 2, respectively, and can predict different parameters of the engine for different modes. The correlation coefficients (R-value) of optimal topology for training, validation, and testing are 0.99992, 0.96612, and 0.93424, respectively. The results for the optimum ANN model showed that the constructed model sufficiently predicts the performance and emissions of the CI diesel engine.

Adaptive VMD based optimized deep learning mixed kernel ELM autoencoder for single and multistep wind power forecasting

In this paper, an efficient new hybrid time series forecasting model combining variational mode decomposition (VMD) and Deep learning mixed Kernel ELM (MKELM) Autoencoder (AE) has been presented for precise prediction of wind power. It is well known that KELM-AE avoids manual tuning of weights of hidden layer nodes, and exhibits superb model generalization property, reduces execution time, and produces exact solution of output weights by generalized least squares resulting in compact storage space, etc. This is in contrast to other deep learning neural network models with nonconvex optimization problems, time consuming training process and limitations of backpropagation learning algorithms. Further the VMD parameters (α, K) are optimized to yield suitable number of intrinsic modes (IMFs) using a newly presented meta-heuristic population based Sine Cosine integrated water cycle algorithm (SCWCA) algorithm. Further the mixed kernel parameters and their associated weights are optimized using the same SCWCA to improve the performance of the Deep MKELM predictor against any outliers or noise in the data. Wind power data of three wind farms located in Sotavento Spain, Wyoming, and California, USA are applied for short-term and multi-step prediction of wind power with significant accuracy.

L and D enantiomer binary mixture determination simultaneously by spectrophotometric method without separation step based on artificial neural network and least squares support vector machine in valsartan pharmaceutical production

In this study, spectrophotometry method as a quick, simple, and accurate approach was proposed for the simultaneous determination of Valsartan (VAL) and its enantiomeric impurity (IMP-A) in binary mixtures using feed-forward artificial neural network (FFANN) with backpropagation learning algorithm and least squares support vector machine (LS-SVM) techniques without using excess solvent and the time-consuming extraction phase. Two different algorithms, including Levenberg–Marquardt (LM) and scaled conjugate gradient (SCG) as training algorithms was used in ANN method. LM algorithm with 2 layers, 9 neurons and 2 layers, 7 neurons in the FFNN with the mean square error (MSE) of 2.84 × 10−11 and 5.08 × 10−12 had the best response for predicting VAL and IMP-A concentrations, respectively. By optimizing regularization parameter (γ) and width of the function (σ) in LS-SVM model, mean recovery percentage and root mean square error (RMSE) were obtained 98.61, 98.95 and 3.22 × 10−4, 2.89 × 10−4 for VAL and IMP-A, respectively. The achieved results from the commercial tablet (Adovan) analysis using the proposed methods were compared to the yielded by HPLC technique via one-way analysis of variance (ANOVA) test. According to the results, they revealed good agreement and there was no significant differences.

A comparative and comprehensive study of prediction of Parkinson’s disease

Objectives: Parkinson's Disease (PD) is a form of neurodegenerative disease that is caused the progressive weakening of dopaminergic nerve cells that affects a large number of people around the world. The event of recent treatment methods principally depends upon the experimental data resulting from assessment balances and patients’ journals that take varied boundaries with reference to legitimacy, inter-rater inconsistency, and incessant monitoring. Methods: Nowadays various techniques and algorithms are utilized in predicting the accuracy in PD. A range of those techniques, including SVM, Artificial Neural Network, Naive Bayes, Kernel based extreme learning through subtractive clustering landscapes, Random Forest, The Multi-Layer Perceptron with Back-Propagation Learning Algorithm are widely applied to form the acceptable decision accurately. During this work, and in-depth review was administered on various techniques proposed by numerous researchers. a replacement system must be proposed which uses DL techniques and considers other attributes of paralysis agitans which can improve the prediction and be an advancement within the medical field. Result: It has been observed that many researches have been done in identifying the PD yet there is a need of suitable method or algorithm to improve the prediction of PD which will help in the clinical management. Conclusion and Future work: Most of the methods have used speech as a major attribute for their research and have produced substantial accuracy. In order to increase the precision approaches involving movements, facial expression and other attributes also be considered for evaluation

State of health estimation for lithium-ion battery based on the coupling-loop nonlinear autoregressive with exogenous inputs neural network

In order to solve the problem of low accuracy of traditional artificial neural networks in approximating functional, a coupling-loop nonlinear autoregressive with exogenous inputs neural network estimation model is proposed and applied to the state of health forecasting of lithium-ion batteries. Firstly, eight health indicators related to the state of health of lithium-ion batteries are mined. In order to eliminate the harm of weakly correlated independent variables and redundant independent variables to the estimation results, principal component feature extraction is applied to the feature extraction of the eight health indicators. Bayesian regularization algorithm is used to learn the weights of the neural network, which solves the problems of slow convergence speed and easy to fall into local extremum of back propagation learning algorithm, and improves the generalization ability of the neural network. In order to verify the rationality of the proposed model and algorithm, the coupling-loop nonlinear autoregressive with exogenous inputs neural network model is used to estimate the state of health of lithium battery under the condition of complete charge discharge test, and the estimation results are compared with other neural network models. The simulation results show that the coupling-loop nonlinear autoregressive with exogenous inputs estimation model using feature extraction method and Bayesian network weight algorithm can approach the functional with higher accuracy, and the absolute error and relative error of lithium-ion battery's state of health estimation can be reduced by more than 50% on average, while the mean square error can be reduced by more than 80%.

P–135 Artificial neural networks (ANNs) for live birth prediction in frozen embryo transfers: the strength of post-warmed blastocyst dynamics

Abstract Study question Does the post-warmed blastocyst dynamics have an impact over the likelihood of achieving a live birth? Summary answer Variables related to dynamics of vitrified/warmed blastocysts have shown a greater effect on the live birth prediction than only embryo morphological quality through artificial intelligence. What is known already Morphological dynamics of vitrified/warmed blastocysts were described by Coello et al., in 2017. The investigated markers were the thickness of zona pellucida (µm) and blastocysts area (µm2) after warming and before transfer, the area of the inner cell mass (µm2), time of initiation of reexpansion (in minutes), and presence of collapse or contraction. They found a correlation between blastocyst reexpansion and implantation rate and developed a hierarchical model for implantation prediction. In our study, we evaluated the post-warmed blastocyst dynamics for live birth prediction by using novel artificial intelligence techniques. Study design, size, duration This retrospective analysis included 415 vitrified/warmed blastocysts with known live birth data. Blastocysts after warming were placed in EmbryoScope (Vitrolife) immediately until embryo transfer. Embryo evaluation and selection were performed by senior embryologists according to fresh blastocyst morphology (before vitrification). Then, parameters related to post-warmed blastocyst dynamics were calculated. Finally, these variables and the embryo morphological grade before the vitrification were used as input data for ANNs optimized with genetic algorithm for live birth prediction. Participants/materials, setting, methods Blastocysts were vitrified and warmed by the Cryotop method (Kitazato,Biopharma). During the period between the warming procedure and the embryo transfer, the following variables were measured with the drawing tools provided by the EmbryoViewer workstation: zona pellucida thinning (µm), blastocyst expansion (um) and the speed of these two events (µm/h). Finally, multilayer perceptron neural networks were trained with data of 331 embryos by using the backpropagation learning algorithm and tested with data of 84 embryos. Main results and the role of chance We trained and tested three architectures of ANNs with different input variables as follows: post-warmed variables (thinning of the zona pellucida, blastocyst expansion, thinning speed and expansion speed) and morphological grade (A, B or C) for ANN1, only post-warmed variables for ANN2 and only morphological grade for ANN3. The highest success rate when ANNs classified embryos as positive and negative live birth (LB+ and LB-) was achieved by combining post-warmed variables and morphological grade before embryo vitrification. The general accuracies for the blind tests were: 73.8% for ANN1, 66.7% for ANN2 and 71.4% for ANN3. Likewise, this combination achieved the highest AUC on test dataset to predict LB- (0.76 for ANN1, 0.74 for ANN2 and 0.67 for ANN3). However, the ANN2 trained with only post-warmed variables showed the best capacity to predict LB+ with an AUC of 0.73 (versus 0.46 for ANN1 and 0.5 for ANN3). Limitations, reasons for caution The main limitation is the subjectivity of manual annotations, although only one embryologist participated in this task. Wider implications of the findings: The dynamics of vitrified/warmed blastocysts prior to embryo transfer could be more relevant variables than the morphological quality on day 5 before the cryopreservation. The analysis of embryo behavior after warming could improve clinical outcomes in frozen embryo transfers. Trial registration number none

TIME SERIES FORECASTING USING ARTIFICIAL NEURAL NETWORK WITH EXTENDED ADAPTIVE LEARNING

Artificial neural network (ANN) mainly consists of learning algorithms, which are require to optimize the convergence of neural networks. We need to optimize the convergence of neural networks in order to improve the speed and accuracy of decision making process. To enable the optimization process one of the widely used algorithm is back propagation learning algorithm. Objective of study is to applied backpropagation algorithm for solving multivariate time series problem. To better the accuracy of neural network it is important to find optimized architecture for the problem under consideration. The learning rate is also an important factor which affects the performance of result. In this study, we proposed extended adaptive learning approach in which learning rate is adapted from number of previous iteration error trend in first half of training. In next half of training learning rate is adapted as per adaptive learning rate algorithm. Compare performance of three variation of backpropagationalgorithm. All these variation experimented on two standard dataset. Experimental result shows that during validation and training ANN with extended adaptive learning rate outperforms other than two variations.

Time Series Forecasting with Trend and Seasonal Patterns using NARX Network Ensembles

In this research, we propose a Nonlinear Auto-Regressive network with exogenous inputs (NARX) model with a different approach, namely the determination of the main input variables using a stepwise regression and exogenous input using a deterministic seasonal dummy. There are two approaches in making a deterministic seasonal dummy, namely the binary and the sine-cosine dummy variables. Approximately half the number of input variables plus one is contained in the neurons of the hidden layer. Furthermore, the resilient backpropagation learning algorithm and the tangent hyperbolic activation function were used to train each network. Three ensemble operators are used, namely mean, median, and mode, to solve the overfitting problem and the single NARX model's weakness. Furthermore, we provide an empirical study using actual data, where forecasting accuracy is determined by Mean Absolute Percent Error (MAPE). The empirical study results show that the NARX model with binary dummy exogenous is the most accurate for trend and seasonal with multiplicative properties data patterns. For trend and seasonal with additive properties data patterns, the NARX model with sine-cosine dummy exogenous is more accurate, except the fact that the NARX model uses the mean ensemble operator. Besides, for trend and non-seasonal data patterns, the most accurate NARX model is obtained using the mean ensemble operator. This research also shows that the median and mode ensemble operators, which are rarely used, are more accurate than the mean ensemble operator for data that have trend and seasonal patterns. The median ensemble operator requires the least average computation time, followed by the mode ensemble operator. On the other hand, all of our proposed NARX models' accuracy consistently outperforms the exponential smoothing method and the ARIMA method.

Application of Taguchi method and artificial neural network model for the prediction of reductive leaching of cobalt(III) from oxidised low-grade ores

The leaching process of cobalt using a wide range of experimental variables is described. The treated cobalt samples were from the Kalumbwe Mine in the south of the Democratic Republic of Congo. In this study, a predictive model of cobalt recovery using both the Taguchi statistical method and an artificial neural network (ANN) algorithm was proposed. The Taguchi method utilising a L25 (55) orthogonal array and an ANN multi-layer, feed-forward, back-propagation learning algorithm were adopted to optimise the process parameters (acid concentration, leaching time, temperature, percentage solid, and sodium metabisulfite concentration) responsible for the high recovery of cobalt by reducing sulfuric acid leaching. The ANN was built with a neuron in the output layer corresponding to the cobalt leaching recovery, 10 hidden layers, and 5 input variables. The validation of the ANN model was performed with the results of the Taguchi method. The optimised trained neural network depicts the testing data and validation data with R2 equal to 1 and 0.5676, respectively.Significance:&#x0D; &#x0D; We statistically investigated the main factors (acid concentration, leaching time, temperature, percentage solid, and sodium metabisulfite concentration) that affect the cobalt(III) leaching performance using both the Taguchi method and artificial neural network model. This allowed us to ascertain that it is indeed possible to leach cobalt(III) from oxide ores and to identify the optimum leaching conditions.&#x0D;

Deep Spiking Neural Network with Neural Oscillation and Spike-Phase Information

Deep spiking neural network (DSNN) is a promising computational model towards artificial intelligence. It benefits from both the DNNs and SNNs through a hierarchy structure to extract multiple levels of abstraction and the event-driven computational manner to provide ultra-low-power neuromorphic implementation, respectively. However, how to efficiently train the DSNNs remains an open question because of the non-differentiable spike function that prevents the traditional back-propagation (BP) learning algorithm directly applied to DSNNs. Here, inspired by the findings from the biological neural networks, we address the above-mentioned problem by introducing neural oscillation and spike-phase information to DSNNs. Specifically, we propose an Oscillation Postsynaptic Potential (Os-PSP) and phase-locking active function, and further put forward a new spiking neuron model, namely Resonate Spiking Neuron (RSN). Based on the RSN, we propose a Spike-Level-Dependent Back-Propagation (SLDBP) learning algorithm for DSNNs. Experimental results show that the proposed learning algorithm resolves the problems caused by the incompatibility between the BP learning algorithm and SNNs, and achieves state-of-the-art performance in single spike-based learning algorithms. This work investigates the contribution of introducing biologically inspired mechanisms, such as neural oscillation and spike-phase information to DSNNs and providing a new perspective to design future DSNNs.