Feed-forward Artificial Neural Network Research Articles

Performance Analysis of Neural Network Architecture in Developing Real-Time Malaysian River Water Quality Model

According to the world health organization, 485,000 people died each year due to water-related health diseases which are mainly contributed by poor river water quality. As a result, water quality monitoring stations have been deployed across the world. Unfortunately, due to the complex nature of the off-site water quality parameters, the water quality index (WQI) cannot be assessed in real-time. This has led to a significant push for the scientific community to develop an accurate and robust water quality prediction model. The dynamic and nonlinear nature of water quality parameters are major challenges for traditional machine learning algorithms such as multi linear regression to capture. In this study, the water quality index prediction model was developed using the feedforward artificial neural network (FANN) algorithms utilizing only on-site parameters. The performance of different nonlinear activation functions in the hidden neurons was thoroughly analysed which includes rectified linear unit (ReLU), scaled exponential linear unit (SELU), and exponential linear unit (ELU). Additionally, various initialization and optimization algorithms were also evaluated for maximum performance and efficiency. The results shows that FANN-ELU model coupled with Glorot initialization technique and AdaGrad optimizer outperformed other model combinations with an R2 value of 0.88 and mean squared error (MSE) value of 22.74.

ANN based Multi Model Predictive Control for pH-Control

Artificial neural network (ANN) has many uses when non-linear behaviour is modelled. Here we are training a feedforward ANN that will mimic the behaviour of a Robust Model Predictive Controller (RMPC) for use in pH control. The training dataset were generated from running multiple tests on RMPC for different requirements and cases of pH-control. The training data focused on the control-inputs relating to the other process inputs. The training algorithm used in this neural network is Levenberg-Marquardt algorithm which is the most widely use algorithm in current machine learning industry. This neural network was trained by using the deep learning toolbox in Matlab®. Eight different cases is presented: four is for deploying neural network purpose, while the other four is for verification purpose. The result shows good control as long as the ANN-controller has been given a similar input and there are no multiplicity in the process input data.

COMPARISON OF GENETIC ALGORITHM AND NEURAL NETWORK APPROACHES FOR THE PROGNOSIS OF MECHANICAL IDLE RUNNING LOSSES IN AGRICULTURE TRACTOR TRANSMISSION

An experimental investigation of mechanical idle running losses in an agriculture tractor transmission was used to collect a wide range of data. The influence of the engine rotation speed, the number of switched-on gears, and the oil level in the transmission gearbox on the idle running losses was determined. Adequate regression models in cases of switched-on and switched-off PTO were received. A genetic algorithm was used to optimize mathematical models obtained using regression analysis. A feed-forward artificial neural network was also developed to estimate the same experimental data for mechanical idle running losses in transmission. A back-propagation algorithm was used when training and testing the network. A comparison of the correlation coefficient, reduced chi-square, mean bias error, and root mean square error between the experimental data and fit values of the obtained models was made. It was concluded that the neural network represented the mechanical idle running losses in tractor transmission more accurately than other models.

Adsorption study of removal of lead ions using Prosopis juliflora and prediction by artificial neural network modeling

The batch mode process is used to remove the lead (II) ions from the wastewater by employing the seed powder of Prosopis juliflora. The effect of concentration of lead ions (20 – 60 mg/L), time (70 – 120 min), and pH (4 – 6) were investigated by Box-Behnken Design (BBD) in response to surface methodology and Feed Forward Artificial Neural Network (ANN). The maximum removal of lead (II) ions was found to be 89.6277 % during the time 119.69 at pH 5.69 from the Box-Behnken design (BBD). An ANN model was developed having 10 neurons in the hidden layer. The SEM photos depict the adsorbent's surface texture as well as lead (II) metal ion adsorption.

BombNose: A Multiple Bomb-Related Gas Prediction Model Using Machine Learning with Electronic Nose Sensor Substitution Technique

The safety and security of an individual is important in our society. Bombing attacks can cause significant destruction and death. Energy efficient and compact bomb removal robots are challenging to develop because these typically involved a large array of sensors individually acquiring gas data. This study addresses this challenge by developing a multiple bomb-related gas prediction model using machine learning and the electronic nose sensor substitution technique. Three models can predict gasses such as ammonia, ethanol, and isobutylene using only carbon monoxide, toluene, and methane sensors. The feedforward artificial neural network (FFNN) with three hidden layers was optimized for the regression of each target gas. Consequently, ammonia, ethanol, and isobutylene predictions achieved R2 values of 1, 1, and 1 as well as MSE values of 0.35696, 0.052995, and 0.0022953, respectively. This study demonstrates that the sensor substitution model (BombNose) is highly reliable and appropriately sensitive in the field of bomb detection.

Artificial Neuron-Based Model for a Hybrid Real-Time System: Induction Motor Case Study

Automatic Machine Learning (AML) methods are currently considered of great interest for use in the development of cyber-physical systems. However, in practice, they present serious application problems with respect to fitness computation, overfitting, lack of scalability, and the need for an enormous amount of time for the computation of neural network hyperparameters. In this work, we have experimentally investigated the impact of continuous updating and validation of the hyperparameters, on the performance of a cyber-physical model, with four estimators based on feedforward and narx ANNs, all with the gradient descent-based optimization technique. The main objective is to demonstrate that the optimized values of the hyperparameters can be validated by simulation with MATLAB/Simulink following a mixed approach based on interleaving the updates of their values with a classical training of the ANNs without affecting their efficiency and automaticity of the proposed method. For the two relevant variables of an Induction Motor (IM), two sets of estimators have been trained from the input current and voltage data. In contrast, the training data for the speed and output electromagnetic torque of the IM have been established with the help of a new Simulink model developed entirely. The results have demonstrated the effectiveness of ANN estimators obtained with the Deep Learning Toolbox (DLT) that we used to transform the trained ANNs into blocks that can be directly used in cyber-physical models designed with Simulink.

RENEWABLE SOLAR ENERGY SYSTEMS FOR SUSTAINABLE DRYING OF BANANA (MUSA ACUMINATA) PEEL BIOMASS.

Huge amount of banana peel biomass are generated annually which are excellent sources of bioenergy, biofuels and other value-added products. The initial moisture content of banana peel is about 70%, which leads to deterioration and limits the efficiency of conversion processes. Solar energy is abundant, free and renewable, so the solar drying of banana peel was investigated. Drying kinetics of banana peel in passive and active solar dryers were compared with that of direct sunlight. Mathematical and artificial neural network (ANN) modelling methods were applied to describe the rate of drying of the peel. The banana peel dried fastest in the active dryer followed by the passive dryer, then direct sunlight. The peels dried in the falling rate period. The Verma, Midilli-Kucuk and Weibull models best described the peel drying kinetics in direct sunlight, passive and active solar dryers, respectively. Feed-forward multilayer perception ANNs having (4-3-1) network topologies best fitted the drying data. The estimated diffusivities of moisture in the peel were 7.835 × 10-11, 9.59 × 10-11 and 1.952 × 10-10 m2 s-1 during its drying in direct sunlight, passive and active solar dryers, respectively. Renewable solar energy can sustainably remove moisture from banana peel biomass. Keywords: ANN Modelling, Banana peel, drying kinetics, mathematical modelling, solar dryer, thin layer

Applicability of Artificial Neural Network for Automatic Crop Type Classification on UAV-Based Images

Recent advances in optical remote sensing, especially with the development of machine learning models have made it possible to automatically classify different crop types based on their unique spectral characteristics. In this article, a simple feed-forward artificial neural network (ANN) was implemented for the automatic classification of various crop types. A DJI Mavic air drone was used to simultaneously collect about 549 images of a mixed-crop farmland belonging to Federal University of Technology Minna, Nigeria. The images were annotated and the ANN algorithm was implemented using custom-designed Python programming scripts with libraries such as NumPy, Label box, and Segmentation Mask, for the classification. The algorithm was designed to automatically classify maize, rice, soya beans, groundnut, yam and a non-crop feature into different land spectral classes. The model training performance, using 70% of the dataset, shows that the loss curve flattened down with minimal over-fitting, showing that the model was improving as it trained. Finally, the accuracy of the automatic crop-type classification was evaluated with the aid of the recorded loss function and confusion matrix, and the result shows that the implemented ANN gave an overall training classification accuracy of 87.7% from the model and an overall accuracy of 0.9393 as computed from the confusion matrix, which attests to the robustness of ANN when implemented on high-resolution image data for automatic classification of crop types in a mixed farmland. The overall accuracy, including the user accuracy, proved that only a few images were incorrectly classified, which demonstrated that the errors of omission and commission were minimal.

Https://dsr.mutah.edu.jo/index.php/jje/article/view/3

This paper presents a new islanding detection technique based on an artificial neural network (ANN) for a doubly fed induction wind turbine (DFIG). This technique takes advantage of ANN as pattern classifiers. Five different ANN systems are presented in this paper based on various inputs: three phase power, phase voltage, phase current, neutral voltage, and neutral current. An ANN structure is trained for each input, and the comparison between the different structures is presented. Feedforward ANN structures are used for the five systems. Three different learning algorithms are used: backpropagation and two artificial optimization techniques: Genetic Algorithm (GA) and Cuckoo optimization algorithm. For each method in each training technique, the results and the cost function are presented. The comparison of different inputs different algorithms is conducted. MATLAB 2020a is used to simulate the ANN structure and code the training algorithms. A detailed discussion of the input sample rate has also been manipulated to make the computational burden a factor in assessing the performance.

Quantification of glycated hemoglobin and glucose in vivo using Raman spectroscopy and artificial neural networks

Undiagnosed type 2 diabetes (T2D) remains a major public health concern. The global estimation of undiagnosed diabetes is about 46%, being this situation more critical in developing countries. Therefore, we proposed a non-invasive method to quantify glycated hemoglobin (HbA1c) and glucose in vivo. We developed a technique based on Raman spectroscopy, RReliefF as a feature selection method, and regression based on feed-forward artificial neural networks (FFNN). The spectra were obtained from the forearm, wrist, and index finger of 46 individuals. The use of FFNN allowed us to achieve an error in the predictive model of 0.69% for HbA1c and 30.12 mg/dL for glucose. Patients were classified according to HbA1c values into three categories: healthy, prediabetes, and T2D. The proposed method obtained a specificity and sensitivity of 87.50% and 80.77%, respectively. This work demonstrates the benefit of using artificial neural networks and feature selection techniques to enhance Raman spectra processing to determine glycated hemoglobin and glucose in patients with undiagnosed T2D.

Exploring the Potential of Neural Networks to Predict Statistics of Solar Wind Turbulence

AbstractTime series data sets often have missing or corrupted entries, which need to be handled in subsequent data analysis. For example, in the context of space physics, calibration issues, satellite telemetry issues, and unexpected events can make parts of a time series unusable. This causes problems for understanding the dynamics of the heliosphere and space weather environment. Various approaches exist to tackle this problem, including mean/median imputation, linear interpolation, and autoregressive modeling. Here, we study the utility of artificial neural networks (ANNs) to predict statistics of sparse time series. Our focus is not on time series prediction but on gleaning the best possible information about the statistical behavior of the system. As an example application, we focus on the structure functions of turbulent time series measured in the solar wind. Using a data set with artificial gaps, a neural network is trained to predict second‐order structure functions and then tested on an unseen data set to quantify its performance. A small feedforward ANN, with only 20 hidden neurons, can predict the large‐scale fluctuation amplitudes better than mean imputation or linear interpolation when the percentage of missing data is high. Although they perform worse than the other methods when it comes to capturing both the shape and fluctuation amplitude together, their performance is better in a statistical sense for large fractions of missing data. Caveats regarding their utility, the optimization procedure, and potential future improvements are discussed.

An ANN-based protection technique for MTDC systems with multiple configurations

A unified protection technique for different configurations of Multi-Terminal high voltage Direct Current (MTDC) systems is presented in this paper. The technique is based on using positive and negative poles’ currents of one end for each line. The spectral energy of two distinct frequency bands captured from fault currents, together with the fault current initial slope, are used to supply a feed forward Artificial Neural Network (ANN) for identifying fault zone and classifying fault type. Different configurations of MTDC systems are used to train and test the proposed ANN including radial system, meshed system without Current Limiting Reactors (CLRs), and meshed system with CLRs. Different fault types, fault locations, fault inception times, and fault resistances are used to train and test the proposed ANN. The proposed algorithm showed an excellent performance in the identification of the fault zone and the classification of the fault type for various MTDC configurations. Moreover, the algorithm does not respond to non- DC fault abnormal conditions.

Intelligent Cattle Detection and Recognition System Using ANN-Fourier Descriptor Techniques

Reoccurring Fulani-farmer crisis in Nigeria as a result of the destruction of farmland and arable crops by cattle during grazing has led to a high rate of loss of life and properties. To address this problem, this research work proposed a shape- based ANN-Fourier descriptor cattle recognition system. Twenty samples of each of cattle, deer, dog, elephant, horse, and camel were obtained from the UCL database. Colour space conversion and a simple image binarization were performed to obtained image segmentation. Edges of the images were extracted, and Discrete Fourier Transform was applied on the edges to obtain the Fourier descriptors. Ten Fourier descriptors were used to train feed-forward back propagation artificial neural networks for cattle recognition systems. The effect of increasing database size, the number of images used for training and testing, and threshold value were investigated. It was observed that there was no effect of increasing the size of the database on the system performance. For better accuracy, sensitivity, specificity, and precision, it was observed that a good threshold value must be carefully chosen and 75% of the image must be used for training and 25% for testing respectively. With twenty image samples, each for cattle, deer, camel, dog, horse, and elephant in the database, fifteen out of twenty samples were used for training, five for testing, and the threshold value of 0.6, 98.7% sensitivity, 98.8% specificity, 98.8% accuracy and 94.5% precision were achieved. This result provides a good method for cattle detection and recognition system.

SVM and ANN Modelling Approach for the Optimization of Membrane Permeability of a Membrane Rotating Biological Contactor for Wastewater Treatment.

Membrane fouling significantly hinders the widespread application of membrane technology. In the current study, a support vector machine (SVM) and artificial neural networks (ANN) modelling approach was adopted to optimize the membrane permeability in a novel membrane rotating biological contactor (MRBC). The MRBC utilizes the disk rotation mechanism to generate a shear rate at the membrane surface to scour off the foulants. The effect of operational parameters (disk rotational speed, hydraulic retention time (HRT), and sludge retention time (SRT)) was studied on the membrane permeability. ANN and SVM are machine learning algorithms that aim to predict the model based on the trained data sets. The implementation and efficacy of machine learning and statistical approaches have been demonstrated through real-time experimental results. Feed-forward ANN with the back-propagation algorithm and SVN regression models for various kernel functions were trained to augment the membrane permeability. An overall comparison of predictive models for the test data sets reveals the model’s significance. ANN modelling with 13 hidden layers gives the highest R2 value of >0.99, and the SVM model with the Bayesian optimizer approach results in R2 values higher than 0.99. The MRBC is a promising substitute for traditional suspended growth processes, which aligns with the stipulations of ecological evolution and environmentally friendly treatment.

Multimodal tract-based MRI metrics outperform whole brain markers in determining cognitive impact of small vessel disease-related brain injury

In cerebral small vessel disease (cSVD), whole brain MRI markers of cSVD-related brain injury explain limited variance to support individualized prediction. Here, we investigate whether considering abnormalities in brain tracts by integrating multimodal metrics from diffusion MRI (dMRI) and structural MRI (sMRI), can better capture cognitive performance in cSVD patients than established approaches based on whole brain markers. We selected 102 patients (73.7 ± 10.2 years old, 59 males) with MRI-visible SVD lesions and both sMRI and dMRI. Conventional linear models using demographics and established whole brain markers were used as benchmark of predicting individual cognitive scores. Multi-modal metrics of 73 major brain tracts were derived from dMRI and sMRI, and used together with established markers as input of a feed-forward artificial neural network (ANN) to predict individual cognitive scores. A feature selection strategy was implemented to reduce the risk of overfitting. Prediction was performed with leave-one-out cross-validation and evaluated with the R2 of the correlation between measured and predicted cognitive scores. Linear models predicted memory and processing speed with R2 = 0.26 and R2 = 0.38, respectively. With ANN, feature selection resulted in 13 tract-specific metrics and 5 whole brain markers for predicting processing speed, and 28 tract-specific metrics and 4 whole brain markers for predicting memory. Leave-one-out ANN prediction with the selected features achieved R2 = 0.49 and R2 = 0.40 for processing speed and memory, respectively. Our results show proof-of-concept that combining tract-specific multimodal MRI metrics can improve the prediction of cognitive performance in cSVD by leveraging tract-specific multi-modal metrics.

A deep learning approach to predict the number of [formula omitted]-barriers for intrusion detection over a circular region using wireless sensor networks

Wireless Sensor Networks (WSNs) is a promising technology with enormous applications in almost every walk of life. One of the crucial applications of WSNs is intrusion detection and surveillance at border areas and in the defence establishments. The border areas are stretched over hundreds to thousands of miles, hence, it is not possible to patrol the entire border region. As a result, an enemy may enter from any point absence of surveillance and cause the loss of lives or destroy the military establishments. WSNs can be a feasible solution for the problem of intrusion detection and surveillance at the border areas. Detection of an enemy at the border areas and nearby critical areas such as military cantonments is a time-sensitive task as a delay of a few seconds may have disastrous consequences. Therefore, it becomes imperative to design systems that can identify and detect the enemy as soon as it comes within the range of the deployed system. In this paper, we have proposed a deep learning architecture based on a fully connected feed-forward Artificial Neural Network (ANN) for the accurate prediction of the number of k-barriers for fast intrusion detection and prevention. We have trained and evaluated the feed-forward ANN model using four potential features, namely area of the circular region, sensing range of sensors, transmission range of sensors, and number of sensor for Gaussian and uniform sensor distribution. These features are extracted through Monte Carlo simulation. In doing so, we found that the model accurately predicts the number of k-barriers for both Gaussian and uniform sensor distribution with correlation coefficient (R = 0.78) and Root Mean Square Error (RMSE = 41.15) for the former and R = 0.79 and RMSE = 48.36 for the latter. Further, the proposed approach outperforms the other benchmark algorithms in terms of accuracy and computational time complexity.

Peak ground acceleration prediction using artificial neural networks for Kachchh, Gujarat, India

The uncertainty in the empirical ground motion prediction models (GMMs) for any region depends on several parameters. In the present work, we apply an artificial neural network (ANN) to design a GMM of peak ground acceleration (PGA) for Kachchh, Gujarat, India, utilizing independent input parameters viz., moment magnitudes, hypocentral distances, focal depths and site proxy (in terms of average seismic shear-wave velocity from the surface to a depth of 30 m (Vs30)). The study has been performed using a PGA dataset consisting of eight engineering seismoscope records of the 2001 Mw7.7 Bhuj earthquake and 237 strong-motion records of 32 significant Bhuj aftershocks of Mw3.3–5.6 (during 2002–2008) with epicentral distances ranging from 1.0 to 288 km. We apply a feed-forward back propagation ANN method with 8 hidden nodes, which is found to be optimal for the selected PGA database and input–output mapping. The standard deviation of the error has been utilized to examine the performance of our model. We also test the ground motion predictability of our ANN model using real recordings of the 2001 Bhuj mainshock, two Mw5.6 Kachchh aftershocks and the 1999 Mw6.4 Chamoli mainshock. The standard deviation of PGA prediction error estimates in log10 units is found to be ± 0.2554. Also, the model predictability of our ANN model suggests a good prediction of the PGA for earthquakes of Mw5.6–7.7, which are occurring in Kachchh, Gujarat, India.

Machine learning and remote sensing-based modeling of the optimal stomatal behavior of crops

The slope (g1) of stomatal conductance to photosynthesis is an important parameter in the optimal stomatal behavior theory-based stomatal conductance model of Medlyn et al. (2011). Although studies have modelled the spatial variations in g1, disclosing its variations over environmental gradients and different plant functional types. However, the above methods are still not accurate enough on a global scale, as they do not consider the temporal variations in g1. To address this issue we used the Ensemble Kalman Filter (EnKF) to assimilate tower-based gross primary productivity (GPP) and latent heat flux (LE) of 17 cropland flux sites into a remote sensing (RS)-based evapotranspiration-photosynthesis coupled model, termed SCOPES-Crop, to derive the temporal variations in g1 for C3 and C4 crops. We also used the feedforward artificial neural network (FANN) along with RS variables to model g1. Results showed g1 to rise rapidly in spring and summer, and then decline in autumn. The value of g1 reached the lowest value and remained stable in wintertime. FANN-based modeling of g1 showed R(RMSE) = 0.81 (1.94 kPa0.5) and 0.90 (0.70 kPa0.5) for C3 and C4 Crops, respectively, for the testing dataset. The estimates of GPP and LE using FANN-derived g1 at the 17 flux sites were improved as compared to that using fixed g1. The mean values of site-level R(RMSE) for GPP and LE simulated using FANN-derived g1 are 0.92 (1.8 gC m−2 d-2) and 0.85 (22.5 W m−2), respectively. Our results revealed notable seasonal variations in g1, indicating the importance of considering the temporal variations in g1 in evapotranspiration-photosynthesis coupled model. The FANN along with RS variables showed great potential of representing the g1 variations.

A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions

In this article we study the stochastic gradient descent (SGD) optimization method in the training of fully connected feedforward artificial neural networks with ReLU activation. The main result of this work proves that the risk of the SGD process converges to zero if the target function under consideration is constant. In the established convergence result the considered artificial neural networks consist of one input layer, one hidden layer, and one output layer (with d in {mathbb {N}} neurons on the input layer, Hin {mathbb {N}} neurons on the hidden layer, and one neuron on the output layer). The learning rates of the SGD process are assumed to be sufficiently small, and the input data used in the SGD process to train the artificial neural networks is assumed to be independent and identically distributed.

A simple and rapid spectrophotometric method coupled with intelligent approaches for the simultaneous determination of antiepileptic drugs in pharmaceutical formulations, biological, serological, and breast milk samples

In this study, simultaneous measurement of Phenytoin (PHT), Levetiracetam (LEV), and Pregabalin (PRG) was assessed by chemometrics methods, including feedforward artificial neural network (FF-ANN) with Levenberg-Marquardt (LM) and Conjugate Gradient with Fletcher Reeves updates (CGF), as well as least squares support vector machine (LS-SVM) along with the spectrophotometric method in the formulation of antiepileptic drugs, biological, serological, as well as breast milk samples were examined. According to better results of the LM algorithm than the other one, layers 2 with 8, 10, and 8 neurons with the least error (mean square error (MSE)) as the best layer and neurons were selected for PHT (MSE = 3 × 10−28), LEV (MSE = 7.67 × 10−27), and PRG (MSE = 4.43 × 10−28), respectively in LM algorithm. The mean recovery values were achieved in the range of 97.39%–102.57% and 85.01%–112.40% for LM and CGF, respectively. The regularization parameter (γ) values of the LS-SVM model were found to be 240, 70, and 80 for PHT, LEV, and PRG, respectively. Also, the width of the function (σ2) were 270,350, 230 for PHT, LEV, and PRG, respectively. The mean recovery of this method was found to be 99.50%, 99.44%, and 99.26% for PHT, LEV, PRG, respectively. The results of the proposed methods on the samples of tablets and capsules were compared with high-performance liquid chromatography (HPLC) as a reference method with one-way analysis of variance (ANOVA) test at 95% confidence level, which did not show a significant difference between the approaches. The proposed methods are fast, simple, inexpensive, accurate, and without the need for the separation steps that can be used to improve the quality of pharmaceutical products.