Weights Of Artificial Neural Network Research Articles

Artificial Neural Network-Based Nonlinear Black-Box Modeling of Synchronous Generators

This article deals with the black-box modeling of synchronous generators based on artificial neural networks (ANN). The ANN is applied to define the relationship between the excitation and terminal generator voltage values, and the Levenberg–Marquardt algorithm is used for determining the ANN weight coefficients. The relation is made based on generator response on reference voltage step changes. The proposed approach is checked using the experimental results obtained from the measurements on a real 120 MVA generator from a hydroelectric power plant Piva in Montenegro. Furthermore, a fair comparison of the nonlinear autoregression model with the exogenous input (NARX) and Hammerstein–Wiener model is made. For the validation, different experiments were conducted—different values of step disturbances, other controller parameters, and different rotating speeds. Based on the presented results, it can be noted that the proposed ANN model is very accurate and provides a very high degree of matching with the experimental results and outperforms the other considered nonlinear models. Furthermore, the proposed test procedure and model are easy to implement and do not require disconnection of the generator from the grid or additional equipment for experimental realization. Such obtained models can be used for different testing types related to the excitation system.

Predicting mechanical behaviors of rubber materials with artificial neural networks

Rubber is considered as a new material for making landing gear shock absorbers for new energy electric aircraft. This investigation tested nitrile butadiene rubber blocks with different hardness at different loading rates to determine their stress-strain relationships. The mechanical behaviors of rubbers are predicted via an artificial neural network model for the first time. A self-adjusting particle swarm optimization was proposed to optimize the artificial neural network's weights and thresholds with the inertia factors and learning factors automatically adjusted. The number of neurons was optimized. The results revealed that the Young's modulus of the rubber increases with the increasing strain. With larger hardness and a larger strain rate, both the Young's modulus and its increasing rate become larger. The proposed self-adjusting particle swarm optimization improves the accuracy and the convergence speed. With the optimal number of neurons, the proposed artificial neural network has a mean square error of 1.611 × 10−4 in predicting the stress-strain relationships of the rubber. Compared with traditional artificial neural network models and an artificial neural network optimized by a traditional particle swarm optimization, the proposed model increases the accuracy by 56.5% and 26.5%, respectively. It is an initial attempt to account the coupling effect of the hardness and the strain rate in the rubber constitutive model.

A physics-constrained data-driven workflow for predicting bottom hole pressure using a hybrid model of artificial neural network and particle swarm optimization

Accurate and efficient prediction of bottom hole pressure (BHP) is important for managed pressure drilling (MPD), which is essential to ensure the safety of drilling in complex formations with a narrow pore/fracture pressure envelope. Without idealized assumptions and iterative solutions, the data-driven machine learning model has higher prediction accuracy and efficiency than the mechanistic hydraulics model. However, the machine learning models suffering from the significant impact of noise data and the strict restriction of the data field, often leads to anomalous deviations. Physical constraints are inherent mappings between output values and characteristic variables, which can be applied to model training to improve the robustness of the model.In this study, wellbore flow mechanism is considered as the physical constraint, and a physics-constrained data-driven workflow is proposed for stable prediction of BHP, which is more consistent with the hydraulic mechanism. Firstly, more than 400,000 groups of field pressure data are extracted as the training dataset by an automatic identification method of drilling state. And twelve characteristic parameters of BHP were optimized, including inlet flow rate, outlet density and wellhead pressure. Embedding physical constraints into the loss function of artificial neural network (ANN) as penalty terms can induce ANN model output results within the wellbore flow mechanism. Finally, particle swarm algorithm is introduced to solve the weight and bias of ANN globally without the derivative of the restructuring loss function.The proposed model is verified based on the field pressure data. It could be found that both experiential and knowledge-based constraints can improve the accuracy and stability of the ANN model, the prediction error is significantly reduced, MRE, RMSE and MAE were respectively reduced to 0.46%, 0.34 MPa and 0.27 MPa. RMSE decreased by 20.9%, MRE and MAE decreased by 19.3% and 18.1%, respectively. Meanwhile, the R2 of the model reached 0.9871. Sensitive parameters such as flow rate have a more significant effect on the prediction accuracy of BHP, while non-sensitive parameters such as mud density can eliminate the abnormal deviation of BHP more effectively, and the constraints of multiple parameters can be superimposed. Therefore, optimizing the constraint combination according to the fluctuation characteristics of BHP is crucial to improve the accuracy and stability of the ANN model. This is an innovative exploration of the physical constraints on the data-driven model of BHP, which can provide accurate and efficient references for managed pressure drilling.

Uncertainty quantification of multimodal surface wave inversion using artificial neural networks

An inversion of surface wave dispersion curves (DCs) is a nonunique and ill-conditioned problem. The inversion result has a probabilistic nature, which becomes apparent when simultaneously restoring the S-wave velocity and layer thickness. Therefore, the problem of uncertainty quantification is relevant. Existing methods through deterministic or global optimization approaches of uncertainty quantification via posterior probability density (PPD) of the model parameters are not computationally efficient because they demand multiple solutions to the inverse problem. We have developed an alternative method based on a multilayer fully connected artificial neural network (ANN). We improve the current unimodal approach, which is known from publications, to multimodal inversion. We use Cox’s and Teague’s algorithm to determine optimal parameterization (number of layers) and the ranges of possible model parameters. We uniformly draw training data sets within estimated ranges and train the ANN. Saved ANN weights map the phase velocity DCs to values of the S-wave velocity and layers thickness. To estimate the uncertainties, we adapt the Monte Carlo simulation strategy, according to which the frequency dependent data noise and the inverse operator errors are projected onto the resulting velocity model. The inverse operator errors are evaluated by the prediction of the training data set. The proposed combination of surface wave data processing methods, configured with each other, provides a novel surface wave multimodal dispersion data inversion and uncertainty quantification approach. We first test our approach on synthetic experiments for various velocity models: a positive gradient velocity, a low-velocity layer, and a high-velocity layer. This is done considering unimodal inversion at first and then compared with the multimodal inversion. Afterward, we apply our approach to field data and compare the resulting models with the body S-wave processing by the generalized reciprocal method. The experiments indicate high-potential results — using ANN yields the possibility to accurately estimate the PPD of restored model parameters without a significant computational effort. The PPD-based comparison demonstrates the advantages of a multimodal inversion over a unimodal inversion. The trained ANN provides reasonable model parameters predictions and related uncertainties in real time.

Biogeography-based Optimization of Artificial Neural Network (BBO-ANN) for Solar Radiation Forecasting

ABSTRACT Renewable energy can help India’s economy and society. Solar energy is everywhere and can be used anywhere, making it popular. Solar energy’s drawbacks are weather and environmental dependencies and solar radiation variations. Solar Radiation Forecasting (SRF) reduces this drawback. SRF eliminates solar power generation variations, grid overvoltage, reverse current, and islanding. Short-term solar radiation forecasts improve photovoltaic (PV) power generation and grid connection. Previous promising SRF studies often fail to generalize to new data. A biogeography-based optimization artificial neural network (BBO-ANN) model for SRF is proposed in this work. 5-year and 6-year data are used to train and validate the model. The data was collected from India’s Jaipur Rajasthan weather station from 2014 to 2019. This work used biogeography-based optimization (BBO) to optimize and adjust the inertia weight of artificial neural networks (ANN) during training. The BBO-ANN model developed in this study had a Mean Absolute Percentage Error (MAPE) of 3.55%, which is promising compared to previous SRF studies. The BBO-ANN SRF model introduced in this work can generalize well to new data because it was able to produce equally accurate autumn and winter forecasts despite the great climatic variation that occurs during the summer and spring.

Dual-gate manipulation of a HfZrOx-based MoS2 field-effect transistor towards enhanced neural network applications.

Artificial neural networks (ANNs) have strong learning and computing capabilities, and alleviate the problem of high power consumption of traditional von Neumann architectures, providing a solid basis for advanced image recognition, information processing, and low-power detection. Recently, a two-dimensional (2D) MoS2 field-effect transistor (FET) integrating a Zr-doped HfO2 (HZO) ferroelectric layer has shown potential for both logic and memory applications with low power consumption, which is promising for parallel processing of massive data. However, the long-term potentiation (LTP) characteristics of such devices are usually non-linear, which will affect the replacement of ANN weight values and degrade the ANN recognition rate. Here, we propose a dual-gate-controlled 2D MoS2 FET employing HZO gate stack with a crested symmetric structure to reduce power consumption. Improved nonlinearity of the LTP properties has been achieved through the electrical control of the dual gates. A recognition rate reaching 100% is obtained after 60 training epochs, and is 7.89% higher than that obtained from single-gate devices. Our proposed device structure and experimental results provide an attractive pathway towards high-efficiency data processing and image classification in the advanced artificial intelligence field.

Complexity emerging from simplicity: Bifurcation analysis of the weights time series in a feedforward neural network

Artificial neural networks (ANN), inspired by biological neural networks and the human brain, have attracted neuroscience researchers’ interest to simulate cognitive processes in recent years. However, the learning mechanisms inside the artificial neural networks are still unclear. Time series analysis of the weights of a network during the training stage can illustrate the dynamics behind the weights and provide new insight into the learning process.In this study, we investigate the dynamics of the weights of a three-neuron feed-forward artificial neural network based on weights recursive maps and time series. The dynamics of the weights are analyzed from the perspective of chaos theory fundamentals such as bifurcation analysis, equilibrium point analysis, and coexisting attractor emergence. While the previous works only examine the effect of in-network factors on the weights dynamics, here we investigate both in-network and out-of-network factors, including parameters, network structure, network input and output, bias, and activation function. The results show that weights time series exhibit rich nonlinear dynamics, including intermittency and chaos, even in the simplest structures of the network. Bifurcation analysis shows the emergence of several coexisting attractors, meaning that the learning process in an artificial neural network, and maybe in the human brain, takes place on different attractors. Hence, this is the initial condition of the brain that determines in which attractor the learning should occur.For the first time, using the recursive map of the weights, we introduce a novel discrete neuronal model with learning capability, which maps an input to an output. This ability distinguishes our model from the previous ones that only simulate the neuron electrical voltage without any learning process inside. This model can simulate cognitive phenomena such as neuronal synchronization in near-realistic condition during the training stage.

In-Depth Case Study on Artificial Neural Network Weights Optimization Using Meta-Heuristic and Heuristic Algorithmic Approach

The Meta-heuristic and Heuristic algorithms that have been introduced for deep neural network optimization is in this paper. Artificial Intelligence, and also the most used Deep Learning methods are all growing in popularity these days, thus we need faster optimization strategies for finding the results of future activities. Neural Network Optimization with Particle Swarm Optimization, Backpropagation (BP), Resilient Propagation (Rprop), and Genetic Algorithm (GA) is used for numerical analysis of different datasets and comparing each other to find out which algorithms work better for finding optimal solutions by reducing training loss. Genetic algorithm and also bio-inspired Particle Swarm Optimization is introduced in this paper. Besides, Resilient Propagation and Conventional Backpropagation algorithms which are application-specific algorithms have also been introduced. Meta-heuristic algorithms GA and PSO are a higher-level formula and problem-independent technique that may be used to a diverse number of challenges. The characteristic of Heuristic algorithms has extremely specific features that vary depending on the problem. The conventional Backpropagation (BP) based optimization, the Particle Swarm Optimization methodology, and Resilient Propagation (Rprop) are all fully presented, and how to apply these procedures in Artificial Deep Neural networks Optimization is also thoroughly described. Applied numerical simulation over several datasets proves that the Meta-heuristic algorithm Particle Swarm Optimization and also Genetic Algorithm performs better than the conventional heuristic algorithm like Backpropagation and Resilient Propagation.

Solar Photovoltaic Power Estimation Using Meta-Optimized Neural Networks

Solar photovoltaic technology is spreading extremely rapidly and is becoming an aiding tool in grid networks. The power of solar photovoltaics is not static all the time; it changes due to many variables. This paper presents a full implementation and comparison between three optimization methods—genetic algorithm, particle swarm optimization, and artificial bee colony—to optimize artificial neural network weights for predicting solar power. The built artificial neural network was used to predict photovoltaic power depending on the measured features. The data were collected and stored as structured data (Excel file). The results from using the three methods have shown that the optimization is very effective. The results showed that particle swarm optimization outperformed the genetic algorithm and artificial bee colony.

Photocatalytic removal of cefazolin in a photoreactor packed with TiO2-P25 nanoparticles supported on glass beads: an artificial neural network modeling

ABSTRACT To operationalise the heterogeneous photocatalysis process, the present study investigated the efficiency of TiO2-P25 nanoparticles immobilised on glass beads using the heat attachment method to remove cefazolin (CEF) as an antibiotic contaminant. The characteristics of the glass beads coated with TiO2-P25 nanoparticles were probed by the Scanning Electron Microscopy (SEM) technique. The immobilised TiO2-P25 nanoparticles manifested considerable efficiency in CEF removal in different operational conditions. The impact of various parameters, such as the ultraviolet light radiation time, initial concentration of CEF, light source power, inlet liquid volumetric flow rate, and pH of the solution, on the efficiency of CEF removal were examined. The results reveal that the removal percentage goes up as the irradiation time, volumetric flow rate, pH of the solution, and light source power increase while declining with a rise in the initial concentration of CEF. According to the findings, an initial CEF concentration of 20 mg L−1 is entirely removed by 40 min UV irradiation with a 16 W light source, the volumetric flow rate of 300 mL min−1, and the pH of 9.5. The reaction rate constant and adsorption coefficient of CEF on immobilised TiO2 were estimated at 0.622 mg L−1 min−1 and 0.095 mg−1 L via Langmuir- Hinshelwood kinetics. The outcomes of the mineralisation studies also display a considerable reduction of TOC and evolution of significant mineralisation products, like , , , and . The experimental results were modelled by artificial neural networks (ANN). Comparing the experimental results with the ANN-predicted data shows the acceptable efficiency of modelling with ANN. The minimum MSE, R2 value for all data (validation, training, and test), and R2 value for simulated data were obtained at 0.00027811, 0.9971, and 0.9809, respectively. The relative importance of the parameters affecting the process evaluated by the ANN weights indicates that the irradiation time is the most important factor in the photocatalytic removal of CEF.

Improving the Performance of Sentiment Analysis Using Enhanced Preprocessing Technique and Artificial Neural Network

With the presence of a massive amount of digitally recorded data, an automated computation can be preferable over the manual approach to evaluate sentiments within given textual fragments. Artificial neural network (ANN) is preferred for sentiment analysis (SA) because of its learning ability and adaptive nature towards diverse data. Handling negation in SA is a challenging task, and to address the same, we propose a specific order of preprocessing (PPR) steps to enhance the performance of SA using ANN. Typically, ANN weights are randomly initialized (R-ANN), which may not give the desired performance. As a potential solution, we propose a novel approach named Matching features with output label based Advanced Technique (MAT) to initialize the ANN weights (MAT-ANN). Simulation results conclude the superiority of the proposed approach PPR+MAT-ANN compared to the existing approach EPR+R-ANN i.e., integrating existing preprocessing (EPR) steps with R-ANN. Moreover, PPR+MAT-ANN architecture is significantly simpler than the existing deep learning-based approach named the NeuroSent tool and gives better performance when evaluated upon the Dranziera protocol.

Neural network training for serial multisensor of autonomous vehicle system

<span>This study aims to find the best artificial neural network weight values to be applied to the autonomous vehicle system with ultrasonic multisensor. The implementation of neural network in the system required long time process due to its training process. Therefore, this research is using offline training before implementing to online training by embedding the best network weight values to obtain the outputs faster according to desired targets. Simulink were used to train the system offline. Eight ultrasonic sensors are used on all sides of the vehicle and arranged in a serial multisensory configuration as inputs of neural network. With eight inputs, one sixteen-depth hidden layer, and five outputs, it was trained using the back-propagation algorithm of artificial neural network. By 100000 iterations, the output values and the target values are almost the same, indicating its convergency with minimum of errors. The result of this training is the best weights of the networks. These weight values can be implemented as fixed-weight in online training.</span>

Machine learning analysis and prediction of N2, N2O, and O2 adsorption on activated carbon and carbon molecular sieve.

This research focuses on predicting the adsorbed amount of N2, O2, and N2O on carbon molecular sieve and activated carbon using the artificial neural network (ANN) approach. Experimental isotherm data (data set 1242) on adsorbent type, gas type, temperature, and pressure of the process adsorption were used as input datasets for network investigation utilizing the Sips and dual-site Langmuir isotherm models. The network's output has been used to assess the quantity of gas adsorbed. The Gaussian algorithm was applied as a single 98-neuron hidden layer from a radial based functions (RBF) approach, and the Bayesian regularization (BR) algorithm was used as a two-layer network deep learning from a multi-layer perceptron (MLP) approach utilizing 20 neurons. The MLP and RBF networks would have the best mean square error (MSE) after 98 and 100 epochs, respectively, validating efficiencies of 0.00008 and 0.00033, while the square of the coefficient of correlations (R2) was 0.9996 and 0.9993, respectively. The ANN weight matrix generated can accurately predict the adsorption process behavior of different carbon-based adsorbents under various process conditions for air separation and N2O adsorption. The results of this study have the potential to assist a wide range of process industries.

Analysis and improvements on feature selection methods based on artificial neural network weights

The alignment of protein networks between different species has become a topic of considerable interest in recent years. To perform such alignments between two protein networks, it is necessary to consider the biological functions in which proteins are involved and the structure of the networks topology. Due to the existing complexity of using both topological and biological similarity during the alignment process, many current tools sacrifice the biological similarity of their solutions in order to improve their topological score. This happens because they try to combine these two objectives that are conflicting into a single objective, which leads to the need to subjectively decide about the weight to be given to each one. To remedy this situation, we present MOMEA, a Multi-Objective Mutation-based Evolutionary Algorithm that uses intelligent mutation operators and a dominance-based approach to generate a variety of alignments. Instead of considering a single objective that relates both similarities, MOMEA analyzes the topological and biological similarities as different objectives. The generated results are high-quality and varied non-dominated alignments, i.e., located in a wide area of the solution space. Finally, to demonstrate the effectiveness of this proposal, data from live organisms have been used in order to perform comparisons with several popular biological tools and with the unique existing multi-objective approach in the scientific literature. Comparison results are good and confirm a positive performance of MOMEA compared to all the other tools.

The Effect of Learning Rate on Fractal Image Coding Using Artificial Neural Networks

The amount by which the artificial neural network weights are updated during the training process is called the learning rate. More precisely, the learning rate is an adjustable parameter used in training neural networks in which small values, often in the interval [0, 1], are handled. The learning rate determines how quickly the model updates its weights to adapt to the problem. Smaller learning rates require more training periods due to small changes to the weights per refresh cycle, while larger learning rates lead to faster changes and require fewer training periods. In this paper, the effect of changing the learning rate value in the artificial neural network designed to solve the inverse problem of fractals was studied. Some results were obtained showing the impact of this change, whether when using large values of the learning rate or small values based on the type of fractal shape required to identify the recursive functions that generate it.

Internal quality assessment of mango fruit: an automated grading system with ensemble classifier

ABSTRACT In this work, a non-destructive and automated mango grading system model is developed for grading the mangoes into three categories depending on their internal features like Soluble Solid Content (SSC) as well as Total Acid Content (TAC). Initially, the prepared database is de-noised with the proposed adaptive Gaussian noise removal approach. Then, the Gray Level Co Occurrence Matrix (GLCM), Gray Level Run-Length Matrix (GLRM) features, and Near Infrared (NIR) spectroscopy features were extracted. As the curse of dimensionality persists, a patch-based Principal Component Analysis (PCA) model for dimensionality reduction is introduced. Subsequently, the dimensional reduced features are subjected to aproposed ensemble classifier that encompasses: ‘Support Vector Machine (SVM), Random Forest (RF), three Artificial Neural Network (ANN), and K-Nearest Neighbor (KNN)’. The weight of the third ANN is optimally-tuned by a novel improved meta-heuristic model depicting the Lion-Binary crossover mask base Whale Optimization (LBWO) Algorithm.

PD with neuro-adaptive compensation control using the signed power function

This paper presents a neuro-adaptive control scheme dedicated to solve the motion trajectory tracking problem of robot manipulators under uncertain parameters and external disturbances. A two degrees of freedom direct-drive robot manipulator was taken as a case of study. The proposed controller is able to guarantee asymptotic convergence of the position and velocity tracking errors, and the weights of the artificial neural network are bounded. Artificial neural network weights are updated online using filtered error approach, adaptive laws and signed power function. This scheme does not require any offline training. The neuro-adaptive controller is experimentally validated and compared with a classical one-layer neuro-adaptive controller; the proposed scheme obtains better quantitative metrics related with the RMS values of the position and velocity errors, and a good robust behaviour against external disturbances.

인공신경망 가중치의 효과적인 최적화를 위한 하이브리드 메타휴리스틱 알고리즘

Due to a number of local minimums, the gradient descent methods have difficulties in searching for the optimal weights of artificial neural networks (ANNs). To resolve this problem, recently, a number of scholars have conducted studies that utilize metaheuristics to optimize the weights of ANNs. Particularly, the swarm intelligence algorithm which is one of the metaheuristics has shown great potential in previous studies. In this study, we propose the hybrid metaheuristic algorithm that effectively optimizes the weight of ANNs by combining particle swarm optimization (PSO) and grey wolf optimizer (GWO), both of which are swarm intelligence algorithms. In the search process, the proposed algorithm resolves the convergence instability in the validation dataset using the swarm memory that was inspired by the habit of remembering the environment in which the swarm was suitable for survival. Numerical experiments demonstrate the proposed algorithm outperforms the existing swarm intelligence algorithms, stochastic gradient descent, and Adam Optimizer in terms of classification accuracy.

Optimasi Particle Swarm Optimization Pada Peningkatan Prediksi dengan Metode Backpropagation Menggunakan Software RapidMiner

Backpropagation is a method of Artificial Neural Networks that is quite reliable in solving prediction problems (forecasting). However, in its application, this algorithm still has weaknesses such as optimizing the artificial neural network weights to avoid local minimums, the problem of long training times to achieve convergence and the process of determining the right parameters (learning rate and momentum) in the training process. The purpose of this research is to solve this problem by using Particle Swarm Optimization (PSO) which is a simple and reliable optimization algorithm to solve optimization problems. The data source is obtained from the site sumut.bps.go.id. There are 5 network architecture models used in this study, including 2-5-1, 2-7-1, 2-9-1, 2-11-1 and 2-13-1. The results of trials conducted with Rapid Miner software, the best architectural model is the 2-9-1 model with a total RMSE of 0.056 +/- 0.000 in the implementation of Backpropagation, while in the implementation of Backpropagation + particle swarm optimization the amount of RMSE is 0.055 +/- 0.000. The smaller the RMSE (Root Mean Squared Error), the better the model

Validation of a Stewart platform inspection system with an artificial neural network controller

The use of thermographic cameras attached to Stewart platforms may become a valuable approach to build non-destructive inspection systems. However, achieving successful inspections requires positioning the platform that holds the camera very accurately, especially when the camera lacks of autofocus capabilities. Thus, if the distance from the lenses to the inspection surface is in range, sharp quality images may be obtained, otherwise, images may suffer defocusing producing poor inspection results. This work presents a new Artificial Neural Network (ANN) control aimed at improving the accuracy of the Stewart platform that holds a camera, hence improving the quality of the inspection system. The proposed ANN architecture positions the platform more accurately than the vendor provided Proportional Integral (PI) control strategy. The system kinematics and dynamics are analyzed in order to design the ANN model for the system. The developed control scheme is based on the inverse ANN model with an additional integral control. The advantages of this novel approach are that (1) the controller considers the interactions among the actuators and (2) includes a training process that continuously fine-tunes the ANN weight matrix. The proposed control scheme was validated experimentally, and the performance compared with tuned PI control, taken as the reference, for two different inspection trajectories. The results show that the ANN control is much more accurate and efficient than the reference PI (above 40% error reduction in the actuator positioning with the same power consumption), hence performing the inspections successfully.