Artificial Neural Network Learning Process Research Articles

An evolutionary crow search algorithm equipped with interactive memory mechanism to optimize artificial neural network for disease diagnosis

Artificial neural network (ANN) is an information processing paradigm that loosely models the thinking patterns of the human brain with specifications such as real-time learning, self-adaption, and self-organization. The learning process of ANNs is complex and tackles shortcomings such as a slow convergence rate, learning time-consuming, and local minimum trapping, especially when using gradient-based optimization techniques. Although many metaheuristics have been proposed to arm ANN and solve these weaknesses, ANN learning still needs solution quality. Therefore, this study proposes an evolutionary crow search algorithm (ECSA) to optimize the hyperparameters of ANNs for diagnosing chronic diseases. ECSA introduces an evolutionary search strategy, self-adaptive adaptive flight length, and an interactive memory mechanism to alleviate the canonical crow search algorithm's shortcomings. The evolutionary search strategy and self-adaptive flight length provide a meaningful search strategy in which crows effectively explore and exploit problem spaces by maintaining population diversity. During the search process, the interactive memory mechanism records the best solution obtained during optimization. The performance of ECSA was evaluated and compared with well-known metaheuristic algorithms in terms of local and global search abilities, local optima avoidance, and convergence speed towards the promising area. Then, the results were statistically analyzed. Ultimately, the effectiveness of an adaptation of ECSA named ECSA-MLP for optimizing hyperparameters of the multilayer perceptron network for diagnosing diseases including coronavirus, breast cancer, diabetes, cardiovascular, cervical cancer, Parkinson's, mammography, and acute inflammation was compared with state-of-the-art competitor algorithms. The experimental results indicated the superiority of ECSA over competitor algorithms in optimizing the network.

Modeling Moisture Absorption of Flax/Sisal Reinforced Hybrid Biocomposites Using Fick’s and ANN Methods

ABSTRACT Natural fiber composites are being increasingly used in several fields, owing to their considerable cost, weight, and environmental benefits. The objective of this research is to study the effect of water absorption on the behavior of laminated biocomposites using flax (F) and sisal (S) fiber biocomposites with different stacking sequences (3S and 3F) and fiber orientation at 90° (F090s and S090s), as well as hybridizations (F/4S90/F and S/4F90/S) to reinforce an epoxy matrix. The kinetics of water diffusion within the proposed biocomposites was monitored and analyzed through a model-based optimization approach using artificial neural network (ANN) method. The results of this study showed that the amount of water absorbed by different types of biocomposites increased with time according to Fick’s law. Non-hybrid flax and sisal fiber composites (UD and (0/90°)s oriented) had lower water absorption rates than the hybrid orientation. It was also observed that sisal fiber was most sensitive to water absorption than flax. A high correlation of experimental data with the model was revealed which denoted that the ANN learning process was perfectly performed. Therefore, it was inferred that the ANN approach can accurately predict biocomposites water absorption.

Prediction of the Compressive Strength of Waste-Based Concretes Using Artificial Neural Network.

In the 21st century, numerous numerical calculation techniques have been discovered and used in several fields of science and technology. The purpose of this study was to use an artificial neural network (ANN) to forecast the compressive strength of waste-based concretes. The specimens studied include different kinds of mineral additions: metakaolin, silica fume, fly ash, limestone filler, marble waste, recycled aggregates, and ground granulated blast furnace slag. This method is based on the experimental results available for 1303 different mixtures gathered from 22 bibliographic sources for the ANN learning process. Based on a multilayer feedforward neural network model, the data were arranged and prepared to train and test the model. The model consists of 18 inputs following the type of cement, water content, water to binder ratio, replacement ratio, the quantity of superplasticizer, etc. The ANN model was built and applied with MATLAB software using the neural network module. According to the results forecast by the proposed neural network model, the ANN shows a strong capacity for predicting the compressive strength of concrete and is particularly precise with satisfactory accuracy (R² = 0.9888, MAPE = 2.87%).

Response Surface Methodology for Copper Flotation Optimization in Saline Systems

Response surface methodology (RSM) is one of the most effective tools for optimizing processes, and it has been used in conjunction with the Analysis of Variance (ANOVA) test to establish the effect of input factors on output factors. However, when this methodology is used in mineral flotation, its polynomial model usually performs poorly. An alternative is to use artificial neural networks (ANNs) in such situations. Within this context, the ANOVA test is not the best option for these model types; moreover, it requires statistical assumptions that are difficult to satisfy in flotation. This work proposes replacing the polynomial model of the RSM with ANNs and the Sobol methods to determine the influential input factors instead of the ANOVA test. This proposal is applied to two porphyry copper ores with a high content of pyrite, clay, and dilution media. In addition, this study shows how other computational intelligence techniques, such as swarm intelligence, can be incorporated into this type of problem to improve the learning process of ANNs. The results gave an adjustment of over 0.98 for R2 using ANNs, in comparison to values of around 0.5 when the polynomial model of RSM was utilized. On the other hand, the application of Global Sensitivity Analysis (GSA) identified the aeration rate and P80 size as the most influential variables in copper recovery under the conditions studied. Additionally, we identified significant interactions that affect the recovery of copper, with the interactions between the aeration rate, frother concentration, and P80 size being the most important.

Multi-Task Reinforcement Meta-Learning in Neural Networks

Artificial Neural Networks (ANN) is one of the main and widespread tools for creating intelligent systems. And, they are actively used for data analysis in many areas such as robotics, computer vision, natural language processing, etc. The learning process of ANN is one of the most labor-intensive stages in ANN. There are many different modifications of ANNs and methods for their training. Currently, deep neural networks are becoming one of the most popular methods of machine learning due to their effectiveness in areas such as speech recognition, medical informatics, computer vision, etc. It is known that ANN training depends on the type of input data. In this paper, reinforcement learning is considered, as popular method used in cases where information is reinforced by signals from the external environment with which the model interacts. The purpose of this paper is to develop a reinforcement meta-learning algorithm that would be efficient in terms of quality and speed of learning. However, despite the significant scientific progress in deep learning, existing algorithms are not efficient enough to solve problems in the real world. In addition, such algorithms require a significant amount of learning time, which complicates the development process. To solve these problems, the use of meta-learning or “learning to learn” algorithms has recently been especially relevant. The paper proposes an approach to reinforcement meta-learning using a multitasking weight optimizer. experimentally shown that the proposed approach is more efficient than the known MAML (Model-Agnostic Meta-Learning) algorithm. The proposed MAML SPSA-Track method shows an improvement in efficiency by an average of 4%, and MAML SPSA-Delta by 8%, respectively. Moreover, the last algorithm spends on average 2 times less time on push-v2 and pick-place-v2 tasks.

(Invited) Random Network Structure of Materials Developing Next Generation AI Devices for Autonomous Robotics

1. IntroductionIn recent years, the superior computational power of deep learning based on software has been widely recognized, and the practical applications of artificial intelligence are rapidly expanding. On the other hand, the hardware for replacing to such artificial intelligence (AI) algorithms is facing the physical limits of scaling in silicon CMOS technology, and performance improvement is expected to hit the ceiling. For this reason, there is a growing interest in hardware technologies that physically implement artificial neural networks (ANNs), neuromorphic or brainmorphic information processing systems, and the applications (hereafter referred as AI systems in this paper), as well as new materials and devices. A critical difference between the presently required device functionality and that in conventional computational systems is the use of dynamics. By cleverly using nanomaterials' nonlinearity and network structure, devices that spontaneously generate pulses, noise, and other physical phenomena are expected to be realized to utilize for the AI hardware. These devices will enable drastically lower power consumption and higher integration of AI systems. In the learning process of ANNs, it is necessary to constantly change and store the weights of the weighted sum (sum-of-products) part. In our research center, we have been working on materials that can complement CMOS for AI systems by using molecules and nanocarbon materials, and further, we are trying to apply them to autonomous AI robots. This paper introduces these nanomaterials and networks’ formation as devices, the key points of the devices’ functionalization, application to robots, and other recent research results.2. Formation and Control of NanomaterialsOrganic and inorganic nanosized molecules can be synthesized to form structures by the current development of synthesis technology; a great variety of molecules can be realized and controlled for a high potential to use as neuromorphic devices. In order to elicit electronic functionalities, it is necessary to transfer electrons to and from the molecules, which is a redox reaction from a chemical point of view. Stable Carbon-based materials that have a close Fermi level to the molecules may be more suitable than metallic electrodes. We have connected nanoparticles of organic molecules to single-walled carbon nanotubes (SWNTs) to control the electrical properties of the nanoscale [1].3. Results obtained from Random NetworksWe formed a random network consisting of SWNTs and POMs and found that it generates neuron-like pulses when a high voltage is applied (Fig. 1, left) [2]. This enabled us to control noise and pulses generation. Such dynamic signal generation is attributed to the multiple discharges of POM molecules. Reservoir computing simulations also showed that this system could be used for time series memory [2].In this study, we applied a sinusoidal signal to an SWNT/Por-POM random network using porphyrin-gradient sandwich POM (Por-POM), which is more redox-sensitive, and the Lissajous plot of the output signal from one of the multiple electrodes plotted against the input signal showed nonlinear switching behavior. A memory state was generated due to the reversible redox of Por-POM, and the input signal was highly interacted with the reservoir system. The interaction can be also checked by the changing of the Lissajous plot shape from linear to elliptical, which indicate the characteristics obtained from current and past inputs through the echo state network. The dynamics are also confirmed by the higher harmonic generation (HHG) from the FFT analysis, and NARMA-2 task which considers states up to a two-step delay. The NARMA-2 task successfully generated signals that predictably follow the target waveform using a linear regression model. These results indicate that the nonlinearity of SWNT/Por-POM and memory integrity are important factors for reservoir operation. We also used Ag/Ag2S core-shell nanoparticles (NP) as a reservoir device (Fig.1, right) to obtain similar results with SWNT/Por-POM.As mentioned above, large power consumption during the sum-of-products operation in the weighted addition part has been a problem in AI systems. While, by using reservoir computing devices consist of random networks of SWNT/POM, core-shell nanoparticles, nanowires, etc., it is expected that the sum-of-products operation can reduce power consumption. Voice recognition and robot-hand grabbing stuff recognition [3] was conducted by the reservoir devices. Details will be presented at the presentation.Refs. [1] H. Tanaka et al., Adv. Mat. 18, 1411 (2006). [2] H. Tanaka et al., Nat. Commun. 9, 2693 (2018). The article was selected as the most read 50 articles published in Nat. Commun. in 2018 (Physics). [3] Kyutech won the RoboCup world series of Domestic Standard Platform League by TOYOTA HSR in 2017 and 2018. The same robot is used for the demonstration. Figure 1

ВИБІР ОПТИМАЛЬНОЇ СТРУКТУРИ ПРИХОВАНИХ ШАРІВ ШТУЧНОЇ НЕЙРОННОЇ МЕРЕЖІ ДЛЯ АНАЛІЗУ ЕФЕКТИВНОСТІ ЕНЕРГОСПОЖИВАННЯ

A method for optimal structure selection of hidden layers of the artificial neural network (ANN) is proposed. Its main idea is the practical application of several internal structures of ANN and further calculation of the error of each hidden layer structure using identical data sets for ANN training. The method is based on the alternate comparison of the expected result values and the actual results of the feedforward artificial neural networks with a different number of inner layers and a different number of neurons on each layer. The method afforces searching the optimal internal structure of ANN for usage in the development of "smart" house systems and for calculation of the optimal energy consumption level in accordance with current conditions, such as room temperature, presence of people, and time of the day. The usage of the presented method allows to reduce the time spent on choosing the effective structure of the artificial neural network at the initial stages of research and to pay more attention to the relationship between the input and output data, as well as to such important parameters of the ANN learning process, as a number of training iterations, minimal training error, etc. The software has been developed that allows to carry out the processes of training, testing, and obtaining the output results of the algorithm of the artificial neural network, such as the expected value of power consumption and operating time of each individual appliance. The disadvantage of the approach used in finding the optimal internal structure of the artificial neural network is that each subsequent structure is created on the basis of the most efficient of the previously created structures without analyzing other structures that showed worse results with fewer hidden layers. It was found that to improve the solution of this problem it is necessary to create a mechanism which will be based on the analysis of input data, output data, will analyze the internal relationships between parameters and will optimize the network structure at each stage using certain logical rules according to the results obtained in the previous step. It is established that this problem is a nonlinear programming problem that can be solved in the further development of this study.

Short Term Load Forecasting Using Bootstrap Aggregating Based Ensemble Artificial Neural Network

Background: Short Term Load Forecasting (STLF) can predict load from several minutes to week plays a vital role to address challenges such as optimal generation, economic scheduling, dispatching and contingency analysis. Methods: This paper uses Multi-Layer Perceptron (MLP) Artificial Neural Network (ANN) technique to perform STFL but long training time and convergence issues caused by bias, variance and less generalization ability, make this algorithm unable to accurately predict future loads. Results: This issue can be resolved by various methods of Bootstraps Aggregating (Bagging) (like disjoint partitions, small bags, replica small bags and disjoint bags) which help in reducing variance and increasing generalization ability of ANN. Moreover, it results in reducing error in the learning process of ANN. Disjoint partition proves to be the most accurate Bagging method and combining outputs of this method by taking mean improves the overall performance. Conclusion: This method of combining several predictors known as Ensemble Artificial Neural Network (EANN) outperforms the ANN and Bagging method by further increasing the generalization ability and STLF accuracy.

Hybrid artificial neural network and cooperation search algorithm for nonlinear river flow time series forecasting in humid and semi-humid regions

Accurate river flow forecasting is of great importance for the scientific management of water resources system. With the advantages of easy implementation and high flexibility, artificial neural network (ANN) has been widely employed to address the complex hydrological forecasting problem. However, the conventional ANN method often suffers from some defects in practice, like slow convergence and local minimum. In order to enhance the ANN performance, this study proposes a hybrid river flow forecasting method by integrating the novel cooperation search algorithm (CSA) into the learning process of ANN. In other words, the computational parameters of the ANN network (like threshold and linking weights) are iteratively optimized by the CSA method in the feasible state space. The proposed method is applied to the river flow data collected from two real-world hydrological stations in China. Several Quantitative indexes are chosen to compare the performance of the developed models, while the comprehensive analysis between the simulated and observed flow data are conducted. The experimental results show that in different scenarios, the hybrid method based on ANN and CSA always outperforms the control models and yields superior forecasting results during both training and testing phases. In Three Gorges Project, the presented method makes 11.10% and 5.42% improvements in the Nash–Sutcliffe efficiency and Coefficient correlation values of the standard ANN method in the testing phase. Thus, this interesting finding shows that the performance of the artificial intelligence models in river flow time series forecasting can be effectively improved by metaheuristic algorithm with outstanding global search ability.

Computational Fluid Dynamics Based Mixing Prediction for Tilt Pad Journal Bearing TEHD Modeling—Part II: Implementation With Machine Learning

Abstract Reynolds based thermo-elasto-hydrodynamic (TEHD) simulations of tilting pad journal bearings (TPJBs) generally provide accurate results; however, the uncertainty of the pad’s leading edge thermal boundary conditions causes uncertainty of the results. The highly complex thermal-flow mixing action between pads (BPs) results from the oil supply nozzle jets and geometric features. The conventional Reynolds approach employs mixing coefficients (MCs), estimated from experience, to approximate a uniform inlet temperature for each pad. Part I utilized complex computational fluid dynamics (CFD) flow modeling to illustrate that temperature distributions at the pad inlets may deviate strongly from being uniform. The present work retains the uniform MC model but obtains the MC from detailed three-dimensional CFD modeling and machine learning, which could be extended to the radially and axially varying MC case. The steps for implementing an artificial neural network (ANN) approach for MC regression are provided as follows: (1) utilize a design of experiment step for obtaining an adaptable training set, (2) conduct CFD simulations on the BP to obtain the outputs of the training set, (3) apply an ANN learning process by Levenverg–Mardquart backpropagation with the Bayesian regularization, and (4) couple the ANN MC results with conventional TEHD Reynolds models. An approximate log fitting method provides a simplified approach for MC regression. The effectiveness of the Reynolds TEHD TPJB model with ANN regression-based MC distributions is confirmed by comparison with CFD based TEHD TPJB model results. The method obtains an accuracy nearly the same as the complete CFD model, but with the computational economy of a Reynolds approach.

Analysis of training parameters in the ANN learning process to mapping the concrete carbonation depth

We propose an artificial neural network (ANN) model to predict the CO2 diffusion through the concrete to determine the carbonation depth over time, analyzing the influence of some training algorithm and the network architecture in the ANN learning process. A reliable experimental test database of the non-accelerated test with 278 results of concrete carbonation depth was created from the published literature. It was used to train, test, and validate the model. Altogether, 120 networks had been trained with different characteristics, verifying its performance. In spite of the non-linearity and complexity of the concrete carbonation phenomenon, the proposed ANN model yielded accurate prediction. Results indicate the best training algorithm and the optimum number of neurons in the hidden layer that allows faster ANN training process and generates the most accurate mapping for the concrete carbonation phenomenon. The use of ANN appears as a robust tool easily applied to the study of the concrete carbonation, aiding in decision making in engineering projects focused on durability.

Optimal power flow control of hybrid renewable energy system with energy storage: A WOANN strategy

This paper proposed an optimal control technique for power flow control of hybrid renewable energy systems (HRESs) like a combined photovoltaic and wind turbine system with energy storage. The proposed optimal control technique is the joined execution of both the whale optimization algorithm (WOA) and the artificial neural network (ANN). Here, the ANN learning process has been enhanced by utilizing the WOA optimization process with respect to the minimum error objective function and named as WOANN. The proposed WOANN predicts the required control gain parameters of the HRES to maintain the power flow, based on the active and reactive power variation in the load side. To predict the control gain parameters, the proposed technique considers power balance constraints like renewable energy source accessibility, storage element state of charge, and load side power demand. By using the proposed technique, power flow variations between the source side and the load side and the operational cost of HRES in light of weekly and daily prediction grid electricity prices have been minimized. The proposed technique is implemented in the MATLAB/Simulink working stage, and the effectiveness is analyzed via the comparison analysis using the existing techniques.

Short-term Load Forecasting based on ABC and ANN for Smart Grids

Short term load forecasting is a subject about estimating future electricity consumption for a time interval from one hour to one week and it has a vital importance for the operation of a power system and smart grids. This process is mandatory for distribution companies and big electricity consumers, especially in liberalized energy markets. Electricity generation plans are made according to the amount of electricity consumption forecasts. If the forecast is overestimated, it leads to the start-up of too many units supplying an unnecessary level of reserve, therefore the production cost is increased. On the contrary if the forecast is underestimated, it may result in a risky operation and consequently power outages can occur at the power system. In this study, a hybrid method based on the combination of Artificial Bee Colony (ABC) and Artificial Neural Network (ANN) is developed for short term load forecasting. ABC algorithm is used in ANN learning process and it optimizes the neuron connections weights of ANN. Historical load, temperature difference and season are selected as model inputs. While three years hourly data is selected as training data, one year hourly data is selected as testing data. The results show that the application of this hybrid system produce forecast values close to the actual values.

Identifikasi Jenis Asap Menggunakan Spektrofotometer Dan Jaringan Syaraf Tiruan

Spectroscopy method can be used to identify the type of gas by passing light into the sample gas and light are described using a monochromator. The resulting light captured using the detector that will produce different spectra for each gas. The light source can use incandescent lamps or using Light Emitting Diode (LED). If the tested gas is a type of gas that is difficult to be taken, such as the type of toxic gas, gas from the output of the volcano, so this way you will have trouble, and the data obtained is not real time. In this study designed a system of identification of gas or smoke in the air using a spectrophotometer using solar light. The spectrum of light that has been absorbed by the gas or fumes captured using a telescope, then described using a monochromator produces gray level curve representing the absorption of each wavelength of light. The sample used in this study is the smoke of burning oil, sulfur and dry leaves. The curve of each sample generated, analyzed and identified the type of smoke using Artificial Neural Network (ANN) with backpropagation training algorithm. The architecture consists of input layer, hidden and output. Input node number is 197, the number of neurons in the first hidden layer is 150, the number of neurons in the second hidden layer is 20, and the number of neurons in the output layer is 4. In this ANN learning process entails iterating as many as 900 epoch and the results can distinguish the smoke oil , sulfur and dry leaves. The success rate of 70% for oil and dried leaves, 80% for sulfur.

Development of an ANN-Based Pressure Transducer

This paper presents the development of an artificial neural network (ANN)-based improved inductive signal conditioning circuit for pressure transducer with bellow as sensor. A bellow is an elastic-type mechanical pressure sensor. The deflection of the bellow due to applied pressure must be translated into an efficient electrical signal for continuous monitoring, wireless transmission, and digital readout. A ferromagnetic wire attached to the bellow as a part of an inductive coil arrangement, gets deflected due to applied pressure, thereby changing the self inductance of the coil. An op-amp inductive signal conditioning circuit (OISCC) is designed to produce voltage proportional to changes in self-inductance, but the OISCC voltage versus applied pressure exhibits a considerable nonlinearity error due to stray inductances and component drifts. The ANN modeling estimates and compensates the nonlinearity of OISCC. An embedded unit is used for implementation of ANN learning process. The pressure transducer with significant stability has exhibited high linearity and sensitivity of ±0.35% and 10 mH/psig, respectively, in the measuring range of 0–70 psig. The design and experimental aspects of the technique are reported.

Artificial neural network based on genetic learning for machining of polyetheretherketone composite materials

In this paper an artificial neural network (ANN) aiming for the efficient modelling of a set of machining conditions for orthogonal cutting of polyetheretherketone (PEEK) composite materials is presented. The supervised learning of the ANN is based on a genetic algorithm (GA) supported by an elitist strategy. Input, hidden and output layers model the topology of the ANN. The weights of the synapses and the biases for hidden and output nodes are used as design variables in the ANN learning process. Considering a set of experimental data, the mean relative error between experimental and numerical results is used to monitor the learning process obtaining the completeness of the machining process modelling. Also a regularization term associated to biases in hidden and output neurons are included in the GA fitness function for learning. Using a different set of experimental results, the optimal ANN obtained after learning is tested. The optimal number of nodes on the hidden layer is searched and the positive influence of the regularization term is demonstrated. This approach of ANN learning based on GA presents low mean relative errors in learning and testing phases.

NEURAL NETWORK APPROACH TO DYNAMIC VOLTAGE STABILITY PREDICTION

ABSTRACT The application of artificial neural network (ANN) on dynamic voltage stability analysis is presented. Two ANN models have been utilized, in which the first ANN model is used to classify the power system as to whether it is dynamically stable or unstable. Then the second ANN model is used for the dynamically stable system to predict the voltage magnitudes at load busbars. Both ANN models are based on the multiperceptron model, and the training is done using the error back propagation scheme. The training set patterns are generated by carrying out dynamic simulations, using induction motor and constant P-Q load models. This paper highlights the method for selection of the optimum number of training sets so as to minimise the time taken in the ANN learning process. The performance of the ANN models have been tested and shown to give good results.

What role can neural networks play in power system engineering

This article discusses the application of the layered perceptron form of artificial neural network (ANN) to solving problems in the field of energy production. The topics of the article include the ANN learning process, problems with back error propagation, scaling, diminished learning, and neural network application to power systems in areas of regression, classification, and combinatorial optimization.