ANN Training Research Articles

A Cascaded Duplex Organic Vertical Memory with Learning Rate Scheduling for Efficient Artificial Neural Network Training

AbstractLearning rate scheduling (LRS) is a critical factor influencing the performance of neural networks by accelerating the convergence of learning algorithms and enhancing the generalization capabilities. The escalating computational demands in artificial intelligence (AI) necessitate advanced hardware solutions capable of supporting neural network training with LRS. This not only requires linear and symmetric analog programming capabilities but also the precise adjustment of channel conductance to achieve tunable slope in weight update behaviors. Here, a cascaded duplex organic vertical memory is proposed with the coupling of ferroelectric polarization effect and Schottky gate control on the same semiconducting channel, exhibiting adjustable‐slope conductance update with high linearity and symmetry. Therefore, in the chest X‐ray image detection, a fast‐to‐slow LRS is used for a bi‐layer ANN training, achieving a rapid, stable convergence behavior within only 15 epochs and a high recognition accuracy. Moreover, the proposed LRS training is also suitable for the Mackey Glass prediction task using long short‐term memory networks. This work integrates LRS into synaptic devices, enabling efficient hardware implementation of neural networks and thus enhancing AI performance in practical applications.

Jaringan Saraf Tiruan untuk Memprediksi Jumlah Pertumbuhan Penduduk di Dinas Kependudukan dan Pencatatan Sipil Kabupate Sumba Barat Daya

The purpose of this research is to create a JST (artificial neural network) model that can forecast population growth at the Population and Civil Registration Office of West Sumba Regency. population growth at the Population and Civil Registration Office of West Sumba Regency. Regency. Regional development planning must consider the increasing number of population, therefore proper forecasting is essential to encourage sustainable policies and initiatives. sustainable policies and initiatives. Because it can identify complex patterns in past data and produce more accurate forecasts than traditional techniques, an ANN model is used. traditional techniques, the ANN model is used. The data used in this study is the population growth of Southwest Sumba Regency over the past including characteristics such as birth and death rates and population movements. deaths and population movements. The backpropagation algorithm is used to optimize the multilayer perceptron (MLP) architecture for ANN training. Separating the data into training and testing sets and assessing the models model using metrics such as Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) based on the error. Error (RMSE) based on the prediction error are the steps involved in the training process. involved in the training process. The research findings show that, with a low level of error, the artificial neural network model can estimate the population increase in Southwest Sumba Regency with a reasonable level of accuracy. reasonable level of accuracy. The model is expected to serve as a reference for relevant authorities to better manage population data and as a tool to create more focused and successful population policies.

Study the effect of ANN splitting ratios and training functions on the prediction of corroded steel-to-concrete bond strength

Study the effect of ANN splitting ratios and training functions on the prediction of corroded steel-to-concrete bond strength

Numerical simulation, ANN training and predictive analysis of phase change material with 3D printing lattice structures

This study simulated and analysed the performance of phase change material with three lattice structures, i.e., simple cubic, body-centered cubic, and face-centered cubic, at various porosities and heat fluxes. Three parameters are analysed, including the maximum temperature of the heated wall, melting and solidifying times of PCM. The findings suggest that as the porosity rises, the maximum temperature decreases, while the increase in heat flux leads to a higher maximum temperature. Both the melting and solidifying times extend as porosity increases. An artificial neural network trained by the Levenberg-Marquardt algorithm is used to predict these three parameters. The lattice structure type, porosity, and heat flux are established as the input parameters for the network. The ANN predictions demonstrated outstanding performance in estimating these parameters with a minimum MSE of 0.00015 and a maximum R of 0.99899. The trained ANN is used to predict the crucial parameters of PCM with 82% SC, BCC, and FCC lattice structures. An excellent agreement is observed between the ANN predicted results and the simulation outcomes with a maximum relative error of 9.43%.

A Hybrid Technique for Fault Classification and Location in a Jointed Overhead-Underground Distribution Line

The electrical distribution system is crucial for the utility grid to transmit power from generators to consumers. Considering the intricate structure of distribution systems and their significant role in power networks, establishing a robust fault classification and location scheme is vital. Due to ageing, distribution systems are often prone to faults from factors like poor operational conditions and wear and tear. The line faulting rate and the pertinent restoration epochs influence the frequency and duration of power disruptions. Thus, precisely locating the fault section is essential to minimize power restoration timeframes. This paper presents a hybrid fault classification and location technique in a combined continuous overhead and underground distribution line. A simulation of the hybrid model was designed in Simulink for an 11 kV combined continuous overhead and underground electrical distribution line, considering short circuit faults as they are the most predominant and cause massive damage in distributed systems. The proposed technique first classifies the fault using Discrete Wavelet Transforms (DWT) and Multi-layer Perceptron-Artificial Neural Networks (MLP-ANN). Next, the impedance and Adaptive Neuro-Fuzzy System (ANFIS) based technique is employed for fault location. At a sample rate of 50 kHz, the DWT was applied to current signals and the coefficients used for ANN training, while phase impedance values were used as input to the ANFIS for training. The simulation results showed accuracy of 96.6% for fault classification and 99.17% for fault location. The developed models can significantly enhance fault location for speedier outage resolution by promptly repairing the affected distribution lines.

Application of an artificial neural network coupled to a genetic algorithm for the production of polyphenols in Parachlorella kessleri grown under mixotrophic conditions

Microalgae are vital sources of high-value compounds, like polyphenols. Boosting polyphenol production requires considering diverse factors and their impacts on synthesis. This study aims to optimise polyphenol yield in microalga Parachlorella kessleri using an artificial neural network coupled with a genetic algorithm (ANN-GA). P. kessleri was grown under mixotrophic conditions and subjected to 13 experimental treatments. Factors studied included glucose, NaNO3, KH2PO4, LED light wavelength, intensity, and photoperiod. Total polyphenol concentration was quantified with Folin-Ciocalteu reagent; the polyphenol profile was obtained via HPLC. A database was constructed for ANN training to predict polyphenol production. Developed models were compared using indicators: coefficient of determination (R2pred), Mean Absolute Error (MAE), and Mean Square Error (MSE). Optimal models were fine-tuned with a genetic algorithm and validated using randomised experiments. The optimal ANN architecture for P. kessleri polyphenol production is: 6 inputs, 4 hidden layers, 6 neurons per layer, one output. This design achieved high prediction accuracy (R2 = 0.93). The genetically algorithm-optimised ANN model has these conditions: 26.60 g/L glucose; 1.42 g/L NaNO3; 0.59 g/L KH2PO4; white light; 1000 lx light intensity; 12:12 photoperiod. In experimental validation, 28 mg gallic acid equivalents per gram dry weight (GAE/g DW) were obtained, surpassing all models. An ANN-GA optimisation procedure was used for the first time for polyphenol production in P. kessleri. The model under mixotrophic conditions showed higher polyphenol production than autotrophic conditions. Polyphenol composition in P. kessleri varied between autotrophic and mixotrophic cultures, indicating inducible polyphenol production. The ANN-GA approach for polyphenol production in P. kessleri proved effective, with potential for maximising other high-value microalgae metabolites.

Models and methods of learning neural networks with differentiated activation functions

Analysis of the literature made it clear that the problem associated with improving the performance and acceleration of ANN learning is quite actual, as ANNs are used every day in more and more industries. The concepts of finding more profitable activation functions have been outlined a lot, but changing their behavior as a result of learning is a fresh look at the problem. The aim of the study is to find new models of optimization tasks for the formulated prob-lem and effective methods for their implementation, which would improve the quality of ANN training, in particular by overcoming the problem of local minima. A studied of models and methods for training neural networks using an extended vector of varying parameters is conducted. The training problem is formulated as a continuous mul-tidimensional unconditional optimization problem. The extended vector of varying parameters implies that it includes some parameters of activation functions in addition to weight coeffi-cients. The introduction of additional varying parameters does not change the architecture of a neural network, but makes it impossible to use the back propagation method. A number of gradient methods have been used to solve optimization problems. Different formulations of optimization problems and methods for their solution have been investigated according to ac-curacy and efficiency criteria. The analysis of the results of numerical experiments allowed us to conclude that it is expedient to expand the vector of varying parameters in the tasks of training ANNs with con-tinuous and differentiated activation functions. Despite the increase in the dimensionality of the optimization problem, the efficiency of the new formulation is higher than the generalized one. According to the authors, this is due to the fact that a significant share of computational costs in the generalized formulation falls on attempts to leave the neighborhood of local min-ima, while increasing the dimensionality of the solution space allows this to be done with much lower costs.

Implementasi Kontrol Nutrisi Dan pH Pada Hidroponik Cerdas Berbasis Arduino Dan JST

This research aims to implement an automated nutrition and pH control system in NFT hydroponic system based on ANN control. NFT hydroponics involves growing plants without soil as a medium. In hydroponics, it is essential to continuously control the nutrient levels and pH of the solution. However, manual control performed by humans continuously is inefficient and time-consuming.The ANN method is used to model and predict the output actuators based on sensor input in the NFT hydroponic system. This ANN architecture consists of several layers with the following number of neurons: input layer 2, first hidden layer 128, second hidden layer 64, and output layer 3, representing multipleoutputs. The ANN training process involves classifying the data samples using various hyperparameters.The research findings demonstrate the ANN classification model successfully applied to control pH and nutrient levels through the predicted output actuators. The pump actuators are activated according to input received from the TDS and pH sensors. Through the variation of hyperparameters, the classification model with a test_size: 0.3, epoch: 400, batch_size: 32, and random_state: 42 provided the best performance in prediction. This ANN classification model achieved the best results in model testing with an accuracy rate: 97.96% from 49 data.

A conformable artificial neural network model to improve the void fraction prediction in helical heat exchangers

This study proposes a conformable artificial neural network model to improve the void fraction prediction in helical heat exchangers. The obtained model had only one neuron in the hidden layer, achieving an algebraic structure simpler than the classic training. Furthermore, this model satisfies the interval condition of [0–1] and is a function of vapor fraction, density ratio, and viscosity ratio. The new conformable ANN void fraction satisfactorily described the two-phase flow in the systems mentioned above because the outlet temperatures were predicted with 2.96% of RMSE, lower than those obtained with other void fraction model analyses. This paper describes the conformable Logistic Sigmoid Transfer Function (CLOGSIG) and its application advantages in the ANN training process. Using CLOGSIG as a transfer function can get different sinusoidal behaviors that can modify the data distribution around the function's sinusoidal area, allowing better data adaptability and improving network performance.

Memristive Neural Networks for Predicting Seizure Activity.

The biological part of the investigation. Young healthy outbred CD1 mice were used in our study. They were divided into two groups: control (n=6) and the group with induced chronic epileptiform activity (n=6). Local field potentials (LFP) were recorded from the hippocampus and medial entorhinal cortex of the mice of both groups to register neuronal activity. The LFP recordings were used for deep ANN training. Epileptiform activity in mice was modeled by intraperitoneal injection of pilocarpine (280 mg/kg). LFP were recorded in the awake mice a month after the induction of epileptiform activity.Mathematical part of the investigation. A deep long short-term memory (LSTM) ANN capable of predicting biological signals of neuronal activity in mice has been developed. The ANN implementation is based on memristive devices, which are described by the equations of the redox processes running in the memristive thin metal-oxide-metal films, e.g., Au/ZrO2(Y)/TiN/Ti and Au/SiO2(Y)/TiN/Ti. In order to train the developed ANN to predict epileptiform activity, a supervised learning algorithm was used, which allowed us to adjust the network parameters and train LSTM on the described recordings of neuronal activity. After training on the LFP recordings from the hippocampus and medial entorhinal cortex of the mice with chronic epileptiform activity, the proposed deep ANN has demonstrated high values of evaluation metric (root-mean-square error, RMSE) and successfully predicted epileptiform activity shortly before its occurrence (40 ms). The results of the numerical experiments have shown that the RMSE value of 0.019 was reached, which indicates the efficacy of proposed approach. The accuracy of epileptiform activity prediction 40 ms before its occurrence is a significant result and shows the potential of the developed neural network architecture. The proposed deep ANN can be used to predict pathological neuronal activity including epileptic seizure (focal) activity in mice before its actual occurrence. Besides, it can be applied for building a long-term prognosis of the disease course based on the LFP data. Thus, the proposed ANN based on memristive devices represents a novel approach to the prediction and analysis of pathological neuronal activity possessing a potential for improving the diagnosis and prognostication of epileptic seizures and other diseases associated with neuronal activity.

Framework for rapid characterization of fresh properties of cementitious materials using point cloud and machine learning

This study presents a framework that utilizes point cloud analysis and machine learning to automate and accelerate the characterization of fresh properties of cementitious materials. The framework collects point cloud data using a depth camera and extracts diameter, height, and curvature information through post-processing techniques. Data augmentation technique is used to generate new data for ANN training based on nonlinear correlations between these parameters and experimentally determined fresh properties. The developed framework is validated through additional experimental results and shows high prediction accuracy, offering a rapid and effective approach for characterizing fresh properties of cementitious materials.

CFD simulation and optimization study on the shell side performances of a plate and shell heat exchanger with double herringbone plates

The purpose of this study is to investigate the flow and heat transfer performance of the plate and shell heat exchanger (PSHE) with double herringbone plates under low temperature difference conditions, while introducing the CFD-ANN-NSGA II method to optimize the corrugated parameters. A PIV experimental bench is built to validate the flow field distribution in the shell side, and the average velocity error between simulation results and experiment results is 2.93%, with a maximum relative error of 13.24%, confirming the accuracy of the turbulence model and numerical method. The flow characteristics analysis and parameter study are conducted within the research scope. For the considered parameters, i.e., the corrugation angle β, corrugation depth b, and corrugation pitch λ, the maximum heat transfer efficiency is achieved when β is 60°. Moreover, the corrugation angle has the greatest impact on both pressure drop and heat transfer performance. β = 30°, λ = 7.4 mm, and b = 7.5 mm are the best performance indicators for the respective operating conditions. Using CFD results as samples, the R2 of ANN training is greater than 0.99, and the multi-objective optimization yields the best configuration of β, λ, and b, which are 33.1°, 11.8 mm, and 7.5 mm, respectively, with the optimal thermal-hydraulic performance.

A competitive learning scheme for deep neural network pattern classifier training

To reduce the computational complexity of training a deep neural network architecture using large data sets of 3D scenes, a competitive learning scheme was devised. The proposed algorithm pits a neural network learning algorithm, in this case the standard Adam optimiser, against an evolutionary algorithm that is used to select the most difficult training examples. The overall scheme is similar to a predator–prey system, where the predator (the neural network) strives to optimise its ability to capture (identify) the prey (the training patterns), and the evolutionary procedure selects the prey that so far evaded capture. As a consequence of the evolutionary process, the neural network is presented only a fraction of the training examples, and the computational complexity of the learning procedure is reduced. Experimental evidence showed that the proposed scheme allows reducing the deep neural network training time on different model sets, sometimes significantly, without affecting the recognition accuracy. The proposed predator–prey scheme is fairly independent of the ANN type and training algorithm employed, and has the potential to be beneficial to a wide range of deep learning applications, where practical implementations are often hindered by the time complexity of the training process.

Artificial-neural-network-based optimal Smoother design for oscillation suppression control of underactuated overhead cranes with distributed mass beams

As a kind of convenient and practical oscillation suppression method, the open-loop controllers are widely utilized in the control of actual cranes, yet most of them need to be designed for each mode frequency when targeting multimodal systems, which will degrade the robustness of the controller. Moreover, most open-loop controllers require linearization of the model in the design process, which will inevitably reduce the performance of the controller in terms of oscillation suppression. To solve the above prevalent problems, an optimal command-smoothing method is proposed in this paper. Specifically, the dynamic analysis of a 2-D overhead crane with distributed-mass beams (DMB) is first performed. Then, the particle swarm optimization algorithm (PSO) is used to solve the optimal values of the parameters of the Smoother under different system parameters, and a data set for ANN training is generated, together with a series of indices to verify the effectiveness of the designed ANN. Finally, in combination with the PD controller, the proposed controller is experimentally proven to accomplish the positioning requirements while improving the oscillation suppression efficiency by an average of 29% compared to the basic Smoother.

Indoor WiFi-Beacon Dataset Construction Using Autonomous Low-Cost Robot for 3D Location Estimation

Datasets used for artificial-neural-network and machine-learning applications play a vital role in the research and application of such techniques in solving real-life problems. The construction and availability of large datasets to be used in the off-line phase of ANN training is usually a crucial and time-consuming step towards system construction. In this work, a framework for autonomous construction of a diverse, extensive, and open dataset* with built-in redundancy is demonstrated. As part of the framework, a low-cost robot using off-the-shelf components is built that constructs the dataset autonomously. The robot includes a controller network with multiple WiFi-transceiver nodes for collecting received-signal-strength indicators (RSSIs) at various elevation points throughout the building. All nodes are configured with direct internet access to streamline the data collection towards an online database that is constructed as part of this framework. Preliminary validation and analysis of the dataset are discussed, and an exploration of the application domain of the dataset is carried out. Moreover, this paper investigates the effect of the height of the hand-held mobile WiFi antenna attached to the robot on the received power strength of the WiFi signal.

Structural stress prediction based on deformations using artificial neural networks trained with artificial noise

AbstractArtificial Neural Networks (ANN) have found application for multiple problems in structural mechanics and civil engineering. In the new approach developed here, ANNs are used for the determination of the maximum stress resultants in a structure on the basis of monitored displacements. Initially, a simple supported beam subjected up to two vertical forces is considered. The beam is solved analytically for different combinations of load positions and magnitudes defined by Monte‐Carlo sampling. The resulting beam deflections and maximum stress resultants are employed for the training of feedforward neural networks in order to generate metamodels able to predict the maximum bending moment knowing only a few beam input deflections. Subsequently, the concept is applied to a real beam experiment, where a reinforced concrete beam has been tested. It is shown, that the perfect correlation of the beam displacements used as inputs for the ANN training leads to extrapolations when the metamodel is applied to the measured displacements of the real experiment, which are not perfectly correlated. This is source of wrong predictions when the network is applied to these data. The introduction of artificial noise in the synthetic input data in a controlled way before training helps to overcome extrapolation and to increase repeatability and robustness of the metamodels. Various noise levels are applied to the synthetic data to monitor how the ANN performances change over increasing noise and the results along with the limitations of the approach are illustrated.

Comparing artificial neural network training algorithms to predict length of stay in hospitalized patients with COVID-19.

The exponential spread of coronavirus disease 2019 (COVID-19) causes unexpected economic burdens to worldwide health systems with severe shortages in hospital resources (beds, staff, equipment). Managing patients' length of stay (LOS) to optimize clinical care and utilization of hospital resources is very challenging. Projecting the future demand requires reliable prediction of patients' LOS, which can be beneficial for taking appropriate actions. Therefore, the purpose of this research is to develop and validate models using a multilayer perceptron-artificial neural network (MLP-ANN) algorithm based on the best training algorithm for predicting COVID-19 patients' hospital LOS. Using a single-center registry, the records of 1225 laboratory-confirmed COVID-19 hospitalized cases from February 9, 2020 to December 20, 2020 were analyzed. In this study, first, the correlation coefficient technique was developed to determine the most significant variables as the input of the ANN models. Only variables with a correlation coefficient at a P-value < 0.2 were used in model construction. Then, the prediction models were developed based on 12 training algorithms according to full and selected feature datasets (90% of the training, with 10% used for model validation). Afterward, the root mean square error (RMSE) was used to assess the models' performance in order to select the best ANN training algorithm. Finally, a total of 343 patients were used for the external validation of the models. After implementing feature selection, a total of 20 variables were determined as the contributing factors to COVID-19 patients' LOS in order to build the models. The conducted experiments indicated that the best performance belongs to a neural network with 20 and 10 neurons in the hidden layer of the Bayesian regularization (BR) training algorithm for whole and selected features with an RMSE of 1.6213 and 2.2332, respectively. MLP-ANN-based models can reliably predict LOS in hospitalized patients with COVID-19 using readily available data at the time of admission. In this regard, the models developed in our study can help health systems to optimally allocate limited hospital resources and make informed evidence-based decisions.

Speech Synthesis Based on EEG Signal for Speech Impaired Patients by Using bLSTM Recurrent Neural Network

The disability rate in Indonesia is still relatively high and is one of the main health problems which reaches 30.38 million people or 14.2% of the Indonesian population. One of these types of disabilities is speech impairment. There are several possible causes for speech impairment, including the focal disturbance. This situation occurs because of disturbances in the vocal cords caused by injuries due to accidents and other conditions, such as throat cancer, which of course will reduce the productivity of the sufferer. Sign language can be used to communicate, but it still has limitations for normal individuals. In addition, speech synthesis using brain computer interface (BCI) based on electrocorticography (ECoG) has been developed. However, this method still has a weakness, namely invasive and allows the emergence of large enough scar tissue, so that it can reduce the quality of brain biopotential to be recorded. Therefore, a non-invasive EEG-based speech synthesis method was initiated. This method uses bLSTM as one of the components of the RNN model, so that it can construct syllables into words. This system consists of datasets, data filter programs, data segmentation programs, feature extraction programs, ANN and RNN deep learning model training programs, and text-to-speech programs. ANN and RNN form a 2-level deep learning. The testing accuracy and accuracy of the ANN are 26.04% and 20.83%, while the accuracy of the RNN is 81.25%. To improve these results, in the future, researchers can improve the data collection process and increase the number of the data, use the correct extraction feature, and compare several machine learning architectures, to produce optimal accuracy.

Artificial Neural Network Technique for Groundwater Modelling of Jaspura Block

As a result of economic development and climatic change, arid and semi-arid areas are confronted with significant difficulties in the management of limited freshwater resources. For the most part, groundwater is the most significant water supply in these regions. Groundwater level prediction is a critical component of appropriate sustainable development and must be done correctly. The use of physical-based models is often used in the modelling and prediction of groundwater. However, owing to data shortages in many arid and semi-arid regions, they are not relevant in these areas. The usefulness of data-driven techniques in modelling complicated and nonlinear hydrological processes has been shown in many studies. It is the implementation and comparison of four algorithm-based models for forecasting groundwater levels that is the focal point of this research. This article compares the different Artificial Neural Network method and applies to the Jaspura Block of Banda Districts which is part of the Yamuna River Basin. For the forecast of groundwater levels four distinct algorithms Levenberg Marquardt, Gradient Descent, Scaled Conjugate Gradient and Bayesian regularization are used for optimal design. The ANN training data for input is collected from Recharge and Discharge data while groundwater level data for output layer were being used. The best algorithm comes to the Levenberg Marquardt algorithm in comparison with the other algorithms.

Подход к проектированию нейронной сети для формирования индивидуальной траектории тестирования знаний

The paper discusses the issues of implementing an adaptive testing system based on the use of artificial neural network (INS) modules, which should solve the problem of intelligent choice of the next question, forming an individual testing trajectory. The aim of the work is to increase the accuracy of the INS to form the level of complexity of the next test question for two types of architectures – direct propagation (FNN – Feedforward Neural Network) and recurrent with long-term short-term memory (LSTM – Long-Short Term Memory). The data affecting the quality of training are analyzed, the architectures of the input layer of the direct propagation INS are considered, which have significantly improved the quality of neural networks. To solve the problem of choosing the thematic block of the question, a hybrid module structure is proposed, including the INS itself and a software module for algorithmic processing of the results obtained from the INS. A study of the feasibility of using direct propagation ANNs in comparison with the LSTM architecture was carried out, the input parameters of the network were identified, various architectures and parameters of the ANN training were compared (algorithms for updating weights, loss functions, the number of training epochs, packet sizes). The substantiation of the choice of a direct distribution network in the structure of the hybrid module for selecting a thematic block is given. The above results were obtained using the Keras high-level library, which allows you to quickly start at the initial stages of research and get the first results. Traditionally, learning has taken place over a large number of eras.