Artificial Neural Network Learning Research Articles

Deep learning of GPS geodetic velocity

Installing permanent global positioning system (GPS) stations and receiving and monitoring long-term crustal deformation requires a high cost. Another solution, which could be an appropriate alternative, is applying some modern and smart estimation methods such as deep learning of artificial neural networks (DLANN). Based on the observations of the 42 GPS permanent stations in NW Iran, the velocity vectors of stations are estimated in unknown locations by four methods: back propagation of artificial neural networks (BPANN), least square collocation (LSC), Bat algorithm (BA) and random forest algorithm (RFA). BPANN has better performance and less variance than RFA, and the largest difference is in the gap areas, which estimates each vector method differently and it is preferred not to estimate these areas. The performances of LSC and BA algorithm is not better than BPANN. Therefore, it seems that the BPANN method can be considered as a suitable option for estimating the geodetic velocity field compared to other methods.

An Adaptive Information Security System for 5G-Enabled Smart Grid Based on Artificial Neural Network and Case-Based Learning Algorithms.

With the deployment of 5G Internet of Things (IoT) in the power system, the efficiency of smart grid is improved by increasing two-way interactions in different layers in smart grid. However, it introduces more attack interfaces that the traditional information security system in smart grid cannot response in time. The neuroscience-inspired models have shown their effectiveness in solving security and optimization problems in smart grid. How to improve the security mechanism in smart grid while taking into account the optimization of data transmission efficiency using neuroscience-inspired algorithms is the problem to be solved in this study. Therefore, an information security system based on artificial neural network (ANN) and improved multiple protection model is proposed. Based on the ANN algorithm, the link state sample space is used to train the model to obtain the optimal transmission path in 5G power communication network. Integrating the intelligent link state module, the zero-trust security protection platform using case-based learning algorithm is designed and taken as the first protection, the network security logical isolation facility is taken as the second protection, and the forward and backward isolation facilities are set as the third protection to achieve the strengthened security of 5G IoT in smart grid. The experimental results show the efficiency and effectiveness of the proposed algorithms. In addition, the experimental results also show that the proposed system can resist malicious terminal access, terminal hijacking, data tampering and eavesdropping, protocol fuzzy, and denial-of-service attacks, so as to reduce the security risks of 5G IoT in smart grid. Since the proposed system can be easily integrated into the existing smart grid structure in China, the proposed system can provide a reference for the design and implementation of 5G IoT in smart grid.

Machine learning (ML) algorithms and artificial neural network for optimizing in vitro germination and growth indices of industrial hemp (Cannabis sativa L.)

In vitro germination of hemp is challenging due to low germination and high contamination rates. Successful establishment of in vitro sterilization is the prerequisite of plant tissue culture studies. Recent advancements in the field of artificial neural network (ANN) and machine learning (ML) algorithms open new horizons for sustainable and precision agriculture. ANN and ML algorithms are powerful tools to evaluate the results and make more precise and high accuracy predictions in the field of plant tissue culture, especially for industrial purposes. Keeping in view, the study was designed to investigate the possible response of variable concentrations of hydrogen peroxide (H2O2) on germination and morphological traits of in vitro-grown hemp seedlings by using ML algorithms. Five different ML algorithms used in this study to evaluate the prediction of the output variables were: Support Vector Classifier (SVC), Gaussian Process (GP), Extreme Gradient Boosting (XGBoost), Random Forest (RF) models, and Multilayer Perceptron (MLP) neural network utilizing accuracy, F1 score, precision, and recall values. Among the tested models, the RF model exhibited better prediction of output variables with a high F1 score in the range of 0.98–1.00. The F1 scores of the other models ranged between 0.69 and 0.86. Response surface methodology (RSM) used to compute the optimum concentration of H2O2 revealed the statistically significant effect of H2O2 on in vitro germination and seedling growth. The optimum value of H2O2 for the maximum germination and seedling was optimized to about ~2.2% by using RSM. The present work is a case study about the application of different ML and ANN models in plant tissue culture and reveals the possibility of application in many other economic crops.

Analysis of the Effectiveness of Various Machine Learning, Artificial Neural Network and Deep Learning Methods in Detecting Fraudulent Credit Card Transactions

A credit card is an important financial tool that has emerged in parallel with the developments in technology from the past to the present and has become an indispensable part of human life. The credit card has many advantages that can be listed as facilitating online shopping, providing installments in purchases, and preventing cash dependence. This is why the rate of use of credit cards worldwide is increasing day by day. On the other hand, there are some risks of the credit cards highlighted by security concerns. The fraudsters who access the identity and credit card information of the consumers through different means use it to shop online without the consumer’s knowledge and gain an unfair advantage. Therefore, it is crucial to eliminate this security vulnerability that the fraudsters exploit and to develop an effective solution to the customer victimization experienced by e-commerce companies due to the fraudulent credit card transactions. With this motivation, the performance of the methods from different research fields was examined to explore the solution space in detail in terms of the problem at hand within the scope of this study. For this purpose, three machine learning algorithms (K-Nearest Neighbor, Naive Bayes, Support Vector Machine), two artificial neural network algorithms (Binary Classifier, Autoencoder), and two deep learning algorithms (Deep Autoencoder and Deep Neural Network Classifier) were implemented. The effectiveness of the algorithms in question was tested with a famous dataset widely used in the literature. Experimental results showed that the Deep Neural Network Classifier outperformed the other algorithms used in this study and the best study ever reported in the literature in detecting fraudulent credit card transactions when accuracy and AUROC performance criteria were taken into account.

Passenger demand forecasting for railway systems

Abstract The rapid increase of the population and the number of motor vehicles brought about the transportation problem today. It has brought the efforts of the operators to determine the headway of the vehicles during the day in order to minimize the waiting times of the passengers at the stops and increase the satisfaction of the passengers, taking into account the passenger demand. Nowadays, especially during the current pandemic period (COVID-19), passenger demand forecasting becomes much more significant, so that measures can be taken and headway planning can be made to adjust the social distance by identifying the number of passengers in advance. In this study, the significance of demand forecasting in the railway sector is considered, and the study tackles the issue in two stages: on line and station basis that make the study different from others. In the first stage of the study, passenger demand forecasting is made on line basis with statistical techniques such as regression analysis and simple average, the mean absolute percentage error values are calculated and compared. Regression analysis is conducted with SPSS Statistics 21.0 programme. In the second stage of the study, passenger demand forecasting is made with artificial neural network and machine learning (ML) algorithms technique on station basis and the error values (mean absolute error, BIAS, mean squared error, mean absolute percentage error, and root mean squared error) are compared. As a result of the study, while the best demand forecasting method is simple average on line basis, it is seen that the most successful and reliable results for demand forecasting on station basis are obtained through decision tree, which is one of the ML algorithms.

Phase formation prediction of high-entropy alloys: a deep learning study

High-entropy alloys (HEAs) represent prospective applications considering their outstanding mechanical properties. The properties in HEAs can be affected by the phase structure. Artificial neural network (ANN) is a promising machine learning approach for predicting the phases of HEAs. In this work, a deep neural network (DNN) structure using a residual network (RESNET) is proposed for the phase formation prediction of HEAs. It shows a high overall accuracy of 81.9%. Compared it with machine learning models, e.g., ANN and conventional DNN, its Micro-F1 score highlights the advantages of phase prediction of HEAs. It can remarkably prevent network degradation and improve the algorithm accuracy. It delivers a new path to develop the phase formation prediction model using deep learning, which can be of universal relevance in assisting the design of the HEAs with novel chemical compositions.

An Application of Improved MODIS-Based Potential Evapotranspiration Estimates in a Humid Tropic Brantas Watershed—Implications for Agricultural Water Management

The reliance on native MODIS-16 PET potential evapotranspiration (PET) in scarce-data-driven areas is growing in support among ecohydrological studies, yet information about its performance is limited or unknown as validation studies are mostly concentrated in developed countries. This study aimed to assess its performance at the monthly level using four ground measurements in a tropical watershed system with complex topography, applying a machine learning artificial neural network (ANN) to improve the estimates, and using the ANN-adjusted MODIS-16 PET to characterize the spatio-temporal patterns of PET in the Brantas watershed, as well as to understand the monthly patterns of water deficiency in areas under eight different vegetation covers. The results showed that the native MODIS-16 PET experienced overestimation with an RMSE of 37–66 mm/month and NRSME of up to 33%. The performance decreased in drier periods. The ANN-based adjustment using only one variable showed improved estimates with a reduction of RSME to only 14 mm and lower than 10% NRMSE. Sari-temporal patterns of PET in the Brantas watershed showed that the PET characteristics were not uniform. The southern part of the Brantas watershed has areas with relatively lower PET that are, thus, more prone to water deficiency. Complex topography and climate gradients within the watershed apparently became the multi-controllers of PET variations. The difference in vegetation cover also influenced the magnitudes of water deficiency.

Application of artificial neural network and machine learning algorithms for modeling the in vitro regeneration of chickpea (Cicer arietinum L.)

Application of artificial neural network and machine learning algorithms for modeling the in vitro regeneration of chickpea (Cicer arietinum L.)

An optimization clustering and classification based on artificial intelligence approach for internet of things in agriculture

This research focused on testing with maize, economical crop grown in Phetchabun province, Thailand, by installing a total of 20 sets of internet of things (IoT) devices which consist of soil moisture sensors and temperature and humidity sensors (DHT11). Data science tools such as rapidminer studio was used for data cleansing, data imputation, clustering, and prediction. Next, these data would undergo data cleansing in order to group them to obtain optimization clustering to identify the optimum condition and amount of water required to grow the maize through k-mean technique. From the analysis, the optimization result showed 3 classes and these data were further analyzed through prediction to identify precision. By comparing several algorithms including artificial neural network (ANN), decision tree, naïve bayes, and deep learning, it was found that deep learning algorithm can provide the most accurate result at 99.6% with root mean square error (RMSE)=0.0039. The algorithm obtained was used to write function to control the automated watering system to make sure that the temperature and humidity for growing maize is at appropriate condition. By using the improved watering system, it improved the efficacy of watering system which saves more water by 13.89%

Spatiotemporal dynamics of grassland aboveground biomass and its driving factors in North China over the past 20 years

Although remote sensing has enabled rapid monitoring of grassland aboveground biomass (AGB) at a regional scale, it is still a difficult challenge to construct an accurate estimation model of grassland AGB in a vast region to support the AGB dynamics analysis over a long time series. In this study, extensive grassland AGB measurements (collected in North China during the grassland growing season of 2000–2019), MODIS data, and environmental factors (climate, topography and soil) were employed to construct the grassland AGB models using four machine learning algorithms (random forest, support vector machine, artificial neural network and extreme learning machine) combined with four variable selections. The spatial distributions of annual grassland AGB from 2000 to 2019 were simulated based on the optimal AGB model. The temporal change and future trend of AGB series from 2000 to 2019 were comprehensively analyzed by the slope model and Hurst exponent. The influences of natural and anthropogenic factors on grassland AGB dynamics were explored quantitatively using the Geodetector model. The results showed that (1) the random forest model constructed from the variables selected by the successive projections algorithm is the optimal grassland AGB model. (2) The 20-year average grassland AGB in North China showed an overall spatial distribution of being low in the central and western parts and high in the southeastern part. (3) The annual maximum grassland AGB in most regions (82.71%) showed an increasing trend during 2000–2019; and most of the grasslands with a decreasing trend of AGB were located in regions with low AGB values and arid climates. (4) The future trend of grassland AGB after the study period may be optimistic, as reflected by more grassland AGB was predicted to increase rather than decrease (70.38% vs. 29.62%). (5) The main driving factors of spatiotemporal dynamics of grassland AGB were precipitation, soil type, and livestock density; the interactive influence of two drivers on AGB showed mutual enhancement.

Utilization of Whittaker-Henderson Smoothing Method for Improving Neural Network Forecasting Accuracy

Health institutions need to ensure the availability of drug stocks for patients. There are challenges related to the uncertainty of the amount of drug use for the next period. Uncertainty can be reduced by analysing historical drug data to predict future demand. Time series can contain spikes or fluctuation pattern which spikes can disguise the main information. Hence, it can affect the accuracy of the prediction model. One widely used forecasting method in the time series data is the artificial neural network (ANN) method. The ANN method requires the pre-processing stage of the data before the training process. The pre-processing stage is essential to obtain information or knowledge. This study focused on applying smoothing methods at the pre-processing stage of the ANN method. The application of the smoothing method was expected to improve the quality of ANN learning data that would lead to better predictive accuracy. This research focuses on implementing the smoothing method in data pre-processing step for ANN method. Smoothing methods used in this research were exponential smoothing (ES) and Whittaker-Henderson (WH) smoothing applied to two time series datasets. The refining method used in this study was the WH method, which was tested on two time series datasets of medicine. The results show that the mean square error (MSE) obtained by applying the WH method was lower than the non-smoothing ANN for both datasets. Evaluation results revealed that implementing WH smoothing method in data pre-processing step for ANN (WH+ANN) provided MSE significantly lower than ANN results with a confidence level of 94% for dataset 1 and 85% for the dataset 2.

Non-parametric multiple inputs prediction model for magnetic field dependent complex modulus of magnetorheological elastomer

This study introduces a novel platform to predict complex modulus variables as a function of the applied magnetic field and other imperative variables using machine learning. The complex modulus prediction of magnetorheological (MR) elastomers is a challenging process, attributable to the material’s highly nonlinear nature. This problem becomes apparent when considering various possible fabrication parameters. Furthermore, traditional parametric modeling methods are limited when applied to solve larger-scale cases involving large databases. Consequently, the application of non-parametric modeling such as machine learning has gained increasing attraction in recent years. Therefore, this work proposes a data-driven approach for predicting multiple input-dependent complex moduli using feedforward neural networks. Besides excitation frequency and magnetic flux density as operating conditions, the inputs consider compositions and curing conditions represented by magnetic particle weight percentage and the curing magnetic field, respectively. Extreme learning machines and artificial neural networks were used to train the models. The simulation results obtained at various curing conditions and other inputs confirm that the predicted complex modulus has high accuracy with an R2 of about 0.997, as compared to the experimental results. Furthermore, the predicted complex modulus pattern and magnetorheological effect agree with the experimental data using both the learned and unlearned data.

Effective adsorption of metolachlor herbicide by MIL-53(Al) metal-organic framework: Optimization, validation and molecular docking simulation studies

Effective removal and optimization models of metolachlor (MET) adsorption was carried out using MIL-53(Al) metal–organic framework (MOF), response surface methodology (RSM), artificial neural network (ANN) and molecular docking simulation. The MOF was hydrothermally synthesized and characterized using the Field Emission Scanning Electron Microscopy (FESEM), Powdered X-ray diffraction (XRD), Fourier Transformed Infrared (FTIR) and Brunauer, Emmett and Teller (BET). High adsorption capacity of MET was recorded (241.617 mg/g) with fast equilibration time of ∼ 25 min at 0.999 coefficient of determination (R2) which was due to the high surface area of the MOF (1104 m2/g). The kinetics followed the pseudo-second order model describing a chemisorption adsorption process. The isotherm showed a multilayer adsorption process of the Freundlich model with R2 of 0.990. The thermodynamic parameters followed an endothermic and spontaneous adsorption. The optimization by the RSM was significant with minimum number of experimental runs, lesser error and showed a simultaneous interaction of the adsorption parameters in predicting MET adsorption capacity. The ANN learning algorithm performed significantly in predicting and validating the experimental conditions designed by the RSM model. Both the RSM and ANN model provide remarkable values that are very similar with the experimental results. The binding mode and binding affinity of MIL-53(Al) and MET were determined by the docking simulation. Prospect for the reusability potential of the MOF was explored and the findings were significant. Hence, the results showed the potential of the MOF for the effective remediation of MET in aqueous medium.

Comparison of sex determination using three methods applied to the greater sciatic notch of os coxae in a Thai population: Dry bone morphology, 2-dimensional photograph morphometry, and deep learning artificial neural network.

The os coxa is commonly used for sex and age estimation with a high degree of accuracy. Our study aimed to compare the accuracy among three methods, which include a deep learning approach to increase the accuracy of sex prediction. A total sample of 250 left os coxae from a Thai population was divided into a 'training' set of 200 samples and a 'test' set of 50 samples. The age of the samples ranged from 26 to 94 years. Three methods of sex determination were assessed in this experiment: a dry bone method, an image-based method and deep learning method. The intra- and inter-observer reliabilities were also assessed in the dry bone and photo methods. Our results showed that the accuracies were 80.65%, 90.3%, and 91.95% for the dry bone, image-based, and deep learning methods, respectively. The greater sciatic notch shape was wide and symmetrical in females while narrow and asymmetrical in males. The intra- and inter-observer agreements were moderate to almost perfect level (Kappa = 0.67-0.93, ICC = 0.74-0.94). Conclusion: The image-based and deep learning methods were efficient in sex determination. However, the deep learning technique performed the best among the three methods due to its high accuracy and rapid analysis. In this study, deep learning technology was found to be a viable option for remote consultations regarding sex determination in the Thai population.

Regression Model-Based AMS Circuit Optimization Technique Utilizing Parameterized Operating Condition

An analog and mixed-signal (AMS) circuit that draws on machine learning while using a regression model differs in terms of the design compared to more sophisticated circuit designs. Technology structures that are more advanced than conventional CMOS processes, specifically the fin field-effect transistor (FinFET) and silicon-on-insulator (SOI), have been proposed to provide the higher computation performance required to meet various design specifications. As a result, the latest research on AMS design optimization has enabled enormous resource savings in AMS design procedures but remains limited with regard to reflecting the intended operating conditions in the design parameters. Hereby, we propose what is termed a supervised learning artificial neural network (ANN) as a means by which to define an AMS regression model. This approach allows for rapid searches of complex design dimensions, including variations in performance metrics caused by process–voltage–temperature (PVT) changes. The method also reduces the considerable computation expense compared to that of simulation-program-with-integrated-circuit-emphasis (SPICE) simulations. Hence, the proposed AMS circuit design flow generates highly promising output to meet target requirements while showing an excellent ability to match the design target performance. To verify the potential and promise of our design flow, a successive approximation register analog-to-digital converter (SAR ADC) is designed with a 14 nm process design kit. In order to show the maximum single ADC performance (6-bit∼8-bit resolution and few GS/s conversion speed), we have set three different ADC performance targets. Under all SS/TT/FF corners, ±6.25% supply voltage variation, and temperature variation from −40 ∘C to 80 ∘C, the designed SAR ADC using our proposed AMS circuit optimization flow yields remarkable figure-of-merit energy efficiency (approximately 15 fJ/conversion step).

Modeling a Practical Dual-Fuel Gas Turbine Power Generation System Using Dynamic Neural Network and Deep Learning

Accurate simulations of gas turbines’ dynamic performance are essential for improvements in their practical performance and advancements in sustainable energy production. This paper presents models with extremely accurate simulations for a real dual-fuel gas turbine using two state-of-the-art techniques of neural networks: the dynamic neural network and deep neural network. The dynamic neural network has been realized via a nonlinear autoregressive network with exogenous inputs (NARX) artificial neural network (ANN), and the deep neural network has been based on a convolutional neural network (CNN). The outputs selected for simulations are: the output power, the exhausted temperature and the turbine speed or system frequency, whereas the inputs are the natural gas (NG) control valve, the pilot gas control valve and the compressor variables. The data-sets have been prepared in three essential formats for the training and validation of the networks: normalized data, standardized data and SI units’ data. Rigorous effort has been carried out for wide-range trials regarding tweaking the network structures and hyper-parameters, which leads to highly satisfactory results for both models (overall, the minimum recorded MSE in the training of the MISO NARX was 6.2626 × 10−9 and the maximum MSE that was recorded for the MISO CNN was 2.9210 × 10−4, for more than 15 h of GT operation). The results have shown a comparable satisfactory performance for both dynamic NARX ANN and the CNN with a slight superiority of NARX. It can be newly argued that the dynamic ANN is better than the deep learning ANN for the time-based performance simulation of gas turbines (GTs).

A Novel Weights of Weights Approach for Efficient Transfer Learning in Artificial Neural Networks

Deep learning is often reputed to be slow at training due to large number of layers. There are several approaches including used for improving training mechanism. Transfer learning sometimes referred as transfer of knowledge is one such approach. However, Transfer learning is often moving weights or layers as a whole including weights that might not be significant.This research tries to implement a novel Weights of Weights approach where significant weights are extracted from a training network. Only the significant weights are transferred to new untrained network instead of transferring all the layers or weights. Experiments are carried out using a 7-layered neural networks on multivariate wine dataset to evaluate the efficiency of the model.The experiment results show that the transferable model has provided better accuracy of over 40%-60% (based on layers) without retraining compared to traditional approach of copying all the layers with same and less number of nodes.

Supervised Machine Learning Techniques for the Prediction of the State of Charge of Batteries in Photovoltaic Systems in the Mining Sector

One of the critical aspects in the mining sector is energy, being of great importance for the operation since if it were to stop, one of the consequences would be the loss of large amounts of money. The research objective is to predict the State of Charge of Batteries of equipment powered by photovoltaic solar panels in the mining sector based on automatic supervised learning techniques. A monitoring system records each energy variable programmed in the photovoltaic system, for which an analysis of the data extracted from the monitoring system was carried out. The data were evaluated using automatic supervised learning techniques using the RapidMiner tool, whose prediction average was 90.12%. The technique of automatic supervised learning of artificial neural networks was chosen to predict the state of charge of batteries for photovoltaic systems. A software tool was built with the neural network. The analysis and discussion of the results of the training of the model were carried out, the contribution of this research being to determine the prediction of the state of charge of batteries in photovoltaic systems in the mining sector using techniques of supervised machine learning which was the neural network. Finally, with the model correctly trained, validation was carried out that allowed comparing the predictive data with the data in real-time, obtaining a good relationship and satisfactory results.

MR-guided non-invasive typing of brain gliomas using machine learning

To evaluate diagnostic accuracy of deep learning methods for glioma typing according to the 2007 WHO classification based on preoperative magnetic resonance imaging (MRI) data. The study included MR scans of patients with glial tumors undergoing neurosurgical treatment at the Burdenko National Medical Research Center for Neurosurgery. All patients underwent preoperative contrast-enhanced MRI. 2D and 3D MR scans were used for learning of artificial neural networks with two architectures (Resnest200e and DenseNet, respectively) in classifying tumors into 4 categories (WHO grades I-IV). Learning was provided on 80% of random examinations. Classification quality metrics were evaluated in other 20% of examinations (validation and test samples). Analysis included 707 contrast-enhanced T1 welghted images. 3D classification based on DenseNet model showed the best result in predicting WHO tumor grade (accuracy 83%, AUC 0.95). Other authors reported similar results for other methods. The first results of our study confirmed the fundamental possibility of grading axial contrast-enhanced T1 images according to the 2007 WHO classes using deep learning models.

Construction and implementation of music recommendation model utilising deep learning artificial neural network and mobile edge computing

The purpose is to better develop China's music industry, facilitate users' online music works queries, and encourage the development of excellent music works. The music recommendation system is analysed, and a new music recommendation system is constructed based on the combination of the hybrid Deep Learning (DL) Artificial Neural Network (ANN) and Mobile Edge Computing (MEC) technologies. First, the principle of MEC is analysed. Then, the combination of Deep Learning Artificial Neural Network (DLANN) technology and MEC technology is discussed. Afterward, DLANN and MEC are fused to implement the music recommendation model, and the proposed model is evaluated through experiment. The results show that the combination of DLANN and MEC technologies can increase the DL efficiency of the computer, the storage capacity, and the overall efficiency of the servers. This proves the feasibility of the proposed music recommendation system. The user satisfaction of the proposed music recommendation system exceeds all mainstream music recommendation algorithms on the market. Thus, the research can provide a reference for the improvement of the music recommendation algorithm in the future and is of great significance. The proposed music recommendation system outperforms most of the current music recommendation systems on the market.