Neural Network Learning Algorithm Research Articles

Prediction and Value of Ultrasound Image in Diagnosis of Fetal Central Nervous System Malformation under Deep Learning Algorithm

This study was to explore the application of deep learning neural network (DLNN) algorithms to identify and optimize the ultrasound image so as to analyze the effect and value in diagnosis of fetal central nervous system malformation (CNSM). 63 pregnant women who were gated in the hospital were suspected of being fetal CNSM and were selected as the research objects. The ultrasound images were reserved in duplicate, and one group was defined as the control group without any processing, and images in the experimental group were processed with the convolutional neural network (CNN) algorithm to identify and optimize. The ultrasound examination results and the pathological test results before, during, and after the pregnancy were observed and compared. The results showed that the test results in the experimental group were closer to the postpartum ultrasound and the results of the pathological result, but the results in both groups showed no statistical difference in contrast to the postpartum results in terms of similarity ( P > 0.05 ). In the same pregnancy stage, the ultrasound examination results of the experimental group were higher than those in the control group, and the contrast was statistically significant ( P < 0.05 ); in the different pregnancy stages, the ultrasound examination results in the second trimester were more close to the postpartum examination results, showing statistically obvious difference ( P < 0.05 ). In conclusion, ultrasonic image based on deep learning was higher in CNSM inspection; and ultrasonic technology had to be improved for the examination in different pregnancy stages, and the accuracy of the examination results is improved. However, the amount of data in this study was too small, so the representative was not high enough, which would be improved.

A deep learning based decision support system for diagnosis of Temporomandibular joint disorder

Temporomandibular Joint sounds are a very common disorder in the general population. Temporomandibular Disorder (TMD) is any discomfort related to Temporomandibular Joint (TMJ). In this paper, a novel decision support system based on deep learning and neural network algorithms for diagnosis of Temporomandibular Joint disorder is introduced. A non-invasive device is designed for the recording of TMJ sounds. An interface is developed that will facilitate the dentist to operate on the recorded audio data. The collected data consist of the patient's left and right Temporomandibular Joint sound, ambient noise sound, the patient's clinical data, the dentist's notes about the patient, diagnosis, and treatment. Then signal processing, artificial neural network and deep learning algorithms are used to classify these measurements, and thus, the method that decides about the patient's condition is developed. Frequency, statistical and deep learning-based methods are compared in terms of classification success. The results show that the classification success of the classification method based on deep learning is consistently over 94.5% and it is more successful than the previous two methods. The proposed system can give the physician an idea about the effectiveness of the treatment methods applied to the patient in order to treat joint sounds which are among the important symptoms of TMD.

Neuromorphic advancements architecture design and its implementations technique

Neuromorphic Architectures (NA) is hardware network systems which, designed on the principles of neural functions. The network systems are inspired from biological neural networks. Each node or neuron in the artificial Neural networks (ANN) are connected to each other using a synapse. Similar to the biological brains, the connection will be controlled with the amplitude of the connection between nodes, which termed as synaptic weights. Unlike in the conventional architecture, in ANNs consists of huge quantity of extremely organized dealing out elements operational in union to resolve the real world problems. NA is considered as the main soft-computing knowledge and has been widely researched. It is applied during last decades for the computational model. This paper basically focuses on the NA and neural networks and implementation. Neural network and machine learning algorithms are used by data classification in NA. This data will be provided a number of of the modern advancement, including super-computer, and single device implementations, approaches dependent on spiking and non-spiking neuron. Machine learning hardware devices are used to utilization of memristive device.

Automatic COVID-19 pneumonia diagnosis from x-ray lung image: A Deep Feature and Machine Learning Solution

Coronavirus disease 2019 was announced after unidentified pneumonia was discovered in Wuhan, China, and quickly spread around the world (COVID-19). This outbreak has claimed the lives of so many people. It has a long-term negative impact on public health. The goal of this study is to develop an intelligent computer-aided system that can detect positive COVID-19 cases automatically, which can help with daily medical problems. The proposed system is based on the convolution neural network (CNN) architecture and can automatically expose discriminative features on chest X-ray images due to its convolution with rich filter families and weight-sharing characteristics. As a deep feature extractor, the CNN model SqueezeNet was used. The extracted deep discriminative features were fed machine Decision Tree, Random Forest, Neural Network (NN), Naive Bayes, Logistic Regression, and k-nearest neighbor learning algorithms. As a result, the NN classifier with an accuracy of 97.24 per cent, a sensitivity of 0.9724, a specificity of 0.9858, and an F-score of 0.972 provided the most effective results. The high detection performance obtained in this study demonstrates the utility of deep CNN features and an NN classifier approach for detecting COVID-19 cases in CXR images. With the current resources, this would be hugely beneficial in speeding up disease diagnosis.

The ensemble approach in comparison with the diverse feature selection techniques for estimating NPPs parameters using the different learning algorithms of the feed-forward neural network

Several reasons such as no free lunch theorem indicate that there is not a universal Feature selection (FS) technique that outperforms other ones. Moreover, some approaches such as using synthetic dataset, in presence of large number of FS techniques, are very tedious and time consuming task. In this study to tackle the issue of dependency of estimation accuracy on the selected FS technique, a methodology based on the heterogeneous ensemble is proposed. The performance of the major learning algorithms of neural network (i.e. the FFNN-BR, the FFNN-LM) in combination with the diverse FS techniques (i.e. the NCA, the F-test, the Kendall's tau, the Pearson, the Spearman, and the Relief) and different combination techniques of the heterogeneous ensemble (i.e. the Min, the Median, the Arithmetic mean, and the Geometric mean) are considered. The target parameters/transients of Bushehr nuclear power plant (BNPP) are examined as the case study. The results show that the Min combination technique gives the more accurate estimation. Therefore, if the number of FS techniques is m and the number of learning algorithms is n, by the heterogeneous ensemble, the search space for acceptable estimation of the target parameters may be reduced from n × m to n × 1. The proposed methodology gives a simple and practical approach for more reliable and more accurate estimation of the target parameters compared to the methods such as the use of synthetic dataset or trial and error methods.

Evaluation of ASR in concrete using acoustic emission and deep learning

Alkali-silica reaction (ASR) is one of main damages causes in concrete structures such as nuclear power plants which may endanger structural serviceability and integrity. Acoustic emission (AE) is a passive nondestructive method for structural health monitoring. It is very sensitive and has the capability of monitoring structures continuously. This method may be an alternative for early damage detection in concrete nuclear structures affected by ASR. The innovation of this paper lies in the implementation of deep learning algorithms to evaluate the ASR progress. ASR was monitored by AE in a concrete specimen, which was cast with reactive coarse aggregates and reinforced by steel rebars. The AE signals recorded during the experiment were filtered and divided into two classes. Two deep learning algorithms of convolutional neural network (CNN) and stacked autoencoder were employed to classify the AE signals into the corresponding classes. The model based on CNN resulted in a classifier with higher accuracy than the model based on the autoencoder network.

Using Deep Learning Neural Network in Artificial Intelligence Technology to Classify Beef Cuts

The objective of this research was to evaluate the deep learning neural network in artificial intelligence (AI) technologies to rapidly classify seven different beef cuts (bone in rib eye steak, boneless rib eye steak, chuck steak, flank steak, New York strip, short rib, and tenderloin). Color images of beef samples were acquired from a laboratory-based computer vision system and collected from the Internet (Google Images) platforms. A total of 1,113 beef cut images were used as training, validation, and testing data subsets for this project. The model developed from the deep learning neural network algorithm was able to classify certain beef cuts (flank steak and tenderloin) up to 100% accuracy. Two pretrained convolution neutral network (CNN) models Visual Geometry Group (VGG16) and Inception ResNet V2 were used to train, validate, and test these models in classifying beef cut images. An image augmentation technique was incorporated in the convolution neutral network models for avoiding the overfitting problems, which demonstrated an improvement in the performance of the image classifier model. The VGG16 model outperformed the Inception ResNet V2 model. The VGG16 model coupled with data augmentation technique was able to achieve the highest accuracy of 98.6% on 116 test images, whereas Inception ResNet V2 accomplished a maximum accuracy of 95.7% on the same test images. Based on the performance metrics of both models, deep learning technology evidently showed a promising effort for beef cuts recognition in the meat science industry.

ПОШУК ОПТИМАЛЬНОЇ ТОПОЛОГІЇ ШТУЧНИХ НЕЙРОННИХ МЕРЕЖ НА БАЗІ БАГАТОВИМІРНИХ ПОЛІНОМІВ ЛЕЖАНДРА.

Earlier in [1], it was concluded that it is necessary to improve the learning algorithms of neural networks operating in systems that generate electricity using renewable energy sources. This article is intended to acquaint the reader with a new type of activation functions of artificial neural networks (ANN), namely - the use of Legendre polynomials, as well as a new method of learning ANN, when this process is not sequential, as usual, but in parallel. The accepted statements made it possible to make sure that the new, designed neural network has better properties (such as training time and less value of learning error) than the standard ones. The relevance of this topic lies in the following provisions: - improving the interaction between the solar station and artificial intelligence systems, through increased productivity; - taking into account the transients in the electrical network by means of intelligent control, through the use of neural networks of the proposed architecture. The developed neural networks have found their application in the work of a photovoltaic station. Their main purpose is to fulfill the forecast in the electrical networks of the amount of generated power. To successfully complete the task, the following goals were set and solved: to analyze and compare standard activation functions and algorithms for ANN training, to show methods and describe the improvement of networks, to demonstrate the application of developed ANN in photovoltaic problems. This article was designed to acquaint with the new method of building neural networks, which is based on seeing the transmission of signals in a non-sequential way, such as parallel, with certain features of the connection with which it was given in the text. The paper also demonstrates the use of the Legendre polynomial using qualitative neural network activation functions that work with solar panels. For confirmation in the article the answers to calculations are given. In future materials it is planned to streamline in more detail the process of modeling and compiling a mathematical calculation for the construction of neural networks.

Cumulative infiltration and infiltration rate prediction using optimized deep learning algorithms: A study in Western Iran

Study regionSixteen different sites from two provinces (Lorestan and Illam) in the western part of Iran were considered for the field data measurement of cumulative infiltration, infiltration rate, and other effective variables that affect infiltration process. Study focusSoil infiltration is recognized as a fundamental process of the hydrologic cycle affecting surface runoff, soil erosion, and groundwater recharge. Hence, accurate prediction of the infiltration process is one of the most important tasks in hydrological science. As direct measurement is difficult and costly, and empirical models are inaccurate, the current study proposed a standalone, and optimized deep learning algorithm of a convolutional neural network (CNN) using gray wolf optimization (GWO), a genetic algorithm (GA), and an independent component analysis (ICA) for cumulative infiltration and infiltration rate prediction. First, 154 raw datasets were collected including the time of measuring; sand, clay, and silt percent; bulk density; soil moisture percent; infiltration rate; and cumulative infiltration using field survey. Next, 70 % of the dataset were used for model building and the remaining 30 % was used for model validation. Then, based on the correlation coefficient between input variables and outputs, different input combinations were constructed. Finally, the prediction power of each developed algorithm was evaluated using different visually-based (scatter plot, box plot and Taylor diagram) and quantitatively-based [root mean square error (RMSE), mean absolute error (MAE), the Nash-Sutcliffe efficiency (NSE), and percentage of bias (PBIAS)] metrics. New Hydrological Insights for the RegionFinding revealed that the time of measurement is more important for cumulative infiltration, while soil characteristics (i.e. silt content) are more significant in infiltration rate prediction. This shows that in the study area, silt parameter, which is the dominant constituent parameter, can control infiltration process more effectively. Effectiveness of the variables in the present study, in the order of importance are time, silt, clay, moisture content, sand, and bulk density. This can be related to the fact that most of study area is rangeland and thus, overgrazing leads to compaction of the silt soil that can lead to a slow infiltration process. Soil moisture content and bulk density are not highly effective in our study because these two factors do not significantly change across the study area. Findings demonstrated that the optimum input variable combination, is the one in which all input variables are considered. The results illustrated that CNN algorithms have a very high performance, while a metaheuristic algorithm enhanced the performance of a standalone CNN algorithm (from 7% to 28 %). The results also showed that a CNN-GWO algorithm outperformed the other algorithms, followed by CNN-ICA, CNN-GA, and CNN for both cumulative infiltration and infiltration rate prediction. All developed algorithms underestimated cumulative infiltration, while overestimating infiltration rates.

Methodology formation of the training sample short-term forecasting electricity load for an energy supply company using data mining technologies

For effective management of the power system operation mode the predictive information about hourly electrical load of all consumers is required. Forecasting errors, as a rule, lead to a decline of the technological and economic indicators of the power system operation, due to unreasonable changes of the generating equipment operating mode, as well as the selection of a non-optimal scheme of electrical networks. This article is devoted to improving the accuracy of short-term load forecasting of delivery points cluster of energy sales company with the use artificial neural networks. One of the most important conditions for achieving high prediction accuracy is the quality of the data sample required for training and testing neural network algorithms for short-term loading forecast. The proposed methodology is based on the authors’ analysis uses the factors influencing the hourly power loading. The proposed methodology is based on the authors’ analysis uses the factors influencing the hourly power loading as well as methods for improving the convergence of learning algorithms for artificial neural networks.

A statistical model of neural network learning via the Cramer–Rao lower bound

The neural networks (NN) remain as black boxes, albeit their quite successful stories everywhere. It is mainly because they provide only the complex structure of the underlying network with a huge validation data set whenever their serendipities reveal themselves. In this paper, we propose the statistical NN learning model related to the concept of universal Turing computer for regression predictive model. Based on this model, we define ’statistically successful NN (SSNN) learning.’ This is mainly done by calculating the well-known Cramer–Rao lower bound for the averaged square error (ASE) of NN learning. Using such formal definition, we propose an ASE-based NN learning (ANL) algorithm. The ANL algorithm not only implements the Cramer–Rao lower bound successfully but also presents an effective way to figure out a complicated geometry of ASE over hyper-parameter space for NN. This enables the ANL to be free of huge validation data set. Simple numerical simulation and real data analysis are done to evaluate performance of the ANL and present how to implement it.

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

It is challenging to train deep spiking neural networks (SNNs) directly due to the difficulties associated with the nondifferentiable neuron model. In this work, an end-to-end learning algorithm based on discrete current-based leaky integrate-and-fire (C-LIF) neuron model and surrogate gradient is proposed to leverage the encoder–decoder network architecture to train deep SNNs directly. The proposed algorithm is capable of learning deep spatiotemporal features relying on current time step only, and several acceleration techniques including backward phase skipping and layerwise FreezeOut are proposed to accelerate the training. Experimental results show that the proposed learning algorithm achieved the classification accuracies of 98.40% and 95.83% on the dynamic neuromorphic data sets MNIST-DVS and DVS-Gestures, respectively, and of 99.58% and 95.97% on static vision data sets MNIST and SVHN, respectively, which are comparable to the existing state-of-the-art results. The training speed was accelerated by up to 49.5% on MNIST and 36.6% on DVS-Gestures with the proposed acceleration techniques while maintaining the same level of accuracy.

Artificial Neural Network Model with PSO as a Learning Method to Predict Movement of the Rupiah Exchange Rate against the US Dollar

The movement of currency exchange rate can be predicted in the next few days, this is used by economic actors to get profit. Artificial Neural Network with the backpropagation learning method is good enough to use for forecasting time series data, it's just that in its application this method was considered to have shortcomings such as a long training time to achieve convergence. The purpose of this research is to form a Multilayer Perceptron Artificial Neural Network model with the Particle Swarm Optimization (PSO) algorithm as a learning method in the case of currency exchange rate prediction. This research produced a model that can predict the movement of the Rupiah exchange rate against the US Dollar, while the model formed was the MLP-PSO model with an error rate of 5.6168 x 10-8, slightly better than the MLP-BP model with an error rate of 6.4683 x 10-8. These results indicated that the PSO algorithm can be used as a learning algorithm in the Multilayer Perceptron Artificial Neural Network.

Delay-weight plasticity-based supervised learning in optical spiking neural networks

We propose a modified supervised learning algorithm for optical spiking neural networks, which introduces synaptic time-delay plasticity on the basis of traditional weight training. Delay learning is combined with the remote supervised method that is incorporated with photonic spike-timing-dependent plasticity. A spike sequence learning task implemented via the proposed algorithm is found to have better performance than via the traditional weight-based method. Moreover, the proposed algorithm is also applied to two benchmark data sets for classification. In a simple network structure with only a few optical neurons, the classification accuracy based on the delay-weight learning algorithm is significantly improved compared with weight-based learning. The introduction of delay adjusting improves the learning efficiency and performance of the algorithm, which is helpful for photonic neuromorphic computing and is also important specifically for understanding information processing in the biological brain.

Comparison of Heat Demand Prediction Using Wavelet Analysis and Neural Network for a District Heating Network

Short-Term Load Prediction (STLP) is an important part of energy planning. STLP is based on the analysis of historical data such as outdoor temperature, heat load, heat consumer configuration, and the seasons. This research aims to forecast heat consumption during the winter heating season. By preprocessing and analyzing the data, we can determine the patterns in the data. The results of the data analysis make it possible to form learning algorithms for an artificial neural network (ANN). The biggest disadvantage of an ANN is the lack of precise guidelines for architectural design. Another disadvantage is the presence of false information in the analyzed training data. False information is the result of errors in measuring, collecting, and transferring data. Usually, trial error techniques are used to determine the number of hidden nodes. To compare prediction accuracy, several models have been proposed, including a conventional ANN and a wavelet ANN. In this research, the influence of different learning algorithms was also examined. The main differences were the training time and number of epochs. To improve the quality of the raw data and remove false information, the research uses the technology of normalizing raw data. The basis of normalization was the technology of the Z-score of the data and determination of the energy‒entropy ratio. The purpose of this research was to compare the accuracy of various data processing and neural network training algorithms suitable for use in data-driven (black box) modeling. For this research, we used a software application created in the MATLAB environment. The app uses wavelet transforms to compare different heat demand prediction methods. The use of several wavelet transforms for various wavelet functions in the research allowed us to determine the best algorithm and method for predicting heat production. The results of the research show the need to normalize the raw data using wavelet transforms. The sequence of steps involves following milestones: normalization of initial data, wavelet analysis employing quantitative criteria (energy, entropy, and energy‒entropy ratio), optimization of ANN training with information energy–entropy ratio, ANN training with different training algorithms, and evaluation of obtained outputs using statistical methods. The developed application can serve as a control tool for dispatchers during planning.

ARTIFICIAL NEURAL NETWORKS IN MEDICAL DIAGNOSIS

Artificial neural networks are finding many uses in the medical diagnosis application. The article examines cases of renopathy in type 2 diabetes. Data are symptoms of disease. The multilayer perceptron networks (MLP) is used as a classifier to distinguish between a sick and a healthy person. The results of applying artificial neural networks for diagnose renopathy based on selected symptoms show the network's ability to recognize to recognize diseases corresponding to human symptoms. Various parameters, structures and learning algorithms of neural networks were tested in the modeling process.

WITHDRAWN: Design and implementation of image classification based convolutional neural networks on hardware

Abstract In recent years, deep learning has grown rapidly. It was used in many fields especially artificial intelligence areas. A strong direction for methodology is the combination of deep learning and embedded systems. The project will design a deep neural network learning algorithm, which can be applied on hardware, such as FPGA. The research on the deep learning neural network and hardware is focused on existing research. As assistant tools, it uses PyTorch and CUDA. This work is focused on a convolutionary neural network image classification (CNN). Many good models such as ResNet, ResNeXt and MobileNet can be tested. An algorithm with the MobileNet model will be determined by adding these models to the design. Models such as FLOPs, number of parameters and classification accuracy are chosen in several ways. The mobile net-based algorithm is chosen, for applications with a 6-class data collection, with a 5.5 percent top-1 error. The MobileNet based algorithm also requires the hardware simulation. The parameters are translated to 8-bit integers from floating point numbers. Each layer's output numbers are reduced to fixed-bits to match the hardware cap. The number change on hardware is simulated by a number handling process. The Top-1 error is up to 12.3 percent on the basis of this simulation process, which is appropriate.

Designing a computerized information processing system to build a movement trajectory of an unmanned aircraft

This paper addresses the issue of developing a computerized system for processing information in the construction of the trajectory of an unmanned aircraft (UAC), a remotely-piloted aviation system (RPAS), or another robotic system. Resolving this task involves the neural network learning algorithms based on the mathematical model of movement. The construction of such a trajectory between two specified destinations has been considered that provides for the possibility of bypassing static and dynamic obstacles. The specified trajectory is divided into several smaller parts. The possibility of restructuring when changing the position of obstacles in space has been considered. A UAC flight control algorithm has been developed, which implies training a neural network for bypassing obstacles of different sizes. To predict the development of the situation when an object moves between two specified points in space, it is proposed to use the Q-Learning algorithm. It has been shown that the smallest number of steps required for moving along a specified trajectory is 18, the largest is 273 steps. In case of distortion during data transmission, the training of the neural network makes it possible to reduce the possibility of collision with obstacles by improving the accuracy and speed of information transfer between the on-board computer and operator. A system of the video support to moving objects was modeled; dependence charts of the normalized frame size at different parameter values were built. Using the charts makes it possible to determine the function of the maneuver intensity. Existing neural network learning methods such as CNN and LSTM were compared. It has been proven that the success rate reaches 74 % when using CNN only, while it amounts to 92 % at the hybrid application of CNN+LSTM. The simulation results have demonstrated the high efficiency of the developed algorithm

Tracing Mechanism of Sports Competition Pressure Based on Backpropagation Neural Network

Through the overall situation of athletes’ competition pressure, the pressure level of participating athletes can be understood and revealed. Analyzing the sources of stress and influencing factors of athletes can find measures to relieve and reduce stress and provide theoretical reference for the regulation of athletes’ competition pressure. Based on genetic algorithm and neural network theory, this paper proposes a method of tracing the sports competition pressure based on genetic algorithm backpropagation (BP) neural network to solve the problem that traditional neural network learning algorithm is slow and easy to fall into local minimum. There is no significant difference between male and female athletes in the level of competition pressure. Athletes have the same training methods and the same goals, and the competition pressure tends to be the same, with no obvious difference; athletes with different educational backgrounds have no significant differences in training, academics, sports injuries, interpersonal relationships, social expectations, and evaluations. Due to the particularity of the stage, the competition pressure of fourth-year undergraduate and third-year masters is significantly higher than that of other grades. The number of athletes participating in college table tennis tournaments has very significant differences in competition dimensions. There is significant difference in training and self-expectation dimension difference. The competition pressure of athletes who participated in the college table tennis championship for the first time was significantly higher than that of athletes who participated repeatedly. There were significant differences between athletes before and after adapting to the venue. Before adapting to the venue, the competition pressure of athletes is generally greater. After adapting to the venue, the competition pressure of athletes has been relieved.

UNSUPERVISED REAL-TIME DIAGNOSIS SYSTEM FOR ECG STREAMING DAT

Detecting anomalies in time series data plays a vital role in the various applications of diagnosis systems. The importance of anomaly detection is increased by its ability to detect abnormalities in Electrocardiogram (ECG) signals to generate alerts for cardiac health problems. An ECG is a time series that provides essential information about the electrical activity of the heart and is used in the diagnosis of numerous heart diseases. An accurate ECG streaming analytics approach requires continuous learning and adaptation in changing data behaviors. We aim to diagnose ECG by investigating healthy ECG and ECG with cardiological disorders by detecting anomalies in ECG signals. The main objective of this paper is to develop an efficient unsupervised diagnosing system for ECG streaming data based on an online sequence memory algorithm called Hierarchical Temporal Memory (HTM). The HTM is based on neural network and machine learning algorithm for continuous learning tasks. The proposed customization of the HTM algorithm based on our problem domain provides a significant performance results of detection of anomalies in the ECG signals.