Number Of Training Epochs Research Articles

Classifying Handwritten Digit Recognition using CNN and PSO

A normal human can easily recognize any written or typed or scanned text, numbers, etc., but when it comes to a machine, it is difficult to find out what exactly that given text or numbers. It will be difficult to recognize a handwritten digit for a machine. Many machine learning methods were used to fix the handwritten digit recognition issue. It is growing in more convoluted domains, so its training complexity is also increasing. To overcome this complexity problem, many algorithms have been implemented. In this paper, the Convolutional Neural Network (CNN) and Particle Swarm Optimization (PSO), those two approaches do use for recognition of the isolated handwritten digit. Customized PSO is used to reduce the overall computation time of the proposed system. The customized PSO used with CNN, to decreases the required number of epochs for training. It is used to identify digits in the MNIST handwritten digital database to predict the number. The system has achieved an average of 94.90% accuracy.

Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues

Pesticide residue is an important factor that affects food safety. In order to achieve effective detection of pesticide residues in apples, a machine-vision-based segmentation algorithm and hyperspectral techniques were used to segment the foreground and background regions of the apple image. By calculating the roundness value and extracting the region with the highest roundness value in the connected region, a region of interest (ROI) mask was created for the apple. Four pesticides (chlorpyrifos, carbendazim and two mixed pesticides) and an inactive control were used at the same concentration of 100 ppm (except for the control group), and the hyperspectral region of the corresponding sample image was extracted by obtaining the different types of pesticide residues in the ROI masks. To increase the diversity of the samples and to expand the dataset, Gaussian white noise with a varying signal-to-noise ratio was added to each of the hyperspectral images of the apple. The number of samples was increased from four types of 12 samples to four types of 72 samples, giving 4608 hyperspectral data images in each category. The structure and parameters of a convolutional neural network (CNN) were determined using theoretical analysis and experimental verification. All the extracted hyperspectral images of apples were normalized to 227 × 227 × 3 pixels as the input of the CNN network for pesticide residue detection. There were 18,432 sample data of four types for 72 samples. Of these, 12,288 images were selected using a bootstrap sampling method as the training set, and 6144 as the test set, with no overlap. The test results show that when the number of training epochs was 10, the accuracy of the test set detection was 99.09%, and the detection accuracy of the single-band average image was 95.35%. A comparison with traditional k-nearest neighbor (KNN) and support vector machine classification algorithms showed that the detection accuracy for KNN was 43.75% and the average time was 0.7645 s. These results demonstrate that our method is a small-sample, non-contact, fast, effective and low-cost technique that can provide effective pesticide residue detection in postharvest apples.

Predicting Three-Dimensional Ground Reaction Forces in Running by Using Artificial Neural Networks and Lower Body Kinematics

This study explored the use of artificial neural networks in the estimation of runners' kinetics from lower body kinematics. Three supervised feed-forward artificial neural networks with one hidden layer each were modelled and assigned individually with the mapping of a single force component. Number of training epochs, batch size and dropout rate were treated as modelling hyper-parameters and their values were optimised with a grid search. A public data set of twenty-eight professional athletes containing running trails of different speeds (2.5 m/sec, 3.5 m/sec and 4.5 m/sec) was employed to train and validate the networks. Movements of the lower limbs were captured with twelve motion capture cameras and an instrumented dual-belt treadmill. The acceleration of the shanks was fed to the artificial neural networks and the estimated forces were compared to the kinetic recordings of the instrumented treadmill. Root mean square error was used to evaluate the performance of the models. Predictions were accompanied with low errors: 0.134 BW for the vertical, 0.041 BW for the anteroposterior and 0.042 BW for the mediolateral component of the force. Vertical and anteroposterior estimates were independent of running speed (p=0.233 and p=.058, respectively), while mediolateral results were significantly more accurate for low running speeds (p=0.010). The maximum force mean error between measured and estimated values was found during the vertical active peak (0.114 ± 0.088 BW). Findings indicate that artificial neural networks in conjunction with accelerometry may be used to compute three-dimensional ground reaction forces in running.

Prediksi Volume Penggunaan Air PDAM Menggunakan Metode Jaringan Syaraf Tiruan Backpropagation

As the population growth rate in Duri increases, the need for clean water also increases as needed. In Indonesia, PDAM is an institution that regulates and manages the provision of clean water for the community. So the amount of water produced and distributed should be adjusted to the demand for water. However, the problem arises in the form of waste of water at PT. PDAM Duri. Purpose of this study is to predict the amount of water consumption at PT. PDAM Duri by implementing Backpropagation Artificial Neural Network method. Variables of data taken from customer data were social, general social, household 1, household 2, household 3, commerce 1, commerce 2 and commerce 3. Data used in the prediction process was training data in 2016 and data testing in 2017. Actual amount of data at PT. PDAM Duri City 2016 until 2017 was 2.840.165 when the prediction result using artificial neural network back propagation method was 2.843.388. The number of training epochs was 4595 and the achievement of MSE (Mean Squared Error) on the test was 0,001 and the result of accuracy was 99,99900000%. Final result of this research was artificial neural network using back propagation method could predict the using of water consumption at PT. PDAM Duri for next year.

Estimation theory and Neural Networks revisited: REKF and RSVSF as optimization techniques for Deep-Learning

Deep-Learning has become a leading strategy for artificial intelligence and is being applied in many fields due to its excellent performance that has surpassed human cognitive abilities in a number of classification and control problems (Ciregan, Meier, & Schmidhuber, 2012; Mnih et al., 2015). However, the training process of Deep-Learning is usually slow and requires high-performance computing, capable of handling large datasets. The optimization of the training method can improve the learning rate of the Deep-Learning networks and result in a higher performance while using the same number of training epochs (cycles). This paper considers the use of estimation theory for training of large neural networks and in particular Deep-Learning networks. Two estimation strategies namely the Extended Kalman Filter (EKF) and the Smooth Variable Structure Filter (SVSF) have been revised (subsequently referred to as RSVSF and REKF) and used for network training. They are applied to several benchmark datasets and comparatively evaluated.

Performance of optimum neural network in rainfall–runoff modeling over a river basin

An optimum architecture backpropagation neural network for rainfall–runoff modeling is rarely visible in terms of number of input vector, hidden neurons, learning rate, momentum factor, number of epochs for training up to global minima, and error minimized at global minima. These parameters are optimized to forecast total rainfall–runoff from Basantpur station of Mahanadi river basin of central India situated at 80°28′ and 86°43′E longitudes as well as 19°8′ and 23°32′N latitudes, with a breadth 587 and 400 km as case study. Intended for four optimum models for the month of July (5, 2, 0.31, 0.72, 5,000,000, 0.001787722), August (8, 2, 0.21, 0.71, 7,500,000, 0.004556357), September (3, 2, 0.31 0.89, 20,000,000, 0.007621867), and October (8, 2, 0.24, 0.85, 7,500,000, 0.001587539) are experimentally constructed. The performance of these models with reference to correlation coefficient between real and simulated models during training and testing time is found excellent except for model of September. Wherein, correlation coefficient as 0.95, 0.71, 0.94 for the training period and 0.86, 0.53, 0.88 for the testing period are found for the corresponding models of July, August, and October, respectively. It is concluded that the optimum backpropagation neural networks are successful in rainfall–runoff modeling over a river basin for forecast of total rainfall discharge. While, for any application an optimum architecture is pre-requisite to find most favorable performance; however, it does not indicate that model will perform well has been seen in the case of September. These facts and results are discussed in this research article.

Set-Membership Type-1 Fuzzy Logic System Applied to Fault Classification in a Switch Machine

This paper focuses on the classification of faults in an electromechanical switch machine, which is an equipment used for handling railroad switches. In this paper, we introduce the use of Set-Membership concept, derived from the adaptive filter theory, into the training procedure of type-1 and singleton/non-singleton fuzzy logic systems, in order to reduce computational complexity and to increase convergence speed. We also present different criteria for using along with Set-Membership. Furthermore, we discuss the usefulness of delta rule delta, local Lipschitz estimation, variable step size, and variable step size adaptive techniques to yield additional improvement in terms of computational complexity reduction and convergence speed. Based on data set provided by a Brazilian railway company, which covers the four possible faults in a switch machine, we present performance analysis in terms of classification ratio, convergence speed, and computational complexity reduction. The reported results show that the proposed models result in improved convergence speed, slightly higher classification ratio, and remarkable computation complexity reduction when we limit the number of epochs for training, which may be required due to real-time constraint or low computational resource availability.

Artificial Neural Networks approach to pharmacokinetic model selection in DCE-MRI studies

PurposeIn pharmacokinetic analysis of Dynamic Contrast Enhanced MRI data, a descriptive physiological model should be selected properly out of a set of candidate models. Classical techniques suggested for this purpose suffer from issues like computation time and general fitting problems. This article proposes an approach based on Artificial Neural Networks (ANNs) for solving these problems. MethodsA set of three physiologically and mathematically nested models generated from the Tofts model were assumed: Model I, II and III. These models cover three possible tissue types from normal to malignant. Using 21 experimental arterial input functions and 12 levels of noise, a set of 27,216 time traces were generated. ANN was validated and optimized by the k-fold cross validation technique. An experimental dataset of 20 patients with glioblastoma was applied to ANN and the results were compared to outputs of F-test using Dice index. ResultsOptimum neuronal architecture ([6:7:1]) and number of training epochs (50) of the ANN were determined. ANN correctly classified more than 99% of the dataset. Confusion matrices for both ANN and F-test results showed the superior performance of the ANN classifier. The average Dice index (over 20 patients) indicated a 75% similarity between model selection maps of ANN and F-test. ConclusionsANN improves the model selection process by removing the need for time-consuming, problematic fitting algorithms; as well as the need for hypothesis testing.

A nonlinear system identification approach based on Fuzzy Wavelet Neural Network

This paper concerns the use of an alternative Fuzzy Wavelet Neural Network (FWNN) to model the input-output maps of nonlinear dynamic systems. The analyzed structure uses only wavelet functions in the consequent part of its fuzzy rules. The advantages and disadvantages of using this FWNN in model identification tasks are listed considering a comparative study performed with other FWNN structures found in literature. The evaluations are carried out using a real multisection liquid storage tank with abrupt transitions between its sections. The analysis is based on usual criteria such as: mean quadratic error, number of training epochs, number of adjustable parameters, quadratic error variance, among others. The results indicate that the modified FWNN structure maintains the capability of generalization and other important characteristics presented by traditional networks FWNN, despite the reduction in the complexity of the structure.

Image Classification Using Convolutional Neural Networks

Deep Learning has emerged as a new area in machine learning and is applied to a number of signal and image applications.The main purpose of the work presented in this paper, is to apply the concept of a Deep Learning algorithm namely, Convolutional neural networks (CNN) in image classification. The algorithm is tested on various standard datasets, like remote sensing data of aerial images (UC Merced Land Use Dataset) and scene images from SUN database. The performance of the algorithm is evaluated based on the quality metric known as Mean Squared Error (MSE) and classification accuracy. The graphical representation of the experimental results is given on the basis of MSE against the number of training epochs. The experimental result analysis based on the quality metrics and the graphical representation proves that the algorithm (CNN) gives fairly good classification accuracy for all the tested datasets.

Neural networks-based modeling applied to a process of heavy metals removal from wastewaters

This article approaches the problem of environment pollution with heavy metals from disposal of industrial wastewaters, namely removal of these metals by means of biosorbents, particularly with Romanian peat (from Poiana Stampei). The study is carried out by simulation using feed-forward and modular neural networks with one or two hidden layers, pursuing the influence of certain operating parameters (metal nature, sorbent dose, pH, temperature, initial concentration of metal ion, contact time) on the amount of metal ions retained on the unit mass of sorbent. In neural network modeling, a consistent data set was used, including five metals: lead, mercury, cadmium, nickel and cobalt, the quantification of the metal nature being done by its electronegativity. Even if based on successive trials, the method of designing neural models was systematically conducted, recording and comparing the errors obtained with different types of neural networks, having various numbers of hidden layers and neurons, number of training epochs, or using various learning methods. The errors with values under 5% make clear the efficiency of the applied method.

Minimising Added Classification Error Using Walsh Coefficients

Two-class supervised learning in the context of a classifier ensemble may be formulated as learning an incompletely specified Boolean function, and the associated Walsh coefficients can be estimated without the knowledge of the unspecified patterns. Using an extended version of the Tumer-Ghosh model, the relationship between added classification error and second-order Walsh coefficients is established. In this brief, the ensemble is composed of multilayer perceptron base classifiers, with the number of hidden nodes and epochs systematically varied. Experiments demonstrate that the mean second-order coefficients peak at the same number of training epochs as ensemble test error reaches a minimum.

Optimization of an Artificial Neural Network Topology for Predicting Drying Kinetics of Carrot Cubes Using Combined Response Surface and Genetic Algorithm

In this study, the advantages of integrated response surface methodology (RSM) and genetic algorithm (GA) for optimizing artificial neural network (ANN) topology of convective drying kinetic of carrot cubes were investigated. A multilayer feed-forward ANN trained by back-propagation algorithms was developed to correlate output (moisture ratio) to the four exogenous input variables (drying time, drying air temperature, air velocity, and cube size). A predictive response surface model for ANN topologies was created using RSM. The response surface model was interfaced with an effective GA to find the optimum topology of ANN. The factors considered for building a relationship of ANN topology were the number of neurons, momentum coefficient, step size, number of training epochs, and number of training runs. A second-order polynomial model was developed from training results for mean square error (MSE) of 50 developed ANNs to generate 3D response surfaces and contour plots. The optimum ANN had minimum MSE when the number of neurons, step size, momentum coefficient, number of epochs, and number of training runs were 23, 0.37, 0.68, 2,482, and 2, respectively. The results confirmed that the optimal ANN topology was more precise for predicting convective drying kinetics of carrot cubes.

Optimization of an artificial neural network topology using coupled response surface methodology and genetic algorithm for fluidized bed drying

In this study, an integrated response surface methodology (RSM) and genetic algorithm (GA) are recommended for developing artificial neural networks (ANNs) with great chances to be an optimal one. A multi-layer feed forward (MLFF) ANN was applied to correlate the outputs (energy and exergy) to the four exogenous inputs (drying time, drying air temperature, carrot cubes size, and bed depth). The RSM was used to build the relationship between the input parameters and output responses, and used as the fitness function to measure the fitness value of the GA approach. In the relationship building, five variables were used (number of neurons, momentum coefficient and step size in the hidden layer, number of epochs and number of training times). A polynomial model was developed from training results to mean square error (MSE) of 50 developed ANNs to generate 3D response surfaces and contour plots. Finally, GA was applied to find the optimal topology of ANN. The ANN topology had minimum MSE when the number of neurons in the hidden layer, momentum coefficient, step size, number of training epochs and training times were 28, 0.66, 0.35, 2877 and 3, respectively. The energy and exergy of carrot cubes during fluidized bed drying were predicted with R 2 values of greater than 0.97 using optimal ANN topology.

A New Constructive Algorithm for Architectural and Functional Adaptation of Artificial Neural Networks

The generalization ability of artificial neural networks (ANNs) is greatly dependent on their architectures. Constructive algorithms provide an attractive automatic way of determining a near-optimal ANN architecture for a given problem. Several such algorithms have been proposed in the literature and shown their effectiveness. This paper presents a new constructive algorithm (NCA) in automatically determining ANN architectures. Unlike most previous studies on determining ANN architectures, NCA puts emphasis on architectural adaptation and functional adaptation in its architecture determination process. It uses a constructive approach to determine the number of hidden layers in an ANN and of neurons in each hidden layer. To achieve functional adaptation, NCA trains hidden neurons in the ANN by using different training sets that were created by employing a similar concept used in the boosting algorithm. The purpose of using different training sets is to encourage hidden neurons to learn different parts or aspects of the training data so that the ANN can learn the whole training data in a better way. In this paper, the convergence and computational issues of NCA are analytically studied. The computational complexity of NCA is found to be O(W xP(t) xtau), where W is the number of weights in the ANN, P(t) is the number of training examples, and tau is the number of training epochs. This complexity has the same order as what the backpropagation learning algorithm requires for training a fixed ANN architecture. A set of eight classification and two approximation benchmark problems was used to evaluate the performance of NCA. The experimental results show that NCA can produce ANN architectures with fewer hidden neurons and better generalization ability compared to existing constructive and nonconstructive algorithms.

Optimization of an adaptive neural network to predict breathing

To determine the optimal configuration and performance of an adaptive feed forward neural network filter to predict breathing in respiratory motion compensation systems for external beam radiation therapy. A two-layer feed forward neural network was trained to predict future breathing amplitudes for 27 recorded breathing histories. The prediction intervals ranged from 100 to 500 ms. The optimal sampling frequency, number of input samples, training rate, and number of training epochs were determined for each breathing history and prediction interval. The overall optimal filter configuration was determined from this parameter survey, and its accuracy for each breathing example was compared to the individually optimal filter setups. Prediction accuracy was also compared to breathing stability as measured by the autocorrelation of the breathing signal. The survey of filter configurations converged on a standard setup for all examples of breathing. For 24 of the 27 breathing histories the accuracy of the standard filter for a 300 ms prediction interval was within a few percent of the individually optimized filter setups; for the remaining three histories the standard filter was 5%-15% less accurate. A standard adaptive neural network filter setup can provide approximately optimal breathing prediction for a wide range of breathing patterns. The filter accuracy has a clear correlation with the stability of breathing.

Artificial neural network for computing the resonant frequency of circular patch antennas

AbstractIn this paper, an artificial neural network (ANN) model for computing the resonant frequency of circular patch antennas (CPA) based on the feed‐forward network and back‐propagation algorithm is presented. The network is trained with the data obtained from CPA measurements. The network is trained by varying the patch radius, the height of the substrate, and the dielectric permittivity. The parameters of the proposed ANN model are the learning rate (0.0125), the number of epochs for training (399), the performance goal (1 × 10−5) and the parameters for the network, the number of hidden layers (two), with 20 neurons for each hidden layer, three neurons for the input layer, and a single neuron in the output layer. the results obtained from the trained neural network are compared with the values obtained from mathematical computations and from measurements presented in other works. The proposed ANN with two hidden layers gives accurate results and requires no additional training epochs. © 2005 Wiley Periodicals, Inc. Microwave Opt Technol Lett 47: 564–566, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.21230

A novel FOCAL technique based on BP-ANN

In this paper, the feed-forward back-propagation artificial neural network (BP-ANN) algorithm is introduced in the traditional Focus Calibration using Alignment procedure (FOCAL) technique, and a novel FOCAL technique based on BP-ANN is proposed. The effects of the parameters, such as the number of neurons on the hidden-layer and the number of training epochs, on the measurement accuracy are analyzed in detail. It is proved that the novel FOCAL technique based on BP-ANN is more reliable and it is a better choice for measurement of the image quality parameters.

Feature extraction from Doppler ultrasound signals for automated diagnostic systems

This paper presented the assessment of feature extraction methods used in automated diagnosis of arterial diseases. Since classification is more accurate when the pattern is simplified through representation by important features, feature extraction and selection play an important role in classifying systems such as neural networks. Different feature extraction methods were used to obtain feature vectors from ophthalmic and internal carotid arterial Doppler signals. In addition to this, the problem of selecting relevant features among the features available for the purpose of classification of Doppler signals was dealt with. Multilayer perceptron neural networks (MLPNNs) with different inputs (feature vectors) were used for diagnosis of ophthalmic and internal carotid arterial diseases. The assessment of feature extraction methods was performed by taking into consideration of performances of the MLPNNs. The performances of the MLPNNs were evaluated by the convergence rates (number of training epochs) and the total classification accuracies. Finally, some conclusions were drawn concerning the efficiency of discrete wavelet transform as a feature extraction method used for the diagnosis of ophthalmic and internal carotid arterial diseases.

Position‐Invariant Neural Network for Digital Pavement Crack Analysis

Abstract: This article presents an integrated neural network‐based crack imaging system to classify crack types of digital pavement images. This system includes three neural networks: (1) image‐based neural network, (2) histogram‐based neural network, and (3) proximity‐based neural network. These three neural networks were developed to classify various crack types based on the subimages (crack tiles) rather than crack pixels in digital pavement images. These spatial neural networks were trained using artificially generated data following the Federal Highway Administration (FHWA) guidelines. The optimal architecture of each neural network was determined based on the testing results from different sets of the number of hidden units, learning coefficients, and the number of training epochs. To validate the system, actual pavement pictures taken from pavements as well as the computer‐generated data were used. The proximity value is determined by computing relative distribution of crack tiles within the image. The proximity‐based neural network effectively searches the patterns of various crack types in both horizontal and vertical directions while maintaining its position invariance. The final result indicates that the proximity‐based neural network produced the best result with the accuracy of 95.2% despite its simplest neural network structure with the least computing requirement.