Neural Network Training Procedure Research Articles

SCATTERING TARGET IDENTIFICATION BASED ON RADIAL BASIS FUNCTION ARTIFICIAL NEURAL NETWORKS IN THE PRESENCE OF NON-STATIONARY NOISE

The paper deals with the radar target discrimination problem performed on complex radar images. The approach based on radial basis function (RBF) artificial neural network (ANN) is proposed for the identification of point scatterers placed within a radar image. The renewed concept of simple adaptive units as the foundation for network assembling allows one to design an ANN-based feature extraction scheme for the two-dimensional signal processing. It was shown that ANN implementing RBF neural processing units could be applied for the identification of radar targets described by the set of separated scatterers, even in cases where the relative distance between the scatterers is comparable to or less than the effective width of each scatterer. The obtained results indicate a high accuracy estimation of separate scatterer centers in the presence of noise which is not limited to the stationary case but supposed to be cyclostationary. It was also shown that the parameters describing the coordinates of scattering centers could be successfully extracted from the trained ANN after about one hundred epochs spent on ANN training process, which is carried out by means of modified gradient descent method. The main result is to demonstrate the possibility of using neural networks to automatically analyze radar images, which is an integral part of a set of tasks that form the target recognition problem. The proposed algorithm implements an approach of identification systems made using a neural network training procedures.

Multiscale topology optimization using neural network surrogate models

We are concerned with optimization of macroscale elastic structures that are designed utilizing spatially varying microscale metamaterials. The macroscale optimization is accomplished using gradient-based nonlinear topological optimization. But instead of using density as the optimization decision variable, the decision variables are the multiple parameters that define the local microscale metamaterial. This is accomplished using single layer feedforward Gaussian basis function networks as a surrogate models of the elastic response of the microscale metamaterial. The surrogate models are trained using highly resolved continuum finite element simulations of the microscale metamaterials and hence are significantly more accurate than analytical models e.g. classical beam theory. Because the derivative of the surrogate model is important for sensitivity analysis of the macroscale topology optimization, a neural network training procedure based on the Sobolev norm is described. Since the SIMP method is not appropriate for spatially varying lattices, an alternative method is developed to enable creation of void regions. The efficacy of this approach is demonstrated via several examples in which the optimal graded metamaterial outperforms a traditional solid structure.

Development of technology for state registration signs of the vehicles recognition in conditions of low resolution on the basis of gradient busting and convolutional neural networks

This paper presents the technology of characters on the vehicle license plate recognition. The Viola-Jones method is used for distinguishing the area of the license plate on the basis of pre-trained Haar cascades. Convolutional neural networks analyzing the selected frames are used for recognition. Training procedure of neural networks is carried out through the created synthetic database containing characters with various types of distortion. The efficiency of neural network training was achieved due to the possibility of a sampling formation with the almost unlimited dimension, modeling of distortions which are rarely found in a real sampling due to its limitations, the possibility of forming a mixed sampling including both synthetic and real data. The necessary methods of augmentation and preprocessing for synthetic and real data are implemented, based on the resolution of the video camera. The results obtained in this work can be used for character recognition in conditions of low resolution of the video camera, distortions and lack of initial data.

A Novel Approach to Feedback Control with Deep Reinforcement Learning

A novel deep reinforcement learning (RL) algorithm is applied for feedback control application. We propose Proximal Actor-Critic, a model-free reinforcement learning algorithm that can learn robust feedback control laws from direct interaction data from the plant. We show efficacy of the algorithm on a benchmark problem in Heating Ventilation and Air Conditioning (HVAC) heating system, with the RL controller achieving lower Integral Absolute Error (IAE) and Integral Square Error(ISE) as compared to baseline Proportional-Integral (PI) and Linear Quadratic Regulator (LQR) controllers. We also provide details on establishing feedback control problems within the deep reinforcement learning framework, including policy parameterization, neural network architecture and training procedures.

Improving deep neural networks with multi-layer maxout networks and a novel initialization method

Abstract For the purpose of enhancing the discriminability of convolutional neural networks (CNNs) and facilitating the optimization, we investigate the activation function for a neural network and the corresponding initialization method in this paper. Firstly, a trainable activation function with a multi-layer structure (named “Multi-layer Maxout Network”, MMN) is proposed. MMN is a multi-layer structured maxout, inheriting advantages of both a non-saturated activation function and a trainable activation function approximator. Secondly, we derive a robust initialization method specifically for the MMN activation with a theoretical proof, which works for the maxout activation as well. Our novel initialization method could reduce internal covariate shift when signals are propagated through layers and solve the so called “exploding/vanishing gradient” problem, which leads a more efficient training procedure of deep neural networks. Experimental results show that our proposed model yields better performance on three image classification benchmark datasets (CIFAR-10, CIFAR-100 and ImageNet) than quite a few state-of-the-art methods and our novel initialization method improves performance further. Furthermore, the influence of MMN in different hidden layers is analyzed, and a trade-off scheme between the accuracy and computing resources is given.

Using neural networks for dynamic light scattering time series processing

A basic experiment to record dynamic light scattering (DLS) time series was assembled using basic components. The DLS time series processing using the Lorentzian function fit was considered as reference. A Neural Network was designed and trained using simulated frequency spectra for spherical particles in the range 0–350 nm, assumed to be scattering centers, and the neural network design and training procedure are described in detail. The neural network output accuracy was tested both on simulated and on experimental time series. The match with the DLS results, considered as reference, was good serving as a proof of concept for using neural networks in fast DLS time series processing.

Sequential simulation and neural network in the stress–strain curve identification over the large strains using tensile test

Two alternative methods for the stress–strain curve determination in the large strains region are proposed. Only standard force–elongation response is needed as an input into the identification procedure. Both methods are applied to eight various materials, covering a broad spectre of possible ductile behaviour. The first method is based on the iterative procedure of sequential simulation of piecewise stress–strain curve using the parallel finite element modelling. Error between the computed and experimental force–elongation response is low, while the convergence rate is high. The second method uses the neural network for the stress–strain curve identification. Large database of force–elongation responses is computed by the finite element method. Then, the database is processed and reduced in order to get the input for neural network training procedure. Training process and response of network is fast compared to sequential simulation. When the desired accuracy is not reached, results can be used as a starting point for the following optimization task.

SOLAR PHOTOVOLTAIC OUTPUT POWER FORECASTING USING BACK PROPAGATION NEURAL NETWORK

Solar Energy is an important renewable and unlimited source of energy. Solar photovoltaic power forecasting, is an estimation of the expected power production, that help the grid operators to better manage the electric balance between power demand and supply. Neural network is a computational model that can predict new outcomes from past trends. The artificial neural network is used for photovoltaic plant energy forecasting. The output power for solar photovoltaic cell is predicted on hourly basis. In historical dataset collection process, two dataset was collected and used for analysis. The dataset was provided with three independent attributes and one dependent attributes. The implementation of Artificial Neural Network structure is done by Multilayer Perceptron (MLP) and training procedure for neural network is done by error Back Propagation (BP). In order to train and test the neural network, the datasets are divided in the ratio 70:30. The accuracy of prediction can be done by using various error measurement criteria and the performance of neural network is to be noted.

Cluster analysis of word frequency dynamics

This paper describes the analysis and modelling of word usage frequency time series. During one of previous studies, an assumption was put forward that all word usage frequencies have uniform dynamics approaching the shape of a Gaussian function. This assumption can be checked using the frequency dictionaries of the Google Books Ngram database. This database includes 5.2 million books published between 1500 and 2008. The corpus contains over 500 billion words in American English, British English, French, German, Spanish, Russian, Hebrew, and Chinese. We clustered time series of word usage frequencies using a Kohonen neural network. The similarity between input vectors was estimated using several algorithms. As a result of the neural network training procedure, more than ten different forms of time series were found. They describe the dynamics of word usage frequencies from birth to death of individual words. Different groups of word forms were found to have different dynamics of word usage frequency variations.

Application of Fractional Order ABC and GA for Neural Network Training and Clustering Process

AbstractLiterature presents several search algorithms to find an item with specified properties from a search space defined by a mathematical formula or procedure. One of the widely accepted algorithms is optimization algorithm which can find the optimal element within a certain period of time if the search space is defined. Recent works formulate several problems as optimization problems which were then solved by many optimization algorithms. Accordingly, in a previous paper, a hybrid optimization algorithm, called FAGA was proposed using fractional order Artificial Bee Colony (ABC) and Genetic Algorithm (GA) for optimization to solve the existing benchmark problems. In this paper, we have planned to apply the FAGA algorithm to well defined-real time problems of neural network training and the clustering process. Through neural network training, data classification will be done by making use of FAGA algorithm as neural network training procedure. Similarly, medical image segmentation will be done using c...

Feedforward Neural Network for Force Coding of an MRI-Compatible Tactile Sensor Array Based on Fiber Bragg Grating

This work shows the development and characterization of a fiber optic tactile sensor based on Fiber Bragg Grating (FBG) technology. The sensor is a 3×3 array of FBGs encapsulated in a PDMS compliant polymer. The strain experienced by each FBG is transduced into a Bragg wavelength shift and the inverse characteristics of the sensor were computed by means of a feedforward neural network. A 21 mN RMSE error was achieved in estimating the force over the 8 N experimented load range while including all probing sites in the neural network training procedure, whereas the median force RMSE was 199 mN across the 200 instances of a Monte Carlo randomized selection of experimental sessions to evaluate the calibration under generalized probing conditions. The static metrological properties and the possibility to fabricate sensors with relatively high spatial resolution make the proposed design attractive for the sensorization of robotic hands. Furthermore, the proved MRI-compatibility of the sensor opens other application scenarios, such as the possibility to employ the array for force measurement during functional MRI-measured brain activation.

Probabilistic neural network training procedure based on Q(0)-learning algorithm in medical data classification

In this article, an iterative procedure is proposed for the training process of the probabilistic neural network (PNN). In each stage of this procedure, the Q(0)-learning algorithm is utilized for the adaptation of PNN smoothing parameter (?). Four classes of PNN models are regarded in this study. In the case of the first, simplest model, the smoothing parameter takes the form of a scalar; for the second model, ? is a vector whose elements are computed with respect to the class index; the third considered model has the smoothing parameter vector for which all components are determined depending on each input attribute; finally, the last and the most complex of the analyzed networks, uses the matrix of smoothing parameters where each element is dependent on both class and input feature index. The main idea of the presented approach is based on the appropriate update of the smoothing parameter values according to the Q(0)-learning algorithm. The proposed procedure is verified on six repository data sets. The prediction ability of the algorithm is assessed by computing the test accuracy on 10 %, 20 %, 30 %, and 40 % of examples drawn randomly from each input data set. The results are compared with the test accuracy obtained by PNN trained using the conjugate gradient procedure, support vector machine algorithm, gene expression programming classifier, k---Means method, multilayer perceptron, radial basis function neural network and learning vector quantization neural network. It is shown that the presented procedure can be applied to the automatic adaptation of the smoothing parameter of each of the considered PNN models and that this is an alternative training method. PNN trained by the Q(0)-learning based approach constitutes a classifier which can be treated as one of the top models in data classification problems.

English

The objective of the present study is to describe the results of research that has been conducted on the buckling behavior of rectangular laminated orthotropic plates that have a rectangular hole. As a result of using some geometrical holes which cause stress concentrations on laminated orthotropic plates, the analysis about the buckling coefficient of the elements become very important. This work deals with buckling analysis of laminated orthotropic plates with a central rectangular hole under in-plane static loading (uniaxial compression). Initially, the critical buckling coefficients of the laminated plates are obtained by finite elements method. However, as a new study, an other numerical method is applied to this work which is called “artificial neural networks” in order to obtain those buckling coefficients. A PASCAL computer program is used for training and testing procedure of neural networks, and finally good results are achieved.   Key words: Artificial neural network, orthotropic plates, buckling analysis.

Experimental calibration of forward and inverse neural networks for rotary type magnetorheological damper

This paper presents a systematic design and training procedure for the feed-forward back-propagation neural network (NN) modeling of both forward and inverse behavior of a rotary magnetorheological (MR) damper based on experimental data. For the forward damper model, with damper force as output, an optimization procedure demonstrates accurate training of the NN architecture with only current and velocity as input states. For the inverse damper model, with current as output, the absolute value of velocity and force are used as input states to avoid negative current spikes when tracking a desired damper force. The forward and inverse damper models are trained and validated experimentally, combining a limited number of harmonic displacement records, and constant and half-sinusoidal current records. In general the validation shows accurate results for both forward and inverse damper models, where the observed modeling errors for the inverse model can be related to knocking effects in the measured force due to the bearing plays between hydraulic piston and MR damper rod. Finally, the validated models are used to emulate pure viscous damping. Comparison of numerical and experimental results demonstrates good agreement in the post-yield region of the MR damper, while the main error of the inverse NN occurs in the pre-yield region where the inverse NN overestimates the current to track the desired viscous force.

Bird strike event monitoring in a composite UAV wing using high speed optical fiber sensing system

In this study, high speed bird strikes on a composite structure were successfully monitored using optical fiber sensors. Four multiplexed optical fiber sensors in a single cable were surface-bonded on the leading edge of a composite UAV wing box. In order to acquire those high frequency signals, a newly developed interrogation system was used to process strain signals from four sensors simultaneously at a sampling frequency of 100kHz. Before the bird strike tests, pre-impact tests using a rubber hammer were performed to verify the suitability of the FBG signal acquisitions. The pre-test data were used in the neural network training procedures to estimate the bird strike locations. Then, the bird strike tests were accomplished using dummy projectiles and a pneumatic gun. The one-pound dummy birds, made of gelatin, hit the leading edge with a maximum speed of 201km/h. The impact signals were successfully recorded during the tests and their frequency characteristics were then analyzed. Finally, the strike locations were estimated with the neural network which was trained through the pre-tests. The average error was 33.6mm.

Mobility Timing for Agent Communities, a Cue for Advanced Connectionist Systems

We introduce a wait-and-chase scheme that models the contact times between moving agents within a connectionist construct. The idea that elementary processors move within a network to get a proper position is borne out both by biological neurons in the brain morphogenesis and by agents within social networks. From the former, we take inspiration to devise a medium-term project for new artificial neural network training procedures where mobile neurons exchange data only when they are close to one another in a proper space (are in contact). From the latter, we accumulate mobility tracks experience. We focus on the preliminary step of characterizing the elapsed time between neuron contacts, which results from a spatial process fitting in the family of random processes with memory, where chasing neurons are stochastically driven by the goal of hitting target neurons. Thus, we add an unprecedented mobility model to the literature in the field, introducing a distribution law of the intercontact times that merges features of both negative exponential and Pareto distribution laws. We give a constructive description and implementation of our model, as well as a short analytical form whose parameters are suitably estimated in terms of confidence intervals from experimental data. Numerical experiments show the model and related inference tools to be sufficiently robust to cope with two main requisites for its exploitation in a neural network: the nonindependence of the observed intercontact times and the feasibility of the model inversion problem to infer suitable mobility parameters.

A neural network approach for the prediction of the refractive index based on experimental data

This article presents a systematic approach for correlating the refractive index of different material kinds and forms with experimentally measured inputs like wavelength, temperature, and concentration. The correlation is accomplished using neural network models, which can deal effectively with the nonlinear nature of the problem without requiring a predefined form of equation, while taking into account all the parameters affecting the refractive index. The proposed methodology employs the powerful radial basis function network architecture and the neural network training procedure is accomplished using an innovative algorithm, which provides results with increased prediction accuracy. The methodology is applied to two cases, involving the estimation of the refractive index of semiconductor material crystals and an ethanol–water mixture and the results show that the refractive index predictions are accurate approximately to the same number of decimal places as the real measurements. Comparisons with other neural network training methods, but also with empirical forms like the Sellmeier equation, highlight the superiority of the proposed approach.

γ-turn types prediction in proteins using the two-stage hybrid neural discriminant model

Due to the slightly success of protein secondary structure prediction using the various algorithmic and non-algorithmic techniques, similar techniques have been developed for predicting γ-turns in proteins by Kaur and Raghava [2003. A neural-network based method for prediction of γ-turns in proteins from multiple sequence alignment. Protein Sci. 12, 923–929]. However, the major limitation of previous methods was inability in predicting γ-turn types. In a recent investigation we introduced a sequence based predictor model for predicting γ-turn types in proteins [Jahandideh, S., Sabet Sarvestani, A., Abdolmaleki, P., Jahandideh, M., Barfeie, M, 2007a. γ-turn types prediction in proteins using the support vector machines. J. Theor. Biol. 249, 785–790]. In the present work, in order to analyze the effect of sequence and structure in the formation of γ-turn types and predicting γ-turn types in proteins, we applied novel hybrid neural discriminant modeling procedure. As the result, this study clarified the efficiency of using the statistical model preprocessors in determining the effective parameters. Moreover, the optimal structure of neural network can be simplified by a preprocessor in the first stage of hybrid approach, thereby reducing the needed time for neural network training procedure in the second stage and the probability of overfitting occurrence decreased and a high precision and reliability obtained in this way.

Structural features of diverse ligands influencing binding affinities to estrogen α and estrogen β receptors. Part II. Molecular descriptors calculated from conformation of the ligands in the complex resulting from previous docking study

A QSAR study is reported, in which the relationship between chemical structure of a set of compounds and the binding affinity to human estrogen receptor alpha and beta (ER-alpha and ER-beta) is modelled. Counterpropagation neural networks are used to predict experimental binding affinity of a range of substances. Several compounds as estrogenic chemicals, phytoestrogens, and natural and synthetic estrogens are treated with a structure-based approach that involves the protein structure. The conformations obtained with a docking methodology are used to calculate molecular descriptors. The models are built up with the neural network training procedure, which encodes the information present in molecular descriptors and related binding affinities of the pre-selected training set of compounds. In order to reach the best possible models, a selection of the descriptors using genetic algorithm was conducted. The selection was directed by the error in the prediction of binding affinities of compounds from the test set. The final models obtained for estrogen receptor alpha and beta were tested with an external validation set and were compared with the models obtained from a receptor-independent approach reported in the accompanying paper.

Mathematical modeling of batch and semibatch reactors for the enzymic synthesis of amoxicillin

The performance of batch and semibatch reactors, under optimal operational conditions of amoxicillin (AN) enzymic synthesis at 25 °C and pH 6.5, were compared. Simulations for a batch reactor indicate that there are two sets of optimal operational conditions, if we consider productivity and global yield ( Y AN, AB) with respect to the ester substrate: (i) low initial concentration of ester and high initial concentration of 6-aminopenicillanic acid (6-APA); (ii) high initial concentration of ester and high initial concentration of 6-APA. Simulations for a semibatch reactor, using the hybrid-neural network model, indicated a consumption of the formed antibiotic. Such pattern was not observed experimentally and was caused by the lack of information during neural network training procedure. This problem was overcome by including new sets of experimental data in the learning phase, performed in a batch reactor in the presence of different initial concentrations of substrates and products. Results for the semibatch reactor indicated there are different sets of optimal operational conditions, if we consider local selectivity, performance index ( I NH), and global yield. Unfortunately, the point of highest local selectivity and performance index is not the one of highest global yield. Therefore, the choice of initial operational conditions for the batch or semibatch production of amoxicillin is connected to an economical evaluation. The semi-continuous operation of the integrated reactor (reaction+crystallization) provided better results than the batch mode.