Kind Of Neural Network Research Articles

Exploration on Korean-Chinese collaborative translation method based on recursive recurrent neural network

In order to realize the historical sequence establishment of abstract dynamics in the cooperative translation of Korean language under machine learning and generate an abstract representation dynamically of the translation decoding tree in the recursive model during translation decoding. Combining the advantages of the two kinds of neural networks, this paper constructs a recursive recurrent neural network model, which can not only model the translation process by using the traditional machine translation features but also gradually construct the abstract representation of translation candidates in the process of translation, mining the important language model and other global features in machine translation effectively. This paper has trained the number of Korean-Chinese translation vocabulary, sentence length, and language pairs. Based on the test results, the model can effectively improve the performance of the machine translation model. In addition, based on the adjustment of pre-order word order to optimized the recursive recurring neural network model, and improved the performance of machine translation significantly.

Recurrent Neural Network for Predicting Dielectric Mirror Reflectivity

Optical devices often achieve their maximum effectiveness by using dielectric mirrors; however, their design techniques depend on expert knowledge in specifying the mirror properties. This expertise can also be achieved by machine learning, although it is not clear what kind of neural network would be effective for learning about dielectric mirrors. In this paper, we clarify that the recurrent neural network (RNN) is an effective approach to machine-learning for dielectric mirror properties. The relation between the thickness distribution of the mirror’s multiple film layers and the average reflectivity in the target wavelength region is used as the indicator in this study. Reflection from the dielectric multilayer film results from the sequence of interfering reflections from the boundaries between film layers. Therefore, the RNN, which is usually used for sequential data, is effective to learn the relationship between average reflectivity and the thickness of individual film layers in a dielectric mirror. We found that a RNN can predict its average reflectivity with a mean squared error (MSE) less than 10-4 from representative thickness distribution data (10 layers with alternating refractive indexes 2.3 and 1.4). Furthermore, we clarified that training data sets generated randomly lead to over-learning. It is necessary to generate training data sets from larger data sets so that the histogram of reflectivity becomes a flat distribution. In the future, we plan to apply this knowledge to design dielectric mirrors using neural network approaches such as generative adversarial networks, which do not require the know-how of experts.

An Octahedric Regression Model of Energy Efficiency on Residential Buildings

System modeling is a main task in several research fields. The development of numerical models is of crucial importance at the present because of its wide use in the applications of the generically named machine learning technology, including different kinds of neural networks, random field models, and kernel-based methodologies. However, some problems involving the reliability of their predictions are common to their use in the real world. Octahedric regression is a kernel averaged methodology developed by the authors that tries to simplify the entire process from raw data acquisition to model generation. A discussion about the treatment and prevention of overfitting is presented and, as a result, models are obtained that allow for the measurement of this effect. In this paper, this methodology is applied to the problem of estimating the energetic needs of different buildings according to their principal characteristics, a problem that has importance in architecture and civil and environmental engineering due to increasing concerns about energetic efficiency and ecological footprint.

On the Working Principle of the Hopfield Neural Networks and Its Equivalence to the GADIA in Optimization.

Hopfield neural networks (HNNs) are one of the most well-known and widely used kinds of neural networks in optimization. In this article, the author focuses on building a deeper understanding of the working principle of the HNN during an optimization process. Our investigations yield several novel results giving some important insights into the working principle of both continuous and discrete HNNs. This article shows that what the traditional HNN actually does as energy function decreases is to divide the neurons into two classes in such a way that the sum of biased class volumes is minimized (or maximized) regardless of the types of the optimization problems. Introducing neuron-specific class labels, the author concludes that the traditional discrete HNN is actually a special case of the greedy asynchronous distributed interference avoidance algorithm (GADIA) [17] of Babadi and Tarokh for the 2-class optimization problems. The computer results confirm the findings.

Kinematics model identification and motion control of robot based on fast learning neural network

As a static network, the slow learning of back propagation (BP) neural network is an insurmountable disadvantage facing dynamic system identification. In contrast, dynamic neural network provides a potential choice, representing the development direction of neural network modeling, identification and control. Based on dynamic Elman network, a new learning neural network structure, called fast learning neural network, is proposed in this paper. It not only conforms to the basic characteristics of biological neural network, but also has the advantages of simple algorithm, fast learning convergence and high identification accuracy of linear and non-linear systems. Based on the experimental data, this network is applied to the identification of the motion model of modular robots to obtain the non-linear kinematics model of robots. Therefore, this kind of neural network is very suitable for robot kinematics model identification and motion control. The simulation results show that it is appropriate to identify the kinematics model and control the motion of the robot based on fast learning neural network; and the combination of fast learning and ELM network speeds up the training efficiency.

Applications of neural networks to dynamics simulation of Landau-Zener transitions

We simulate the dynamics of a qubit-oscillator system obeying the Landau-Zener (LZ) model, by employing the nonlinear autoregressive neural network and the long short-term memory neural network. Initially, time-dependent transition probability of the LZ model is obtained by the Dirac-Frenkel time dependent variation with the multiple Davydov D2 Ansatz. With the first stage of a two-dimensional (2D) dataset (time versus transition probability), two different kinds of neural networks are trained and validated successfully with sufficient information to predict the future values of transition probability (the second stage) with considerable accuracy. Furthermore, we also develop a framework under which an entire time series of a LZ model with fixed tunneling strength Δ and a given qubit-bath coupling strength γ can be predicted, using neural networks that are trained on a set of pre-generated time series corresponding to various values of γ (3D data: time, γ and transition probability). Considerable accuracy is also achieved in 3D data prediction.

Classification of alkaloids according to the starting substances of their biosynthetic pathways using graph convolutional neural networks

BackgroundAlkaloids, a class of organic compounds that contain nitrogen bases, are mainly synthesized as secondary metabolites in plants and fungi, and they have a wide range of bioactivities. Although there are thousands of compounds in this class, few of their biosynthesis pathways are fully identified. In this study, we constructed a model to predict their precursors based on a novel kind of neural network called the molecular graph convolutional neural network. Molecular similarity is a crucial metric in the analysis of qualitative structure–activity relationships. However, it is sometimes difficult for current fingerprint representations to emphasize specific features for the target problems efficiently. It is advantageous to allow the model to select the appropriate features according to data-driven decisions for extracting more useful information, which influences a classification or regression problem substantially.ResultsIn this study, we applied a neural network architecture for undirected graph representation of molecules. By encoding a molecule as an abstract graph and applying "convolution" on the graph and training the weight of the neural network framework, the neural network can optimize feature selection for the training problem. By incorporating the effects from adjacent atoms recursively, graph convolutional neural networks can extract the features of latent atoms that represent chemical features of a molecule efficiently. In order to investigate alkaloid biosynthesis, we trained the network to distinguish the precursors of 566 alkaloids, which are almost all of the alkaloids whose biosynthesis pathways are known, and showed that the model could predict starting substances with an averaged accuracy of 97.5%.ConclusionWe have showed that our model can predict more accurately compared to the random forest and general neural network when the variables and fingerprints are not selected, while the performance is comparable when we carefully select 507 variables from 18000 dimensions of descriptors. The prediction of pathways contributes to understanding of alkaloid synthesis mechanisms and the application of graph based neural network models to similar problems in bioinformatics would therefore be beneficial. We applied our model to evaluate the precursors of biosynthesis of 12000 alkaloids found in various organisms and found power-low-like distribution.

A Study of the Chinese spam Classification with Doc2vec and CNN

Convolution neural network is a kind of neural network, which has been proved to be very effective in image recognition and classification. In recent years, convolution neural networks have gradually shifted to the field of natural language processing and become one of the research hotspots. For the construction of word vector text using convolution neural network, only considering the relationship between word granularity level, not considering the relationship between words, nor considering the relationship between semantics, affecting the classification results. In this paper, a method based on Doc2vec and CNN is proposed to classify spam. Firstly, the spam is preprocessed, then the sentence vectors and word vectors of Chinese text are trained by Doc2vec, and finally the trained text vectors are classified by convolution neural network.

A Blind Spectrum Sensing Method Based on Deep Learning.

Spectrum sensing is one of the technologies that is used to solve the current problem of low utilization of spectrum resources. However, when the signal-to-noise ratio is low, current spectrum sensing methods cannot well-handle a situation in which the prior information of the licensed user signal is lacking. In this paper, a blind spectrum sensing method based on deep learning is proposed that uses three kinds of neural networks together, namely convolutional neural networks, long short-term memory, and fully connected neural networks. Experiments show that the proposed method has better performance than an energy detector, especially when the signal-to-noise ratio is low. At the same time, this paper also analyzes the effect of different long short-term memory layers on detection performance, and explores why the deep-learning-based detector can achieve better performance.

Semi-wrapper feature subset selector for feed-forward neural networks: Applications to binary and multi-class classification problems

This paper explores widely the data preparation stage within the process of knowledge discovery and data mining via feature subset selection in the context of two very well-known neural models: radial basis function neural networks and multi-layer perceptron. It is known the best performance of wrapper attribute selection methods based on the evaluation measure provided by a classifier, although the temporal complexity of learning neural networks practically precludes the use of wrapper techniques, especially in complex databases with high dimensionality and a large number of labels. In this paper, we propose the use of the Naïve Bayes classifier as a fitness function within a semi-wrapper feature selection approach. The Naïve Bayes classifier is a good fast approach to a neural network and utilising it as a measure of goodness in a backward search on a ranking provides a specific attribute selection method for neural networks in complex data. The test-bed consists of 34 binary and multi-class classification problems and 7 feature selectors. Of these, there are 6 data sets with upwards of 5 classes. According to the reported accuracy results that have been supported by non-parametric statistical tests in different scenarios, our method has been shown to be very suitable for both kinds of neural networks. Moreover, the reduced feature-space is around 20% of the full attribute space. The speedup with the aforementioned semi-wrapper is very outstanding and its value fluctuates, on average, from about 1.5 with radial basis function neural networks to around 30 with multi-layer perceptron.

Gait-Based Diplegia Classification Using LSMT Networks.

Diplegia is a specific subcategory of the wide spectrum of motion disorders gathered under the name of cerebral palsy. Recent works proposed to use gait analysis for diplegia classification paving the way for automated analysis. A clinically established gait-based classification system divides diplegic patients into 4 main forms, each one associated with a peculiar walking pattern. In this work, we apply two different deep learning techniques, namely, multilayer perceptron and recurrent neural networks, to automatically classify children into the 4 clinical forms. For the analysis, we used a dataset comprising gait data of 174 patients collected by means of an optoelectronic system. The measurements describing walking patterns have been processed to extract 27 angular parameters and then used to train both kinds of neural networks. Classification results are comparable with those provided by experts in 3 out of 4 forms.

Stream Flow Forecasting in Mahanadi River Basin using Artificial Neural Networks

Suspended sediment prediction is essential for wearing awayof soil and sedimentation at time of peakinundation period. Several kinds of neural networks have been projected in preceding papers for applications in hydrological events. This research intelligence focuses on assessment of applicability of Recurrent Neural Network (RNN) and Radial Basis Function Network (RBFN) for forecasting flow on daily basis at gauging station in Mahanadi river basin. Superlative performance is assessed on basis of RMSE, R2 which was attained by the precipitation and antecedent flow as model inputs. Based on performance value RNN gives prominent value as compare to RBFN. In case of RBFN three transfer function Tan-sig, Log-sig, purelin are utilized for evaluating model performance. While Tan-sig function used model gives extent value of R2 that is0.9614 and 0.9843 for training and testing phase respectively.

Recurrent neural networks as approximators of non-linear filters operators

In cases that the mathematical model of a device is complicated or it can not be constructed because of the absence of sufficient information about a researched object, the approach based on the description of unique relationship between the sets of input and output signals is used. The point is the approximation of a non-linear operator, establishing the unique input-output mapping, by mathematical constructions (multidimensional polynomials, regression models, neural networks). Demand to reach the high accuracy of approximation often arises in practice. Recurrent neural networks possessing the properties of dynamics and nonlinearity are considered as mathematical models within the framework of the input-output approach. The non-linear filters synthesis is the constructing of mathematical models using the available sets of input and output signals. Non-linear filters serve for cancelling non-Gaussian noise from distorted signals. As an example, images distorted by the impulse noise is recovered with the help of non-linear filtering performed by different kinds of neural networks.

Determination of Shear Strength Parameters of Municipal Solid Waste from Its Physical Properties

In this research, artificial neural networks (ANNs) were used to present models for predicting the shear strength parameters of the municipal solid waste (MSW) obtained from the triaxial tests results from the literature. Two kinds of neural networks were used in this paper, including radial basis function (RBF) and multilayer perceptron (MLP). Physical properties of MSW, including the fiber content, moisture content, dry unit weight and axial strain, were used as the input data and the drained MSW shear strength parameters; that is, cohesion and friction angle were considered as the output data. In this paper, 80 and 90% of the total set of data were used for training of the networks and then they were tested for the total set of data (100% of the data) to obtain the best neural network model and to find out the effect of learning for prediction of both sets of data (learning and testing set of data). The results of the correlation coefficient parameter lead us to the fact that the networks with similar geometry, but more training data, predict more accurately. This value for the MLP with 80% training data is the minimum and equal to 0.95 and 0.86 for the cohesion and friction angle, respectively. It is maximum for RBF network with 90% training data and equal to 0.97 and 0.89 for the cohesion and friction angle, respectively. In addition, the RBF networks predict less mean relative values relative to the MLP networks. Among the different networks, mean relative error for RBF network with 90% training data is minimum and equal to 7.4 for the cohesion and 8 for the friction angle, respectively. Based on the analysis of the statistical parameters, the RBF network predicts more accurately. Results show that the ANNs are powerful tools for predicting the shear strength parameters of MSW.

Comparison of different wind data interpolation methods for a region with complex terrain in Central Asia

While climatological data of high spatial resolution are largely available in most developed countries, the network of climatological stations in many other regions of the world still constitutes large gaps. Especially for those regions, interpolation methods are important tools to fill these gaps and to improve the data base indispensible for climatological research. Over the last years, new hybrid methods of machine learning and geostatistics have been developed which provide innovative prospects in spatial predictive modelling. This study will focus on evaluating the performance of 12 different interpolation methods for the wind components $$\overrightarrow{u}$$ and $$\overrightarrow{v}$$ in a mountainous region of Central Asia. Thereby, a special focus will be on applying new hybrid methods on spatial interpolation of wind data. This study is the first evaluating and comparing the performance of several of these hybrid methods. The overall aim of this study is to determine whether an optimal interpolation method exists, which can equally be applied for all pressure levels, or whether different interpolation methods have to be used for the different pressure levels. Deterministic (inverse distance weighting) and geostatistical interpolation methods (ordinary kriging) were explored, which take into account only the initial values of $$\overrightarrow{u}$$ and $$\overrightarrow{v}$$ . In addition, more complex methods (generalized additive model, support vector machine and neural networks as single methods and as hybrid methods as well as regression-kriging) that consider additional variables were applied. The analysis of the error indices revealed that regression-kriging provided the most accurate interpolation results for both wind components and all pressure heights. At 200 and 500 hPa, regression-kriging is followed by the different kinds of neural networks and support vector machines and for 850 hPa it is followed by the different types of support vector machine and ordinary kriging. Overall, explanatory variables improve the interpolation results.

Event-driven optimal control for uncertain nonlinear systems with external disturbance via adaptive dynamic programming

Abstract This paper investigates the optimal control of uncertain nonlinear systems with external disturbance via adaptive dynamic programming, where two kinds of controllers are introduced and event-driven strategy is used to update the controllers as well as the network weights. One controller is constrained-input, and another is used to offset the external disturbance. Three kinds of neural networks (identifier network, critic network and actor network) are used to approximate the uncertain dynamics, the optimal value, control inputs and external disturbance, respectively. To the best knowledge of the authors, the advantage of this paper is that almost all the existing systems about optimal control via adaptive dynamic programming can be considered as special cases of this system, where the external disturbance is considered. By designing appropriate parameters, the uncertain nonlinear system is asymptotically stable with the event-driven controller. And, under the given learning rates of neural networks, the system state and the estimation error of neural networks are proved to be uniform ultimate boundedness (UUB) via adaptive dynamic programming. Finally, numerical simulation results are presented to verify the theoretical analysis.

Machine learning in pain research.

Machine learning in pain research.

On the use of back propagation and radial basis function neural networks in surface roughness prediction

Various artificial neural networks types are examined and compared for the prediction of surface roughness in manufacturing technology. The aim of the study is to evaluate different kinds of neural networks and observe their performance and applicability on the same problem. More specifically, feed-forward artificial neural networks are trained with three different back propagation algorithms, namely the adaptive back propagation algorithm of the steepest descent with the use of momentum term, the back propagation Levenberg–Marquardt algorithm and the back propagation Bayesian algorithm. Moreover, radial basis function neural networks are examined. All the aforementioned algorithms are used for the prediction of surface roughness in milling, trained with the same input parameters and output data so that they can be compared. The advantages and disadvantages, in terms of the quality of the results, computational cost and time are identified. An algorithm for the selection of the spread constant is applied and tests are performed for the determination of the neural network with the best performance. The finally selected neural networks can satisfactorily predict the quality of the manufacturing process performed, through simulation and input–output surfaces for combinations of the input data, which correspond to milling cutting conditions.

Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene.

Hierarchical neural networks have been shown to be effective in learning representative image features and recognizing object classes. However, most existing networks combine the low/middle level cues for classification without accounting for any spatial structures. For applications such as understanding a scene, how the visual cues are spatially distributed in an image becomes essential for successful analysis. This paper extends the framework of deep neural networks by accounting for the structural cues in the visual signals. In particular, two kinds of neural networks have been proposed. First, we develop a multitask deep convolutional network, which simultaneously detects the presence of the target and the geometric attributes (location and orientation) of the target with respect to the region of interest. Second, a recurrent neuron layer is adopted for structured visual detection. The recurrent neurons can deal with the spatial distribution of visible cues belonging to an object whose shape or structure is difficult to explicitly define. Both the networks are demonstrated by the practical task of detecting lane boundaries in traffic scenes. The multitask convolutional neural network provides auxiliary geometric information to help the subsequent modeling of the given lane structures. The recurrent neural network automatically detects lane boundaries, including those areas containing no marks, without any explicit prior knowledge or secondary modeling.

Are Evolutionary Algorithms Effective in Calibrating Different Artificial Neural Network Types for Streamwater Temperature Prediction?

Streamwater temperature may be severely affected by the global warming. Different types of models could be used to evaluate the regime of water temperatures in future climatic conditions, including artificial neural networks. As neural networks have no physical background, they require calibration of large number of parameters. This is typically done by gradient-based algorithms, however there is an ongoing debate on usefulness of metaheuristics for this task. In this paper more than ten Swarm Intelligence and Evolutionary Algorithms, including one developed especially for this study, are tested to train four kinds of artificial neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) for daily water temperature prediction in a natural river located in temperate climate zone. The results are compared with the ones obtained when the classical Levenberg-Marquardt algorithm is used. Finally, the ensemble aggregating approach is tested. Although the research confirms that most metaheuristics do not suite well for training any kind of neural networks, there are exceptions that include the newly proposed heuristic. However, the gain achieved when using even the best metaheuristics is low, comparing to the effort (computational time and complexity of such algorithms). Using ensemble aggregation approach seems to have higher impact on the model performance than seeking for new training methods.