Ordinary Neural Network Research Articles

Study on estimating quantum discord by neural network with prior knowledge

Machine learning has achieved success in many areas because of its powerful fitting ability, so we hope it can help us to solve some significant physical quantitative problems, such as quantum correlation. In this research, we will use neural networks to predict the value of quantum discord. Quantum discord is a measure of quantum correlation which is defined as the difference between quantum mutual information and classical correlation for a bipartite system. Since the definition contains an optimization term, it makes analytically solving hard. For some special cases and small systems, such as two-qubit systems and some X-states, the explicit solutions have been calculated. However, for general cases, we still know very little. Therefore, we study the feasibility of estimating quantum discord by machine learning method on two-qubit systems. In order to get an interpretable and high-performance model, we modify the ordinary neural network by introducing some prior knowledge which comes from the analysis about quantum discord. Our results show that prior knowledge actually improves the performance of neural network.

High-Order Neural Networks are Equivalent to Ordinary Neural Networks

Equivalence of computational systems can assist in obtaining abstract systems, and thus enable better understanding of issues related their design and performance. For more than four decades, artificial neural networks have been used in many scientific applications to solve classification problems as well as other problems. Since the time of their introduction, multilayer feedforward neural network referred as Ordinary Neural Network (ONN), that contains only summation activation (Sigma) neurons, and multilayer feedforward High-order Neural Network (HONN), that contains Sigma neurons, and product activation (Pi) neurons, have been treated in the literature as different entities. In this work, we studied whether HONNs are mathematically equivalent to ONNs. We have proved that every HONN could be converted to some equivalent ONN. In most cases, one just needs to modify the neuronal transfer function of the Pi neuron to convert it to a Sigma neuron. The theorems that we have derived clearly show that the original HONN and its corresponding equivalent ONN would give exactly the same output, which means; they can both be used to perform exactly the same functionality. We also derived equivalence theorems for several other non-standard neural networks, for example, recurrent HONNs and HONNs with translated multiplicative neurons. This work rejects the hypothesis that HONNs and ONNs are different entities, a conclusion that might initiate a new research frontier in artificial neural network research.

Neural processor in artificial intelligence advancement

A neuron network is a computational model based on structure and functions of biological neural networks. Information that flows through the network affects the structure of the neuron network because neural network changes-or learns, in a sense-based on that input and output. Although neural network being highly complex (for example change of weights for every new data within the time frame) an experimental model of high level architecture of neural processor is proposed. Neural Processor performs all the functions that an ordinary neural network does like adaptive learning, self-organization, real time operations and fault tolerance. In this paper, analysis of neural processing is discussed and presented with experiments, graphical representation including data analysis.

Two-Stage Approach to Image Classification by Deep Neural Networks

The paper demonstrates the advantages of the deep learning networks over the ordinary neural networks on their comparative applications to image classifying. An autoassociative neural network is used as a standalone autoencoder for prior extraction of the most informative features of the input data for neural networks to be compared further as classifiers. The main efforts to deal with deep learning networks are spent for a quite painstaking work of optimizing the structures of those networks and their components, as activation functions, weights, as well as the procedures of minimizing their loss function to improve their performances and speed up their learning time. It is also shown that the deep autoencoders develop the remarkable ability for denoising images after being specially trained. Convolutional Neural Networks are also used to solve a quite actual problem of protein genetics on the example of the durum wheat classification. Results of our comparative study demonstrate the undoubted advantage of the deep networks, as well as the denoising power of the autoencoders. In our work we use both GPU and cloud services to speed up the calculations.

Symmetric Complex-Valued Hopfield Neural Networks.

Complex-valued neural networks, which are extensions of ordinary neural networks, have been studied as interesting models by many researchers. Especially, complex-valued Hopfield neural networks (CHNNs) have been used to process multilevel data, such as gray-scale images. CHNNs with Hermitian connection weights always converge using asynchronous update. The noise tolerance of CHNNs deteriorates extremely as the resolution increases. Noise tolerance is one of the most controversial problems for CHNNs. It is known that rotational invariance reduces noise tolerance. In this brief, we propose symmetric CHNNs (SCHNNs), which have symmetric connection weights. We define their energy function and prove that the SCHNNs always converge. In addition, we show that the SCHNNs improve noise tolerance through computer simulations and explain this improvement from the standpoint of rotational invariance.

Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications

This paper presents a novel neural network having variable weights, which is able to improve its learning and generalisation capabilities, to deal with classification problems. The variable weight neural network (VWNN) allows its weights to be changed in operation according to the characteristic of the network inputs so that it can adapt to different characteristics of input data resulting in better performance compared with ordinary neural networks with fixed weights. The effectiveness of the VWNN is tested with the consideration of two real-life applications. The first application is on the classification of materials using the data collected by a robot finger with tactile sensors sliding along the surface of a given material. The second application considers the classification of seizure phases of epilepsy (seizure-free, pre-seizure and seizure phases) using real clinical data. Comparisons are performed with some traditional classification methods including neural network, k-nearest neighbours and naive Bayes classification techniques. It is shown that the VWNN classifier outperforms the traditional methods in terms of classification accuracy and robustness property when the input data is contaminated with noise.

ADAPTIVE SELECTION OF NEURAL NETWORKS FOR A COMMITTEE DECISION

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilising all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on three well known real data sets and also applied to fault identification of the actuator valve at one sugar factory within the DAMADICS RTN.

Simulation and Research of Boiler Combustion Process Based On the Improved RBF Neural Network

Due to the use of time, machine wear degree, coal and other reasons, the original set parameters of the boiler have been unable to meet the control requirements, therefore using a large amount of data to build a real model of the power station based on the neural network, therefore, to establish a boiler combustion optimization neural network model by using of the power plant operating data. According to the shortcomings on RBF neural networks traditional training methods with slow convergence speed, easy to fall into the local minimum. Firstly, this paper Set the model to single input and single output system as the research object, optimize neural network by the particle swarm optimization algorithm. Finally, this modeling method is expanded to the multiple input multiple output system field. Use MATLAB to establish the simulation model and the simulation research, the simulation results show that improved method for combustion boiler system efficiency has been significantly improved, combustion efficiency of the entire system reached 94%, the accuracy of the system model was significantly better than ordinary neural network, system training error controls in less than 5% .We can see that the improved method is feasible and effective.

Twisted quaternary neural networks

AbstractThe quaternary neural network (QNN) proposed by Nitta is a high‐dimensional neural network. Nitta showed that its learning is faster than that of ordinary neural networks and the number of required parameters is almost one‐third of that of real‐valued neural networks by computer simulations. In this paper, we propose the twisted quaternary neural network (TQNN) which modifies the directions of multiplications of the QNN. Since quaternions are noncommutative on multiplication, we can get another neural network. When the activation function is linear, multilayered neural networks can be expressed by single‐layered neural networks. But the TQNN cannot be expressed by a single‐layered QNN even if the activation function is linear. Therefore, the TQNN is expected to produce a variety of signal‐processing systems. We performed computer simulations to compare the QNN and the TQNN. Then we found that the latter's learning is a little faster. Moreover, computer simulation showed that the QNN tended to be trapped in local minima or plateaus but the TQNN did not. It is said that reducibility causes local minima and plateaus. We discuss the differences of reducibility between the QNN and the TQNN as well. © 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Performance analysis of Bayesian networks and neural networks in classification of file system activities

Precise comprehension of a file system state at any given time is vital for performing digital forensic analyses. To uncover evidence of the digital crime, the logical representation of file system activities helps reconstruct post-event timeline of the unauthorized or malicious accesses made on a system. This paper describes a comparative performance analysis of the Bayesian networks and neural networks techniques to classify the state of file system activities in terms of execution of applications based on the pattern of manipulation of specific files during certain period of time. In particular, this paper discusses the construction of a Bayesian networks and neural networks from the predetermined knowledge of the manipulation of file system artifacts and their corresponding metadata information by a set of software applications. The variability amongst the execution patterns of various applications indicate that the Bayesian network-based model is a more appropriate tool as compared to neural networks because of its ability to learn and detect patterns even from an incomplete dataset. The focus of this paper is to highlight intrinsic significance of the learning approach of Bayesian network methodology in comparison to the techniques used for supervised learning in ordinary neural networks. The paper also highlights the efficacy of Bayesian network technique to proficiently handle large volumes of datasets.

Identification of thermal error in a feed system based on multi-class LS-SVM

Research of thermal characteristics has been a key issue in the development of high-speed feed system. The thermal positioning error of a ball-screw is one of the most important objects to consider for high-accuracy and high-speed machine tools. The research work undertaken herein ultimately aims at the development of a comprehensive thermal error identification model with high accuracy and robust. Using multi-class least squares support vector machines (LS-SVM), the thermal positioning error of the feed system is identified with the variance and mean square value of the temperatures of supporting bearings and screw-nut as feature vector. A series of experiments were carried out on a self-made quasi highspeed feed system experimental bench HUST-FS-001 to verify the identification capacity of the presented method. The results show that the recommended model can be used to predict the thermal error of a feed system with good accuracy, which is better than the ordinary BP and RBF neural network. The work described in this paper lays a solid foundation of thermal error prediction and compensation in a feed system.

Wind Speed Forecast for Wind Farms Based on Phase Space Reconstruction of Wavelet Neural Network

Wind speed forecast is a non-linear and non-smooth problem. nonlinear and non-stationary are two kinds of mathematical problem, it is difficult to model with a single method, so that, a wavelet neural network model is set, the non-linear process of wind speed is forecast by neural networks and the non-stationary process of wind speed is decomposed into quasi-stationary at different frequency scales by multi-scale characteristics of wavelet transforms. wavelet combined with neural network model avoid the neural network model that can not handle non-stationary questions .while, the effect of indefinite inputs are removed by embedding dimension of phase space to determine neural networks inputs. The simulation results show that phase space reconstruction of wavelet neural network is more accuracy than the ordinary BP neural network. It could be well applied in wind speed forecasts.

Forecasting increasing rate of power consumption based on immune genetic algorithm combined with neural network

Considering the factors affecting the increasing rate of power consumption, the BP neural network structure and the neural network forecasting model of the increasing rate of power consumption were established. Immune genetic algorithm was applied to optimizing the weight from input layer to hidden layer, from hidden layer to output layer, and the threshold value of neuron nodes in hidden and output layers. Finally, training the related data of the increasing rate of power consumption from 1980 to 2000 in China, a nonlinear network model between the increasing rate of power consumption and influencing factors was obtained. The model was adopted to forecasting the increasing rate of power consumption from 2001 to 2005, and the average absolute error ratio of forecasting results is 13.521 8%. Compared with the ordinary neural network optimized by genetic algorithm, the results show that this method has better forecasting accuracy and stability for forecasting the increasing rate of power consumption.

Extracting Evidences from Filesystem Activity Using Bayesian Network

This research aims to ascertain fi lesystem access patterns produced by different application programs, and evaluates their potential utility in improving digital forensic analyses. The access patterns produced by the proposed methodology can serve as a decision support system for determining the possible execution of certain applications in the event of computer misuse. For this purpose, we propose the use of a causal Bayesian network that summarizes the most important relationships among integral parameters relating to fi lesystem activities such as access, creation, modifi cation, fi le deletion, audit logs, registry entries and the manner in which the applications manipulate these parameters. Determining the state of a fi lesystem at a particular period of time is vital for conducting digital forensic analyses. Herein, we describe a Bayesian network-based technique to determine the state of a computer fi lesystem in terms of the program execution and fi les manipulated during some specific time period. Specifi cally, we discuss the construction of a Bayesian network from our prior knowledge of the manipulation of the fi lesystem and metadata information by a set of applications. The variations among the execution patterns of different applications indicate that the Bayesian network-based model is an appropriate tool, due to its ability to enable pattern learning and detection, even from an incomplete dataset. The focus of this paper is to highlight the merits of the Bayesian methods for learning, with regard to the techniques used for supervised learning in ordinary neural networks.

教師あり学習·教師なし学習·強化学習を複合したbrain-like学習システム

Our brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. And it is suggested that those learning paradigms relate deeply to the cerebellum, cerebral cortex and basal ganglia in the brain, respectively. Inspired by these knowledge of brain, we present a brain-like learning system with those three different learning algorithms. The proposed system consists of three parts: the supervised learning (SL) part, the unsupervised learning (UL) part and the reinforcement learning (RL) part. The SL part, corresponding to the cerebellum of brain, learns an input-output mapping by supervised learning. The UL part, corresponding to the cerebral cortex of brain, is a competitive learning network, and divides an input space to subspaces by unsupervised learning. The RL part, corresponding to the basal ganglia of brain, optimizes the model performance by reinforcement learning. Numerical simulations show that the proposed brain-like learning system optimizes its performance automatically and has superior performance to an ordinary neural network.

Self-organising associative kernel memory for multi-domain pattern classification

This paper proposes a novel self-organising associative neural network model in terms of kernel memory. The objective of this paper is not to give a sophisticated learning scheme and its rigorous mathematical accounts but rather attempt to address a paradigm shift, which could potentially answer a number of critical issues related to the current artificial neural network architectures. In the new memory model, the notion of ‘weights’ between the nodes is totally different from that as in ordinary neural network models, in which the weights simply represent the strengths of the connections between pairs of nodes in the kernel memory each realised by a kernel unit. Hence any arduous and iterative tuning of weight parameters is not involved and thereby the neural memory does not inherently suffer from any numerically-related problems. The associative memory is constructed via a simple unsupervised learning algorithm motivated from the traditional Hebbian principle. In the simulation study, both the plasticity and performance of the novel neural network architecture are discussed within the pattern classification context through single and simultaneous multi-domain classification tasks.

Selecting neural networks for a committee decision.

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilizing all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on two artificial and three real data sets.

One day before foF2 neural network based prediction models. A performance comparison between ordinary, fuzzy and recurrent neural networks

One day before <i>f</i>_oF2 neural network based prediction models. A performance comparison between ordinary, fuzzy and recurrent neural networks

A GENERAL UPDATING RULE FOR DISCRETE HOPFIELD-TYPE NEURAL NETWORK WITH TIME-DELAY AND THE CORRESPONDING SEARCH ALGORITHM

In this paper, the Hopfield neural network with delay (HNND) is studied from the standpoint of regarding it as an optimizing computational model. Two general updating rules for networks with delay (GURD) are given based on Hopfield-type neural networks with delay for optimization problems and characterized by dynamic thresholds. It is proved that in any sequence of updating rule modes, the GURD monotonously converges to a stable state of the network. The diagonal elements of the connection matrix are shown to have an important influence on the convergence process, and they represent the relationship of the local maximum value of the energy function to the stable states of the networks. All the ordinary discrete Hopfield neural network (DHNN) algorithms are instances of the GURD. It can be shown that the convergence conditions of the GURD may be relaxed in the context of applications, for instance, the condition of nonnegative diagonal elements of the connection matrix can be removed from the original convergence theorem. A new updating rule mode and restrictive conditions can guarantee the network to achieve a local maximum of the energy function with a step-by-step algorithm. The convergence rate improves evidently when compared with other methods. For a delay item considered as a noise disturbance item, the step-by-step algorithm demonstrates its efficiency and a high convergence rate. Experimental results support our proposed algorithm.

Application of MLP Networks to Bond Rating and House Pricing

Feedforward neural networks are receiving growing attention as a data modelling tool in economic classification problems. It is well known that controlling the design of a neural network can be cumbersome. Inaccuracies may lead to numerous problems in the application, such as higher errors due to local optima, overfitting and ill-conditioning of the network, especially when the number of observations is small. In this paper we provide a method to overcome these difficulties by regulating the flexibility of the network, and by rendering measures for validating the final network. In particular, a method is proposed to equilibrate the number of hidden neurons based on 5-fold cross-validation. In the validation process, the performance of the neural network is compared with a linear model using 5- fold cross-validation. In both case studies, the degree of monotonicity of the output of the neural network, with respect to each input variable, is calculated by numerical differentiation. The outcomes of this analysis are compared to what is expected from economic theory. Furthermore, a special class of monotonic neural networks and a corresponding training algorithm are developed. It is shown in the second case study that networks in this class have less tendency to overfitting than ordinary neural networks. The methods are illustrated in two case studies: predicting the price of housing in the Dutch city of Den Bosch; and the classification of bond ratings.