Two-layer Perceptron Network Research Articles

Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse

Asymmetric nonlinear weight update is considered as one of the major obstacles for realizing hardware neural networks based on analog resistive synapses because it significantly compromises the online training capability. This paper provides new solutions to this critical issue through co-optimization with the hardware-applicable deep-learning algorithms. New insights on engineering activation functions and a threshold weight update scheme effectively suppress the undesirable training noise induced by inaccurate weight update. We successfully trained a two-layer perceptron network online and improved the classification accuracy of MNIST handwritten digit dataset to 87.8/94.8% by using 6-bit/8-bit analog synapses, respectively, with extremely high asymmetric nonlinearity.

Prediction of degree of crystallinity for the LTA zeolite using artificial neural networks

AbstractZeolites are microporous aluminosilicate/silicate crystalline materials with three-dimensional tetrahedral configuration. In this study, the degree of crystallinity of the synthesized Linde Type A (LTA) zeolite, which is the main indicator of its quality/purity is tried to be modeled. Effect of crystallization time, temperature, molar ratio of the synthesis gel on the relative crystallinity of the LTA zeolites is investigated using artificial neural networks. Our experimental observations and some data collected from literature have been used for adjusting the parameters of the proposed model and evaluating its performance. It has been observed that two-layer perceptron network with eight hidden neurons is the most accurate approach for the considered task. The designed model predicts the experimental datasets with a mean square error of 3.99 × 10-6, absolute average relative deviation of 8.69 %, and regression coefficient of 0.9596. The proposed model can decrease the required time and number of experiments to evaluate the extent of crystallinity of the LTA zeolites.

System Identification in Difficult Operating Conditions Using Artificial Neural Networks

To investigate an ability of system identification in difficult operating conditions. A simulation based experiment was performed on a simple second order system with white noise signal superimposed to the output signal. Interferences are added to the output signal in order to simulate difficult operating conditions present in a real system environment. Based on system simulation measurements, the system was identified using conventional method with least squares estimate and an alternative method, a multi-layer perceptron (MLP) network. Graphical evaluation of simulation results showed that MLP network produced better results than conventional model, with significantly better results in case of interferences in the output signal. To model dynamic system, a simple two-layer perceptron network with external dynamic members was trained in Matlab using Levenberg-Marquardt algorithm.

A Non-Linear Blind Source Separation Method Based on Perceptron Structure and Conjugate Gradient Algorithm

The linear mixing model has been considered previously in most of the researches which are devoted to the blind source separation (BSS) problem. In practice, a more realistic BSS mixing model should be the non-linear one. In this paper, we propose a non-linear BSS method, in which a two-layer perceptron network is employed as the separating system to separate sources from observed non-linear mixture signals. The learning rules for the parameters of the separating system are derived based on the minimum mutual information criterion with conjugate gradient algorithm. Instead of choosing a proper non-linear functions empirically, the adaptive kernel density estimation is used in order to estimate the probability density functions and their derivatives of the separated signals. As a result, the score function of the perceptron's outputs can be estimated directly. Simulations show good performance of the proposed non-linear BSS algorithm.

Neural network-based software sensor: training set design and application to a continuous pulp digester

A neural network-based strategy for detection of feedstock variations in a continuous pulp digester is presented. A feedforward two-layer perceptron network is trained to detect and isolate unmeasured variations in the feedstock. Training and validation data sets are generated using a rigorous first principles model. The most important issue discussed here is the design of the data set required to train the artificial neural network. Efficiency and limitation of such an approach are demonstrated using simulations.

A two-layer paradigm capable of forming arbitrary decision regions in input space

It is well known that a two-layer perceptron network with threshold neurons is incapable of forming arbitrary decision regions in input space, while a three-layer perceptron has that capability. The effect of replacing the output neuron in a two-layer perceptron with a bithreshold element is studied. The limitations of this modified two-layer perceptron are observed. Results on the separating capabilities of a pair of parallel hyperplanes are obtained. Based on these, a new two-layer neural paradigm based on increasing the dimensionality of the output of the first layer is proposed and is shown to be capable of forming any arbitrary decision region in input space. Then a type of logic called bithreshold logic, based on the bithreshold neuron transfer function, is studied. Results on the limits of switching function realizability using bithreshold gates are obtained.

Neural network models of perceptual learning of angle discrimination.

We study neural network models of discriminating between stimuli with two similar angles, using the two-alternative forced choice (2AFC) paradigm. Two network architectures are investigated: a two-layer perceptron network and a gating network. In the two-layer network all hidden units contribute to the decision at all angles, while in the other architecture the gating units select, for each stimulus, the appropriate hidden units that will dominate the decision. We find that both architectures can perform the task reasonably well for all angles. Perceptual learning has been modeled by training the networks to perform the task, using unsupervised Hebb learning algorithms with pairs of stimuli at fixed angles theta and delta theta. Perceptual transfer is studied by measuring the performance of the network on stimuli with theta' not equal to theta. The two-layer perceptron shows a partial transfer for angles that are within a distance a from theta, where a is the angular width of the input tuning curves. The change in performance due to learning is positive for angles close to theta, but for magnitude of theta-theta' approximately a it is negative, i.e., its performance after training is worse than before. In contrast, negative transfer can be avoided in the gating network by limiting the effects of learning to hidden units that are optimized for angles that are close to the trained angle.