Linear Output Neurons Research Articles

Classified image compression using optimally structured auto-association networks

Here, an application of a set of auto-association networks with linear output neurons and sigmoidal hidden neurons for classified image compression is carried out. Simulations and statistical analysis of this type of network have shown that, at convergence, the hidden neurons operate mainly in their linear region. The nearly linear behaviour of the hidden neurons is exploited in finding out the minimum number of hidden neurons needed to reconstruct image data within a certain error threshold. Four optimally structured auto-association networks are set up so that each network is trained to encode a certain variance-based class of image blocks. Results have shown excellent performance of the proposed architecture in reproducing high-quality images at a low bit rate.

Analysis and pruning of nonlinear auto-association networks

In the paper, an analysis of a three‐layer nonlinear auto‐association network with linear output neurons and sigmoidal hidden neurons is carried out. Simulations have shown that the hidden layer neurons of this network operate mainly in their linear region. By studying the statistical relations governing the operation of such a network, the nearly linear behaviour of the sigmoidal hidden neurons was verified. Dealing with the network as being totally linear, a pruning algorithm is proposed to find out the minimum number of hidden neurons needed to reconstruct the input data within a certain error threshold. The performance of the pruning algorithm is illustrated with two examples.

Isotropic sequence order learning.

In this article, we present an isotropic unsupervised algorithm for temporal sequence learning. No special reward signal is used such that all inputs are completely isotropic. All input signals are bandpass filtered before converging onto a linear output neuron. All synaptic weights change according to the correlation of bandpass-filtered inputs with the derivative of the output. We investigate the algorithm in an open- and a closed-loop condition, the latter being defined by embedding the learning system into a behavioral feedback loop. In the open-loop condition, we find that the linear structure of the algorithm allows analytically calculating the shape of the weight change, which is strictly heterosynaptic and follows the shape of the weight change curves found in spike-time-dependent plasticity. Furthermore, we show that synaptic weights stabilize automatically when no more temporal differences exist between the inputs without additional normalizing measures. In the second part of this study, the algorithm is is placed in an environment that leads to closed sensor-motor loop. To this end, a robot is programmed with a prewired retraction reflex reaction in response to collisions. Through isotropic sequence order (ISO) learning, the robot achieves collision avoidance by learning the correlation between his early range-finder signals and the later occurring collision signal. Synaptic weights stabilize at the end of learning as theoretically predicted. Finally, we discuss the relation of ISO learning with other drive reinforcement models and with the commonly used temporal difference learning algorithm. This study is followed up by a mathematical analysis of the closed-loop situation in the companion article in this issue, "ISO Learning Approximates a Solution to the Inverse-Controller Problem in an Unsupervised Behavioral Paradigm" (pp. 865-884).

Neural Network Predictions of Human Psychological Perceptions of Clothing Sensory Comfort

The objective of this'paper is to investigate the predictability of clothing sensory comfort from psychological perceptions by using a feed-forward back-propagation net work in an artificial neural network (ANN) system. In order to achieve the objective, a series of wear trials is conducted in which ten sensory perceptions ( clammy, clingy, damp, sticky, heavy, prickly, scratchy, fit, breathable, and thermal) and overall clothing comfort ( comfort) are rated by twenty-two professional athletes in a controlled la ratory. They are asked to wear four different garments in each trial and rate the sensations above during a 90-minute exercising period. The scores are were input into five different eed-forward back-propagation neural network models, consisting of six different numbers of hidden and output transfer neurons. Results showing a good correlation between redicted and actual comfort ratings with a significance of p < 0:001 for all five models indicate overall comfort performance is predictable with neural networks, particularly models with log sigmoid hidden neurons and pure linear output neurons. Models with a single log sigmoid hidden layer with fifteen neurons or three hidden layers, each with ten log sigmoid hidden neurons, are able to produce better predictions than the other models for is particular data set in the study.

Stochastic neural computation. II. Soft competitive learning

For pt. I see ibid., p.891-905. An investigation has been made into the use of stochastic arithmetic to implement an artificial neural network solution to a typical pattern recognition application. Optical character recognition is performed on very noisy characters in the E-13B MICR font. The artificial neural network is composed of two layers, the first layer being a set of soft competitive learning subnetworks and the second a set of fully connected linear output neurons. The observed number of clock cycles in the stochastic case represents an order of magnitude improvement over the floating-point implementation assuming clock frequency parity. Network generalization capabilities were also compared based on the network squared error as a function of the amount of noise added to the input patterns. The stochastic network maintains a squared error within 10 percent of that of the floating-point implementation for a wide range of noise levels.

The statistical mechanics of the Ising perceptron

The authors investigate the problem of loading a single-layered perceptron of Ising (+or-1) weights, focusing on the cases of linear and binary output neurons. Previous studies of this problem have been made using the canonical ensemble, which in many cases require the breaking of replica symmetry. They consider here an alternative approach, based on the microcanonical ensemble, and show that many results that were obtained previously using replica symmetry breaking within the canonical ensemble can be easily obtained using the replica-symmetric assumption within the microcanonical ensemble. Since the nature of the replica symmetry breaking in many of the models with discrete weights is similar, they believe that the results can immediately be extended to other cases such as learning a rule from examples, and utilizing discrete weights of larger synaptic depth.