Simple Neural Network Model Research Articles

Long-lasting transients of activation in neural networks

The question has been investigated whether long-lasting transients of activation (i.e. slow waves), observed to occur in the intact cerebral cortex (EEG ‘delta’ waves and ‘K’ complexes) as well as an isolated tissue cultured in vitro, can also emerge in a simplified neural network model of interconnected excitatory and inhibitory cells. It is shown that slow waves can indeed occur even if the cells in the network do not have explicitly built-in slow processes. The mechanism underlying the termination of transient activity depends crucially upon the presence of a refractory period and random activity, rather than upon inhibitory suppression. A wide range of characteristic unit firing patterns is associated with transient population activities, even though all the cells in the network model have identical response properties.

Eigenvalues of covariance matrices: Application to neural-network learning.

The learing time of a simple neural-network model is obtained through an analytic computation of the eigenvalue spectrum for the Hessian matrix, which describes the second-order properties of the objective function in the space of coupling coefficients. The results are generic for symmetric matrices obtained by summing outer products of random vectors. The form of the eigenvalue distribution suggests new techniques for accelerating the learning process, and provides a theoretical justification for the choice of centered versus biased state variables.

A neural network approach to infrared spectrum interpretation

The simple linear neural network model was investigated as a method for automated interpretation of infrared spectra. The model was trained using a database of infrared spectra of organic compounds of known structure. The model was able to learn, without any prior input of spectrum-structure correlations, to recognize and identify 76 functional groupings with accuracies ranging from fair to excellent. The effect of network input parameters and of training set composition were studied, and several sources of spurious correlations were identified and corrected.

Selective pattern recall in neural networks by chemical modulation

The authors present a simple neural network model in which the outputs of groups of neurons are chemically modulated. This feature takes into account the existence of neuromodulators which are known to have a strong effect on the behaviour of real biological networks. The effective (non-symmetric) connection matrix thereby becomes a function of the modulator concentrations. By deriving evolution equations for the pattern overlaps they show that, during recall, the system can select distinct subclasses from the stored patterns by varying these concentrations. This model may increase the understanding of the function of neuromodulators.

Memory capacity in neural network models: Rigorous lower bounds

We consider certain simple neural network models of associative memory with N binary neurons and symmetric lth order synaptic connections, in which m randomly chosen N-bit patterns are to be stored and retrieved with a small fraction δ of bit errors allowed. Rigorous proofs of the following are presented both for l = 2 and l ≥ 3: 1. 1. m can grow as fast as αN l−1. 2. 2. α can be as large as B l/ln(1/ δ) as δ → 0. 3. 3. Retrieved memories overlapping with several initial patterns persist for (very) small α. These phenomena were previously supported by numerical simulations or nonrigorous calculations. The quantity (l!) · α represents the number of stored bits per distinct synapse. The constant (l!) · B l is determined explicitly; it decreases monotonically with l and tends to zero exponentially fast as l → ∞. We obtain rigorous lower bounds for the threshold value (l!) · α c (the maximum possible value of (l!) · α with δ unconstrained): 0.11 for l = 2 (compared to the actual value between 0.28 and 0.30 as estimated by Hopfield and by Amit, Gutfreund, and Sompolinsky), 0.22 for l = 3 and 0.16 for l = 4; as l → ∞, the bound tends to zero as fast as (l!) · B l.

A synaptic model for spatial frequency adaptation

A model for visual adaptation to spatial grating is developed based on the assumption that inhibitory synapses within the visual system may be temporarily modified as a function of recent usage. Specifically, it is hypothesized that inhibitory synaptic weights are altered as a function of the correlation between recent presynaptic and postsynaptic activity. When such modifiable synapses are incorporated into a simple neural network model having the spatial filtering properties of the human visual system, two coupled equations are obtained which may be solved analytically. The model accounts for experimental data on adaptation to sinusoidal gratings, square wave gratings, single bars, and tilted gratings. The relationship of the model to single and multiple channel models of the human visual system is discussed.