Simple Learning Rule Research Articles

The Information Content of Spontaneous Retinal Waves

Spontaneous neural activity that is present in the mammalian retina before the onset of vision is required for the refinement of retinotopy in the lateral geniculate nucleus and superior colliculus. This paper explores the information content of this retinal activity, with the goal of determining constraints on the nature of the developmental mechanisms that use it. Through information-theoretic analysis of multielectrode and calcium-imaging experiments, we show that the spontaneous retinal activity present early in development provides information about the relative positions of retinal ganglion cells and can, in principle, be used at retinogeniculate and retinocollicular synapses to refine retinotopy. Remarkably, we find that most retinotopic information provided by retinal waves exists on relatively coarse time scales, suggesting that developmental mechanisms must be sensitive to timing differences from 100 msec up to 2 sec to make optimal use of it. In fact, a simple Hebbian-type learning rule with a correlation window on the order of seconds is able to extract the bulk of the available information. These findings are consistent with bursts of action potentials (rather than single spikes) being the unit of information used during development and suggest new experimental approaches for studying developmental plasticity of the retinogeniculate and retinocollicular synapses. More generally, these results demonstrate how the properties of neuronal systems can be inferred from the statistics of their input.

Modeling Choice Behavior for New Pharmaceutical Products

This paper presents a dynamic generalization of a model often used to aid marketing decisions relating to conventional products. The model uses stated-preference data in a random-utility framework to predict adoption rates for new pharmaceutical products. In addition, this paper employs a Markov model of patient learning in drug selection. While the simple learning rule presented here is only a rough approximation to reality, this model nevertheless systematically incorporates important features including learning and the influence of shifting preferences on market share. Despite its simplifications, the integrated framework of random-utility and product attribute updating presented here is capable of accommodating a variety of pharmaceutical marketing and development problems. This research demonstrates both the strengths of stated-preference market research and some of its shortcomings for pharmaceutical applications.

Complex dynamics in learning systems

To describe the dynamics of learning an input–output relation from a set of examples, the evolution of an appropriate choice of macroscopic dynamical variables have to be found. Recent progress in on-line learning only addresses the often unrealistic case of an infinite training set. For restricted training sets, previous studies have so far been limited to asymptotic dynamics or simple learning rules. Using the cavity method and diagrammatic techniques, we introduce a new framework to model batch learning of restricted sets of examples, widely applicable to any learning cost function, and fully taking into account the temporal correlations introduced by the recycling of the examples.

On the emergence of exchange and mediation in a production economy

In this paper we simulate the behavior of a population of boundedly rational agents in a two good economy where all agents can spend their time budget for the production of one or both goods or trading. Agents update their strategies according to a simple imitation type learning rule with noise. It is shown that in several different setups both direct trade and trade via mediators who specialize in trading can emerge. Both increasing returns to scale in production and heterogeneity of production technologies facilitates the development of trade. For heterogeneous production technologies we can also observe the transition from a pure production economy via direct trade to an economy with mediated trade.

Adaptive leaky integrator models of cerebellar Purkinje cells can learn the clustering of temporal patterns

We have shown previously that the metabotropic glutamate receptor signalling network in a cerebellar Purkinje cell can implement adaptive postsynaptic delays. Here we present a leaky integrator version of the Purkinje cell model which uses a simple synaptic delay learning rule. We show that a single leaky integrator can learn a radial basis function-like response to temporal parallel fibre patterns, and that different leaky integrators in a group are able to discover different clusters in a temporal parallel fibre input space. The clustering performance of the model can be improved by desensitization of the input currents.

Win–Stay, Lose–Shift Strategies for Repeated Games—Memory Length, Aspiration Levels and Noise

Win–stay, lose–shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win–stay, lose–shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success'’. I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win–stay, lose–shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win–stay, lose–shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win–stay, lose–shift principle is a very successful strategy in repeated games with noise.

On the stability of monotone discrete selection dynamics with inertia

In this paper we study the learning behavior of a population of boundedly rational players who interact by repeatedly playing an evolutionary game. A simple imitation type learning rule for the agents is suggested and it is shown that the evolution of the population strategy is described by the discrete time replicator dynamics with inertia. Conditions are derived which guarantee the local stability of a Nash-equilibrium with respect to these dynamics and for quasi-strict ESS with support of three pure strategies the crucial level of inertia is derived which ensures stability and avoids overshooting. These results are illustrated by an example and generalized to the class of monotone selection dynamics.

Associative memory based on synchronized firing of spiking neurons with time-delayed interactions

We study associative memory of a neural network of spiking neurons with time-delayed synaptic interactions incorporating the time taken by an action potential to propagate along the axon. Individual spiking neurons are described by a set of nonlinear differential equations capable of exhibiting excitability such as that of Hodgkin-Huxley and FitzHugh neurons. When a simple learning rule of the autocorrelation type based on random patterns is assumed, memory retrieval is shown to be accompanied by synchronized firing of neurons. The reduced dynamics with a few degrees of freedom of the network with a finite number of stored patterns is analytically derived in the limit of infinitely many neurons. The dependence of the appearance of retrieval states on the distribution of time delay and on the size of refractory period given implicitly in the model is obtained, showing good agreement between the result of numerical simulations and that obtained from the reduced dynamics. The behavior of the network with an extensive number of patterns is also investigated and an approximate analysis is presented to discuss the storage capacity.

Generative models for discovering sparse distributed representations.

We describe a hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demonstrate that the network learns to extract sparse, distributed, hierarchical representations.

The Formation of Topographic Maps That Maximize the Average Mutual Information of the Output Responses to Noiseless Input Signals

This article introduces an extremely simple and local learning rule for to pographic map formation. The rule, called the maximum entropy learning rule (MER), maximizes the unconditional entropy of the map's output for any type of input distribution. The aim of this article is to show that MER is a viable strategy for building topographic maps that maximize the average mutual information of the output responses to noiseless input signals when only input noise and noise-added input signals are available.

Beyond Bayes's theorem: Effect of base-rate information in consensus games.

Many of the practical implications of behavioral decision-making research are based on the assumption that behavioral trends have to be compared with prescriptions. The present article demonstrates that in certain settings this approach is both inapplicable because there is no normative prescription and unnecessary because robust quantitative predictions can be made without reference to prescriptions. Experiment 1 demonstrates that a simple learning rule can be used to predict the base-rate effect in consensus games with multiple equilibria. Experiment 2 shows that information about the payoff rule affects participants' initial propensities but does not affect the learning process. Some implications of these results for the understanding of decision groups in social contexts, such as employment decisions in organizations, are pointed out.

Hebbian learning and the development of direction selectivity: the role of geniculate response timings

Zero-sum Hebbian learning rules that reinforce well correlated inputs have been used by others to model the competitive self-organization of afferents from the lateral geniculate nucleus to produce orientation selectivity and ocular dominance columns. However, the application of these simple Hebbian rules to the development of direction selectivity (DS) is problematic because the best correlated inputs are those that are well correlated in both the preferred and nonpreferred directions of motion. Such afferents would combine to produce non-DS cortical units. Afferents that are in spatiotemporal quadrature would combine to produce DS cortical units, but are poorly correlated in the nonpreferred direction. In this paper, the development of DS is reduced to the problem of associating a pair of units in spatiotemporal quadrature in the face of competition from a third, non-quadrature unit. As expected, simple Hebbian learning rules perform poorly at associating the quadrature pair. However, two additional Hebb-type learning rules, postsynaptic gating and BCM (Bienenstock, Cooper and Munro), improve performance. Results from this three-input model are shown to generalize to a larger network. We conclude that learning rules in which the postsynaptic response determines the magnitude and/or direction of synaptic change perform better than simple Hebbian rules at establishing direction selectivity.

Role of reinforcement in spaced-trial operant learning in pigeons (Columba livia).

The author designed 3 experiments to study pigeons' adjustment to unexpected shifts in reinforcer magnitude with a single trial per day. Extinction was faster or poorer after training with a small magnitude (1 food pellet) than after training with a large magnitude (10 or 15 pellets). A shift from IS to 1 pellet was accompanied by a gradual adjustment, with no indication of a successive negative contrast effect. Pigeons discriminated the reinforcer magnitudes but yielded no evidence of spaced-trial simultaneous negative contrast. Moreover, extinction was faster for a stimulus paired with 1 pellet than for a stimulus paired with 15 pellets. The results can be interpreted in terms of simple strengthening-weakening learning rules and without reference to anticipatory frustration. The acquisition and extinction of simple instrumental responses is affected by variables such as the magnitude and probability of reinforcement in a variety of vertebrate species. However, when training is administered on a spaced-trial basis (i.e., a single trial per day, with intertrial intervals of 24 hr or longer), the generality of some of these

Simple neuron models for independent component analysis.

Recently, several neural algorithms have been introduced for Independent component Analysis. Here we approach the problem from the point of view of a single neuron. First, simple Hebbian-like learning rules are introduced for estimating one of the independent components from sphered data. Some of the learning rules can be used to estimate an independent components which has a negative kurtosis, and the others estimate a component of positive kurtosis. Next, a two-unit system is introduced to estimate an independent component of any kurtosis. The results are then generalized to estimate independent components from non-sphered (raw) mixtures. To separate several independent components, a system of several neurons with linear negative feedback is used. The convergence of the learning rules is rigorously proven without any unnecessary hypotheses on the distribution of the independent components.

Self-similar Neural Networks Based on a Kohonen Learning Rule

One of the most striking features about the perceptual machinery of mammals is its regularity of structure. This is particularly evident in the mammalian visual system, as the work pioneered by Hubel and Wiesel has demonstrated. The likely source of this regularity is the visual stimulus, which does not change randomly from instant to instant, but is affected primarily by motions of both the animal and objects in the environment. These motions induce structured changes in the visual stimulus, which might well be expected to have a significant effect in shaping the structure of the visual machinery, whether through individual plasticity or longer-term genetic changes. The work reported in this paper is an investigation of the structures that may evolve in a simple artificial neural network driven not by random changes of input pattern, but directly by transformations which are themselves related to transformations of the input signal through an analysis of motion-prediction error. Results are presented which demonstrate that such networks can evolve a remarkable degree of regularity which reflects the underlying symmetry group of the transformations, both in one and two dimensions. An appropriate and visually plausible choice of transformation group can lead to the development of foveal structures in two-dimensional networks. We also present some preliminary results on parametrised function spaces which support the general conclusion that global structure bearing a considerable resemblance to that found in the mammalian visual system can evolve as the result of a simple learning rule in networks driven by transformations similar to those typically encountered in vision. Copyright © 1996 Elsevier Science Ltd

A dot product neuron for hardware implementation of competitive networks

Competitive models based on a simple dot product neuron often deal with normalized vectors, which adds a hard computational cost. Using Euclidean distance nodes without normalization is only a partial solution, because they are less plausible from a biological point of view and the computational cost of the Euclidean distance is greater than that of the dot product. In this work the author proposes a dot product neuron, formally equivalent to a Euclidean neuron, which does not require vector normalization. The only requirement for such a neuron model is subtracting from the dot product an iteratively computed bias. A simple incremental learning rule for this neuron is also introduced. The proposed model is suitable for hardware implementation of competitive networks.

Reduced Representation by Neural Networks with Restricted Receptive Fields

Model neural networks can perform dimensional reductions of input data sets using correlation-based learning rules to adjust their weights. Simple Hebbian learning rules lead to an optimal reduction at the single unit level but result in highly redundant network representations. More complex rules designed to reduce or remove this redundancy can develop optimal principal component representations, but they are not very compelling from a biological perspective. Neurons in biological networks have restricted receptive fields limiting their access to the input data space. We find that, within this restricted receptive field architecture, simple correlation-based learning rules can produce surprisingly efficient reduced representations. When noise is present, the size of the receptive fields can be optimally tuned to maximize the accuracy of reconstructions of input data from a reduced representation.

The Role of Weight Normalization in Competitive Learning

The effect of different kinds of weight normalization on the outcome of a simple competitive learning rule is analyzed. It is shown that there are important differences in the representation formed depending on whether the constraint is enforced by dividing each weight by the same amount (“divisive enforcement”) or subtracting a fixed amount from each weight (“subtractive enforcement”). For the divisive cases weight vectors spread out over the space so as to evenly represent “typical” inputs, whereas for the subtractive cases the weight vectors tend to the axes of the space, so as to represent “extreme” inputs. The consequences of these differences are examined.

A biologically inspired control approach for distributed elastic systems

A multi-degree of freedom control approach, which is largely inspired by biological systems, is presented. Control inputs to a structure are achieved by multiple piezoelectric actuators. One actuator chosen as the ‘‘master’’ actuator is under the direction of the central, sophisticated controller. The other ‘‘slave’’ actuators derive their control inputs by localized, simple learning rules related to the behavior of their neighbor actuators including the ‘‘master.’’ Simulations on the control of the vibrational energy density of a harmonically excited simply supported beam demonstrate a significant improvement in control performance over a single actuator case, particularly for off-resonance frequencies. Thus high attenuations are achieved with a multiple degree of freedom actuator with a single main channel of control.

Anti-Hebbian learning in topologically constrained linear networks: a tutorial

Using standard results from the adaptive signal processing literature, we review the learning behavior of various constrained linear neural networks made up of anti-Hebbian synapses, where learning is driven by the criterion of minimizing the node information energy. We point out how simple learning rules of Hebbian type can provide fast self-organization, under rather wide connectivity constraints. We verify the results of the theory in a set of simulations.