Semilinear Models Research Articles

Generalized neural networks for spectral analysis: dynamics and Liapunov functions

This paper analyzes local and global behavior of several dynamical systems which generalize some artificial neural network (ANN) semilinear models originally designed for principal component analysis (PCA) in the characterization of random vectors. These systems implicitly performed the spectral analysis of correlation (i.e. symmetric positive definite) matrices. Here, the proposed generalizations cover both nonsymmetric matrices as well as fully nonlinear models. Local stability analysis is performed via linearization and global behavior is analyzed by constructing several Liapunov functions.

Developments of the generative topographic mapping

The generative topographic mapping (GTM) model was introduced by Bishop et al. (1998, Neural Comput. 10(1), 215–234) as a probabilistic re-formulation of the self-organizing map (SOM). It offers a number of advantages compared with the standard SOM, and has already been used in a variety of applications. In this paper we report on several extensions of the GTM, including an incremental version of the EM algorithm for estimating the model parameters, the use of local subspace models, extensions to mixed discrete and continuous data, semi-linear models which permit the use of high-dimensional manifolds whilst avoiding computational intractability, Bayesian inference applied to hyper-parameters, and an alternative framework for the GTM based on Gaussian processes. All of these developments directly exploit the probabilistic structure of the GTM, thereby allowing the underlying modelling assumptions to be made explicit. They also highlight the advantages of adopting a consistent probabilistic framework for the formulation of pattern recognition algorithms.

Morphological associative memories

The theory of artificial neural networks has been successfully applied to a wide variety of pattern recognition problems. In this theory, the first step in computing the next state of a neuron or in performing the next layer neural network computation involves the linear operation of multiplying neural values by their synaptic strengths and adding the results. A nonlinear activation function usually follows the linear operation in order to provide for nonlinearity of the network and set the next state of the neuron. In this paper we introduce a novel class of artificial neural networks, called morphological neural networks, in which the operations of multiplication and addition are replaced by addition and maximum (or minimum), respectively. By taking the maximum (or minimum) of sums instead of the sum of products, morphological network computation is nonlinear before possible application of a nonlinear activation function. As a consequence, the properties of morphological neural networks are drastically different than those of traditional neural network models. The main emphasis of the research presented here is on morphological associative memories. We examine the computing and storage capabilities of morphological associative memories and discuss differences between morphological models and traditional semilinear models such as the Hopfield net.

A biologically plausible model of early visual motion processing. I: theory and implementation.

A model of local image encoding is described which explicitly incorporates quantitative data about the number density, bandwidth and receptive field organisation of neurons involved in motion detection. The model solves the problem of extracting local velocity on the basis of inputs tuned to spatiotemporal frequency and sensitive to contrast. The spatiotemporally tuned, opponent motion filters are followed by a compressive non-linearity and comprise a first stage. The inter-stage signals are interpreted as those from single neurons and the second stage is modelled as a neural-network layer. The second stage uses semilinear units and models the effect of lateral, on-centre off-surround, intra-layer connections. Characterisation of the first stage leads to a clarification of the concept of the psychophysical 'channel' and its relation to physiological data. The quantitative parametrisation of the model allows the simulation of several psychophysical phenomena which are reported in a companion paper.

Some remarks on the behaviour of the solution in dynamic processes for rate-type models

This work examines a dynamic problem in the study of semilinear rate-type models for which the plastic rate of deformation depends also on a parameter κ. The continuous dependence of the solution with respect to κ is obtained and the problem of finite time stability is also discussed. In the case when κ is interpreted as the absolute temperature, the dynamic problem is studied in the context of a Cattaneo-type heat law and also using the classical Fourier law. In the case when κ is interpreted as an internal state variable an existence and uniqueness result is obtained using a fixed point method and the finite time stability is also investigated.

Semi-linear stochastic models: Computation of moments and conditional moments

Semi-linear stochastic models: Computation of moments and conditional moments

Semi-linear stochastic models: Computation of moments and conditional moments

In this article we show how linearity with respect to the output of a stochastic dynamic model can be exploited in order to simplify the computation of moments or conditional moments. The results are presented for two examples, one of which includes delays. This feature is often encountered in biological models.

Comparison of 2 models for lateral inhibition

Two semi-linear models for lateral inhibition are discussed. The interaction between “receptor units” is assumed to be linear, as demonstrated by Hartline and Ratliff in the eye of the horseshoe crab Limulus polyphemus. Yet a model of such an inhibitory system must be nonlinear, since the output values correspond to nerve activities, which cannot be negativ. Models with forward inhibition were used often to describe contrast phenomena in the human nervous system. However, in order to simulate the input-output relation in systems similar to the eye of Limulus, a model with backward inhibition must be constructed. Two important properties of backward inhibition not shared by forward inhibition are: (1) Inhibition in a receptor unit has an influence upon its excitation, as well as upon its ability to inhibit other units (Disinhibition). (2) The range of interaction between sensory units is not necessarily the same as the range of direct cross connections. It is shown in this paper, that also forward inhibition may possess these two properties, provided that it is repeated on subsequent levels. Some properties of systems with backward and forward inhibition are studied and compared in models consisting of three units. The input-output relation for large systems with backward inhibition was calculated under special assumptions concerning the inhibitory coefficients. If the inhibitory coefficients in a system with backward inhibition decrease like a power series, as a function of the distance between receptor units, only neighboring receptors have an effect upon each other. That is, in an equivalent system with forward inhibition the inhibitory interaction is confined to neighbouring receptors. Conversely, when backward inhibition exists only between neighbouring receptors, the inhibitory coefficients in an equivalent system with forward inhibition are described, as a function of the distance between the receptor units, by a power series with alternating sign.