Neuron-like Elements Research Articles

Learning structured representations

SHRUTI is a connectionist model that demonstrates how a network of neuron-like elements can encode a large body of semantic, episodic, and causal knowledge, and rapidly make decisions and perform explanatory and predictive reasoning. To further ground this model in the functioning of the brain it must be shown that components of the model can be learned in a neurally plausible manner. Previous work has already demonstrated the rapid learning of episodic facts via cortico-hippocampal interactions. Here we discuss how other SHRUTI representations such as causal rules, statistical and semantic knowledge, and categories might be learned.

Binding and multiple instantiation in a distributed network of spiking nodes

An implementation of a distributed connectionist network of spiking neuron-like elements is presented. Spiking nodes fire at a precise moment and transmit their activation, with particular strengths and delays, to nodes connected to them. When the potential of the node reaches a particular threshold, it emits a spike. Thereafter, the potential is reset to a resting value. The receiving nodes accumulate potential, but also slowly lose their potential through decay. As with real neurons, after firing there is a short refractory period during which the node will be completely insensitive to incoming signals, after which its sensitivity will slowly increase. Precise timing properties are used to represent symbols in a distributed manner and to solve the problems of variable binding and multiple instantiation. Several predictions about human short-term memory, predicate processing, complex reasoning and multiple instantiation arise from this model. This network shows how symbolic processing can be achieved using neurobiologically and psychologically plausible mechanisms that also have the advantages of generalization and noise tolerance found in connectionist networks.

Analysis of the effect of backward masking by means of a complex model of a net of neuron-like elements

According to the experiments with a projective-associative model of the neuronal net, the phenomenon of “backward masking” of the first stimulus of a pair of stimuli at a small time gap between the stimuli is caused by two events: (1) pre-excitation inhibition of the first stimulus-induced activation by the second stimulus and (2) disturbance of information processing connected with the deficiency of time needed to match the recalled symbol in memory to the symbol presented to the input subsystem and also to name it. Identification of the second stimulus may be impaired with a decreasing time interval due to: (1) superposition of the second (2) recurrent inhibition occurring in the neuronal net upon recognition of the first stimulus. It was found that in conditions of activity of neuron-like elements of the neuronal net, simulating the states of somnolence or slow-wave sleep, corresponding subsystems failed to learn, while time needed to identify already “learned” symbols substantially increased. The data obtained are in agreement with the hypothesis concerning the causes of backward masking and also with the facts on optimal conditions of learning and reproducing its results in living nervous system. It seems reasonable that discussed disturbances of information processing should be kept in mind in designing computers of a new generation, based on the use of principles of brain functioning, in order to increase the reliability and operation speed of technical systems.

The role of cortical area MST in a model of combined smooth eye-head pursuit.

The cortical medial superior temporal area (MST) is essential for the normal execution of smooth pursuit eye movements. Many pursuit-related neurons (visual-tracking neurons = VT neurons) in the lateral part of area MST (MSTl) are responsive to retinal image slip (r) as well as to eye (e) and head velocity (h) with similar preferred directions (isodirectionality). We show, by running a connectionist network with VT neuron-like elements, that an assembly of MSTl-VT neurons is able to reconstruct target motion in world-centered coordinates (t'). When t' is fed into a subsequent model stage, converting t' into gaze velocity (g') with varying contributions of e and h, the overall model is able to account for many of the salient properties of visually guided pursuit including the consequences of MSTl lesions. However, the analysis of the MSTl network also clearly indicates that isodirectionality is not a prerequisite for its performance. The investigation of a second model suggests that isodirectionality indeed does not result from functional but from developmental constraints. This second model is a connectionist network with hidden units, which similar to MSTl-VT neurons receive input from modality specific units encoding retinal slip, eye and head velocity. After training this network to offer t' as output, two subsets of hidden units emerged, one exhibiting isodirectionality, but not the other. Since only isodirectional hidden units contributed to the flow of information, the preponderance of isodirectional MSTl-VT neurons might be the result of developmental pruning, eliminating the second group.

Speech recognition apparatus using neural network and learning method therefor

A speech recognition apparatus using a neural network. A neuron-like element according to the present invention has a means for storing a value of the inner condition thereof, a means for updating a value of internal status on the basis of an output from the neuron-like element itself, outputs from other neuron-like elements and an input from the ouside, and an ouput value generating means for converting a value of internal status into an outside output. Accordingly, the neuron-like element itself can retain the history of input data. This enables the time series data, such as speech to be processed without providing any special means in the neural network.

Advances in SHRUTI—A Neurally Motivated Model of RelationalKnowledge Representation and Rapid Inference Using Temporal Synchrony

We are capable of drawing a variety of inferences effortlessly, spontaneously, and with remarkable efficiency—as though these inferences are a reflex response of our cognitive apparatus. This remarkable human ability poses a challenge for cognitive science and computational neuroscience: How can a network of slow neuron-like elements represent a large body of systematic knowledge and perform a wide range of inferences with such speed? The connectionist model SHRUTI attempts to address this challenge by demonstrating how a neurally plausible network can encode a large body of semantic and episodic facts, systematic rules, and knowledge about entities and types, and yet perform a wide range of explanatory and predictive inferences within a few hundred milliseconds. Relational structures (frames, schemas) are represented in SHRUTI by clusters of cells, and inference in SHRUTI corresponds to a transient propagation of rhythmic activity over such cell-clusters wherein dynamic bindings are represented by the synchronous firing of appropriate cells. SHRUTI encodes mappings across relational structures using high-efficacy links that enable the propagation of rhythmic activity, and it encodes items in long-term memory as coincidence and coincidence-error detector circuits that become active in response to the occurrence (or non-occurrence) of appropriate coincidences in the on going flux of rhythmic activity. Finally, “understanding” in SHRUTI corresponds to reverberant and coherent activity along closed loops of neural circuitry. Over the past several years, SHRUTI has undergone several enhancements that have augmented its expressiveness and inferential power. This paper describes some of these extensions that enable SHRUTI to (i) deal with negation and inconsistent beliefs, (ii) encode evidential rules and facts, (iii) perform inferences requiring the dynamic instantiation of entities, and (iv) seek coherent explanations of observations.

Time Intervals Comparing Neural Network

The proposed neuronal network model consisting of several tens of neuron-like elements with graded responses performs a continual analysis of differences (> 25 ms) in time intervals between successive input signals. The network comprises the following building blocks: clock, pattern developing modules, dynamic memory units, coincidence detectors, identifiers. Two types of network organization were developed: one with a loop of modules and another one with a cascade of modules. The activity of the proposed “neuronal” assembly can be channeled to different outputs—depending on whether the stimulus presentation is regular or random. The described computation mechanism can be involved in various time-dependent brain functions. Copyright © 1996 Elsevier Science Ltd

On Alan Turing's anticipation of connectionism

AbstractIt is not widely realised that Turing was probably the first person to consider building computing machines out of simple, neuron-like elements connected together into networks in a largely random manner. Turing called his networks ‘unorganised machines’. By the application of what he described as ‘appropriate interference, mimicking education’ an unorganised machine can be trained to perform any task that a Turing machine can carry out, provided the number of ‘neurons’ is sufficient. Turing proposed simulating both the behaviour of the network and the training process by means of a computer program. We outline Turing's connectionist project of 1948.

Testis of Prepubertal Rhesus Monkeys Receives a Dual Catecholaminergic Input Provided by the Extrinsic Innervation and an Intragonadal Source of Catecholamines1

The mammalian testis is innervated by extrinsic catecholaminergic nerves and responds to catecholamines with steroid secretion. Although the primate testis has also been shown to be innervated, potential differences in the density of this innervation between immature and sexually developed individuals have not been described. A recent study demonstrated that the primate ovary contains a network of neuron-like cells and that some of these cells are catecholaminergic. It is thus possible that the male gonad is also endowed with a similar intragonadal source of catecholamines. The present study addresses these two issues. Catecholaminergic nerves were identified as such by their content of immunoreactive tyrosine hydroxylase (TH; the rate-limiting step in catecholamine biosynthesis), and in some cases by glyoxylic acid histochemistry. Fibers containing TH were abundant in testes from juvenile animals (1-2 yr of postnatal life), but the density of this innervation was not maintained in adult animals, whose testis showed only a few TH-positive fibers scattered in the interstitial tissue. Testicular norepinephrine (NE) concentration was much lower in adult than in juvenile animals, suggesting that the marked increase in testicular weight that occurs with the attainment of sexual maturity is not accompanied by corresponding changes in NE content. At the ultrastructural level, testicular nerve fibers contained pleiomorphic, dense-core and clear vesicles, suggesting the presence of catecholamines and other neurotransmitters. In addition to this extrinsic catecholaminergic innervation, prepubertal testes, but not adult gonads, contained an intrinsic population of TH-immunopositive neuron-like elements, identified as cells by confocal scanning laser microscopy. To determine whether the prepubertal monkey testis indeed expresses the TH gene, testicular RNA was subjected to reverse transcriptase polymerase chain reaction to amplify the 5' end of TH mRNA, which encodes the regulatory domain of the enzyme. The cDNA that was obtained predicts an amino acid sequence similar, but not identical, to that encoded by the alternatively spliced type 1 TH mRNA form present in the adrenal gland. These results indicate 1) that the primate testis receives a dual catecholaminergic input, one provided by the extrinsic innervation and the other by neuron-like cells located within the gonad itself, and 2) that the influence exerted by both sources on testicular function may be more prominent during the prepubertal period than in adulthood. The presence in the testis of a TH mRNA variant encoding amino acid substitutions in its 5' end suggests that regulation of testicular TH enzyme activity may include a gonad-specific component.

Chaos and associative generation of information by networks of neuronal oscillators.

Networks of neuronlike elements described by a simple model are studied. For a single neuron, a number of bifurcations and chaotic attractors are observed. A mechanism of an associative production of information based on a transition fixed point\ensuremath{\rightarrow}chaotic attractor\ensuremath{\rightarrow}new fixed point is proposed and discussed. \textcopyright{} 1996 The American Physical Society.

From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony

AbstractHuman agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency – as though these inferences were a reflexive response of their cognitive apparatus. Furthermore, these inferences are drawn with reference to a large body of background knowledge. This remarkable human ability seems paradoxical given the complexity of reasoning reported by researchers in artificial intelligence. It also poses a challenge for cognitive science and computational neuroscience: How can a system of simple and slow neuronlike elements represent a large body of systemic knowledge and perform a range of inferences with such speed? We describe a computational model that takes a step toward addressing the cognitive science challenge and resolving the artificial intelligence paradox. We show how a connectionist network can encode millions of facts and rules involvingn-ary predicates and variables and perform a class of inferences in a few hundred milliseconds. Efficient reasoning requires the rapid representation and propagation of dynamic bindings. Our model (which we refer to as SHRUTI) achieves this by representing (1) dynamic bindings as the synchronous firing of appropriate nodes, (2) rules as interconnection patterns that direct the propagation of rhythmic activity, and (3) long-term facts as temporal pattern-matching subnetworks. The model is consistent with recent neurophysiological evidence that synchronous activity occurs in the brain and may play a representational role in neural information processing. The model also makes specific psychologically significant predictions about the nature of reflexive reasoning. It identifies constraints on the form of rules that may participate in such reasoning and relates the capacity of the working memory underlying reflexive reasoning to biological parameters such as the lowest frequency at which nodes can sustain synchronous oscillations and the coarseness of synchronization.

Glutamate receptor-mediated currents and toxicity in embryonal carcinoma cells.

While primary neuronal cell cultures have been used to investigate excitotoxicity, development of cell lines exhibiting glutamate receptor-mediated death is desirable. P19 mouse embryonal carcinoma cells, exposed to retinoic acid and plated onto a layer of cultured mouse cortical glial cells, differentiated into neuron-like elements immunoreactive for neurofilaments and neuron-specific enolase. Whole-cell recordings revealed inward currents in response to extracellular application of either NMDA or kainate. The NMDA-induced currents exhibited a voltage-dependent blockade by magnesium, required glycine for maximal activation, and were blocked by the NMDA antagonist dizocilpine. Kainate-induced currents were blocked by the AMPA/kainate receptor antagonist CNQX. Exposure to 500 microM NMDA for 24 h destroyed most P19 cells (EC50 approximately 70 microM); death was prevented by dizocilpine or D-APV. Exposure to 500 microM kainate also resulted in widespread death reduced by CNQX. Thus differentiated P19 cells exhibited both excitatory amino acid responses and vulnerability to excitotoxicity, characteristic of CNS neurons. These cells may provide a genetically open system useful for studying glutamate receptor-mediated phenomena at a molecular level.

Process control using genetically trained neural networks

This paper presents a novel approach to the general problem of the control of processes whose dynamic characteristics are not known, or little known. It demonstrates how a system consisting of a relatively small number of neuronlike elements can be used to control a wide variety of processes with little or no prior knowledge of the process to be controlled. The general procedure involved uses a form of simulated evolution in which a group of controllers, each of which is represented by a small neural network, gradually improves over time by combining their connection weights or by small mutational changes to their weights, together with selective reproduction using fitness values assigned to each network based on a global evaluation of each network's performance.

Low-level aspects of segmentation and recognition.

This paper discusses two problems related to three-dimensional object recognition. The first is segmentation and the selection of a candidate object in the image, the second is the recognition of a three-dimensional object from different viewing positions. Regarding segmentation, it is shown how globally salient structures can be extracted from a contour image based on geometrical attributes, including smoothness and contour length. This computation is performed by a parallel network of locally connected neuron-like elements. With respect to the effect of viewing, it is shown how the problem can be overcome by using the linear combinations of a small number of two-dimensional object views. In both problems the emphasis is on methods that are relatively low level in nature. Segmentation is performed using a bottom-up process, driven by the geometry of image contours. Recognition is performed without using explicit three-dimensional models, but by the direct manipulation of two-dimensional images.

Time course of neural responses discriminating different views of the face and head.

1. Measurements of the magnitude and time course of response were made from 44 cells responsive to static head views at different levels of stimulus effectiveness. In this way responses to complex stimulus patterns evoking good, poor, and midrange responses could be compared across the cell population. 2. Cells exhibiting both good and poor initial discrimination between head views were found at short and long latencies; there was no correlation of any of the temporal response parameters measured with cell response latency. 3. The time course of the population response to the most effective stimuli showed a rapid increase to a peak firing rate (onset to peak, rise time, 58 ms) that was on average 115 spikes/s above spontaneous activity (S/A), followed by slower decay (decay time, 93 ms) to a maintained discharge rate (15% of the peak rate above S/A). 4. Discrimination between responses to different head views exhibited by the population showed a sharp rise and reached highly significant levels within 25 ms after the population's response onset. 5. On average, activity in a single neuron (the Average Cell) rises to 44% of its peak response rate within 5 ms of the response onset. 6. The Average Cell also showed exceptionally fast discrimination between views, significant within 5 ms of response onset. 7. It is argued that the fast rise in firing rate, followed by a decay to a lower rate and the very fast emergence of discrimination are features of pattern processing present in real neural systems that are lacking in many processing models based on artificial networks of neuronlike elements, particularly those where discrimination relies on top-down and/or lateral competitive inhibition. 8. It is concluded that the only way to account for the rapid discrimination is to consider a coding system in which the first spike from multiple sources is used to transmit information between stages of processing.

Neural networks with close nonlocal coupling for analyzing composite image

Peculiarities of designing a multiprocessor computer system for analyzing composite images on the basis of universal homogeneous neural networks with close nonlinear coupling are considered. Some possible patterns of collective activity in those neural networks are investigated. Possibilities of extracting the most typical features of images by separate matrices (two-dimensional layers) of neuron-like elements are shown. Examples of image transformations in the processes of composite image segmentation and pattern recognition in hierarchical neural network systems are given.

A self-learning rule-based controller employing approximate reasoning and neural net concepts

A self-learning controller is proposed as an intelligent controller for dynamic processes, employing a control policy which evolves and improves automatically. A key component of the controller is a rule-based system which provides a linguistic description of control strategy. This strategy has the form of a collection of fuzzy conditional statements which are implemented and manipulated using fuzzy set theory. the inference engine of the controller is based on the principles of approximate reasoning, while its learning capability is provided by neuron-like elements, which are derived from animal conditioning theory. It is shown that the system can solve a fairly difficult control learning problem. More concretely, the task is 1-D pole balancing, in which a pole is hinged to a movable cart to which a continously variable control force is applied. Simulation results demonstrate that improved learning performance can be achieved in relation to previously described systems employing bang-bang control. Furthermore, the proposed controller is relatively insensitive to variations in the parameters of the system, for example, changes in the length and mass of the pole, initial angle, failure criteria, and slanting the base of the car-pole system.

A stochastic version of the delta rule

Self-organizing networks of neuron-like elements naturally lead to high-dimensional, nonlinear parameter spaces which prove difficult to search. Back-propagation is one of the simplest neural network/connectionist models that uses a gradient descent (delta rule) in a high multi-dimensional parameter space. Search in such a space is subject to many difficulties including minima that are locally stable but do not at the same time provide solutions. Search time can also be shown under relatively good conditions to scale poorly. Although gradient descent in error is attractive as a “weak” learning method it also seems to suffer from many search inefficiencies. A principle is proposed about the relation between constrained local noise injections and global search. The delta rule is modified to include synaptic noise in the transmission of information and modification of the connection strength. This stochastic version of the delta rule seems to promote escape from poor locally stable minima, and can improve convergence speed and likehood.

Serotonin‐immunoreactive neurons in the cnidarian Renilla koellikeri

The cellular localization of 5-hydroxytryptamine (5-HT) was investigated in the pennatulid anthozoan Renilla koellikeri by means of peroxidase-antiperoxidase-immunohistochemistry with an antiserum against 5-HT-formaldehyde-protein conjugate. In many colonies, strong 5-HT-immunoreactivity was displayed by the cell bodies and beaded processes of relatively small neuronlike elements predominating in the outer ectoderm or scattered in the underlying mesoglea. The immunostained neurons of the mesoglea were generally bipolar and their relatively short processes extended toward myoepithelial cells. In the ectoderm, most immunostained neurons appeared pseudounipolar. These cell bodies were endowed with a small, superficially directed, conical appendage reaching the outer surface of the epithelium. Their neurites emerged from the inner pole of the perikarya and branched toward other immunopositive ectodermal and mesogleal nerve cells, or nematocytes in the tentacles. The networklike distribution of the presumed 5-HT ectodermal neurons varied between the different regions of colonies and along the autozooid column. In the context of earlier observations in cnidarians, these cytological features suggest a sensory as well as a modulatory function for 5-HT in Renilla koellikeri.

Neural network control of simple limb movements

It is possible to embed the control and computation of a simple single-joint movement at different speeds by a small non-linear network of neuron-like elements. The network "learns" by appropriate adjustment of the strengths of interconnection, or synaptic weights, between the neuron-like elements. The learning of a few movement trajectories is generalized to the learning of a family of unlearned trajectories. These observations are in support of our hypothesis that relaxation of a network from an initial state to a final equilibrium state is both causal and computational to movement generation and control.