Maximize Information Transfer Research Articles

Integration of Sensory Quanta in Cuneate Nucleus Neurons In Vivo

Discriminative touch relies on afferent information carried to the central nervous system by action potentials (spikes) in ensembles of primary afferents bundled in peripheral nerves. These sensory quanta are first processed by the cuneate nucleus before the afferent information is transmitted to brain networks serving specific perceptual and sensorimotor functions. Here we report data on the integration of primary afferent synaptic inputs obtained with in vivo whole cell patch clamp recordings from the neurons of this nucleus. We find that the synaptic integration in individual cuneate neurons is dominated by 4–8 primary afferent inputs with large synaptic weights. In a simulation we show that the arrangement with a low number of primary afferent inputs can maximize transfer over the cuneate nucleus of information encoded in the spatiotemporal patterns of spikes generated when a human fingertip contact objects. Hence, the observed distributions of synaptic weights support high fidelity transfer of signals from ensembles of tactile afferents. Various anatomical estimates suggest that a cuneate neuron may receive hundreds of primary afferents rather than 4–8. Therefore, we discuss the possibility that adaptation of synaptic weight distribution, possibly involving silent synapses, may function to maximize information transfer in somatosensory pathways.

Quantifying and Tracing Information Cascades in Swarms

We propose a novel, information-theoretic, characterisation of cascades within the spatiotemporal dynamics of swarms, explicitly measuring the extent of collective communications. This is complemented by dynamic tracing of collective memory, as another element of distributed computation, which represents capacity for swarm coherence. The approach deals with both global and local information dynamics, ultimately discovering diverse ways in which an individual’s spatial position is related to its information processing role. It also allows us to contrast cascades that propagate conflicting information with waves of coordinated motion. Most importantly, our simulation experiments provide the first direct information-theoretic evidence (verified in a simulation setting) for the long-held conjecture that the information cascades occur in waves rippling through the swarm. Our experiments also exemplify how features of swarm dynamics, such as cascades’ wavefronts, can be filtered and predicted. We observed that maximal information transfer tends to follow the stage with maximal collective memory, and principles like this may be generalised in wider biological and social contexts.

A small world of weak ties provides optimal global integration of self-similar modules in functional brain networks

The human brain is organized in functional modules. Such an organization presents a basic conundrum: Modules ought to be sufficiently independent to guarantee functional specialization and sufficiently connected to bind multiple processors for efficient information transfer. It is commonly accepted that small-world architecture of short paths and large local clustering may solve this problem. However, there is intrinsic tension between shortcuts generating small worlds and the persistence of modularity, a global property unrelated to local clustering. Here, we present a possible solution to this puzzle. We first show that a modified percolation theory can define a set of hierarchically organized modules made of strong links in functional brain networks. These modules are "large-world" self-similar structures and, therefore, are far from being small-world. However, incorporating weaker ties to the network converts it into a small world preserving an underlying backbone of well-defined modules. Remarkably, weak ties are precisely organized as predicted by theory maximizing information transfer with minimal wiring cost. This trade-off architecture is reminiscent of the "strength of weak ties" crucial concept of social networks. Such a design suggests a natural solution to the paradox of efficient information flow in the highly modular structure of the brain.

Learning to Learn: The Practical Application of Learning Theory (511)

1.Describe the steps of the learning cycle.2.Describe a technique for building on learners’ existing knowledge.3.Identify and describe three strategies to increase learner involvement in the learning process including no opt out, cold call and ratio. Successful learning relies on gathering information, storing and accessing facts in long term memory and efficiently using working memory. Components of the learning cycle include concrete experience, reflective observation, abstract hypotheses and active testing. Recent educational theory and data focus on applying strategies aimed at different points in the cycle to increase knowledge transfer. Two highlighted techniques are (a) building on previous knowledge and (b) requiring active engagement. Educational techniques that identify and add to learners’ existing neuronal networks maximize information transfer. These techniques include assessing learners’ prior knowledge, asking what new information brings to mind, asking peers to describe connections they have made, and using metaphors/similes/analogies to make connections between the familiar and new. Active participation increases learning. These techniques include no opt out, cold calling and ratio. No opt out guards against unwilling or unknowledgeable learners and encourages learners to participate in the learning cycle by repeating the correct answer even if he/she first responded, “I don't know.” Through cold calling, educators ask a question and, regardless of raised hands, call on someone. This systematic and scaffolded (easier followed by harder questions) technique requires all learners to go through learning cycle. Ratio requires the teacher consider the amount of work the student must do to arrive at an answer. Educators maximize the learner/teacher effort ratio by using half-statements which require learners to finish, feigning ignorance or asking for supporting evidence. In this session, participants will apply educational precepts to teaching palliative care. This will be done using the following methods: (a) brief didactic description of advances in education that stress the importance of integrating new information into prior learning and active engagement in the learning process, (b) description of techniques that master educators use to operationalize these precepts, (c) watching videotapes of master educators, and (d) practicing these techniques.

The Conundrum of Functional Brain Networks: Small-World Efficiency or Fractal Modularity

The human brain has been studied at multiple scales, from neurons, circuits, areas with well-defined anatomical and functional boundaries, to large-scale functional networks which mediate coherent cognition. In a recent work, we addressed the problem of the hierarchical organization in the brain through network analysis. Our analysis identified functional brain modules of fractal structure that were inter-connected in a small-world topology. Here, we provide more details on the use of network science tools to elaborate on this behavior. We indicate the importance of using percolation theory to highlight the modular character of the functional brain network. These modules present a fractal, self-similar topology, identified through fractal network methods. When we lower the threshold of correlations to include weaker ties, the network as a whole assumes a small-world character. These weak ties are organized precisely as predicted by theory maximizing information transfer with minimal wiring costs.

The Effects of Unilateral Naris Occlusion on Gene Expression Profiles in Mouse Olfactory Mucosa

Unilateral naris occlusion has been the method of choice for effecting stimulus deprivation in studies of olfactory plasticity. Early experiments emphasized the deleterious effects of this technique on the developing olfactory system while more recent studies have pointed to several apparently "compensatory" responses. However, the evidence for deprivation-induced compensatory processes in olfaction remains fragmentary. High-throughput methods such as microarray analysis can help fill the deficits in our understanding of naris occlusion as a mode of stimulus deprivation. Here we report for young adult mice the effects of early postnatal naris occlusion on the olfactory mucosal transcriptome using microarray analysis with RT-PCR confirmation. The transcripts of key genes involved in olfactory reception, transduction, and transmission were up-regulated in deprived-side olfactory mucosa, with opposite effects in non-deprived-side mucosa, compared to controls. Results support the hypothesis that odor environment triggers a previously unknown homeostatic control mechanism in olfactory receptor neurons designed to maximize information transfer.

Short-Term Plasticity Optimizes Synaptic Information Transmission

Short-term synaptic plasticity (STP) is widely thought to play an important role in information processing. This major function of STP has recently been challenged, however, by several computational studies indicating that transmission of information by dynamic synapses is broadband, i.e., frequency independent. Here we developed an analytical approach to quantify time- and rate-dependent synaptic information transfer during arbitrary spike trains using a realistic model of synaptic dynamics in excitatory hippocampal synapses. We found that STP indeed increases information transfer in a wide range of input rates, which corresponds well to the naturally occurring spike frequencies at these synapses. This increased information transfer is observed both during Poisson-distributed spike trains with a constant rate and during naturalistic spike trains recorded in hippocampal place cells in exploring rodents. Interestingly, we found that the presence of STP in low release probability excitatory synapses leads to optimization of information transfer specifically for short high-frequency bursts, which are indeed commonly observed in many excitatory hippocampal neurons. In contrast, more reliable high release probability synapses that express dominant short-term depression are predicted to have optimal information transmission for single spikes rather than bursts. This prediction is verified in analyses of experimental recordings from high release probability inhibitory synapses in mouse hippocampal slices and fits well with the observation that inhibitory hippocampal interneurons do not commonly fire spike bursts. We conclude that STP indeed contributes significantly to synaptic information transfer and may serve to maximize information transfer for specific firing patterns of the corresponding neurons.

Maximal Information Transfer and Behavior Diversity in Random Threshold Networks

Random Threshold Networks (RTNs) are an idealized model of diluted, non-symmetric spin glasses, neural networks or gene regulatory networks. RTNs also serve as an interesting general example of any coordinated causal system. Here we study the conditions for maximal information transfer and behavior diversity in RTNs. These conditions are likely to play a major role in physical and biological systems, perhaps serving as important selective traits in biological systems. We show that the pairwise mutual information is maximized in dynamically critical networks. Also, we show that the correlated behavior diversity is maximized for slightly chaotic networks, close to the critical region. Importantly, critical networks maximize coordinated, diverse dynamical behavior across the network and across time: the information transmission between source and receiver nodes and the diversity of dynamical behaviors, when measured with a time delay between the source and receiver, are maximized for critical networks.

Bayesian Spiking Neurons II: Learning

In the companion letter in this issue ("Bayesian Spiking Neurons I: Inference"), we showed that the dynamics of spiking neurons can be interpreted as a form of Bayesian integration, accumulating evidence over time about events in the external world or the body. We proceed to develop a theory of Bayesian learning in spiking neural networks, where the neurons learn to recognize temporal dynamics of their synaptic inputs. Meanwhile, successive layers of neurons learn hierarchical causal models for the sensory input. The corresponding learning rule is local, spike-time dependent, and highly nonlinear. This approach provides a principled description of spiking and plasticity rules maximizing information transfer, while limiting the number of costly spikes, between successive layers of neurons.

Sound production and spectral hearing sensitivity in the Hawaiian sergeant damselfish, Abudefduf abdominalis

Sounds provide important signals for inter- and intraspecific communication in fishes, but few studies examine fish acoustic behavior in the context of coevolution of sound production and hearing ability within a species. This study characterizes the acoustic behavior in a reproductive population of the Hawaiian sergeant fish, Abudefduf abdominalis, and compares acoustic features to hearing ability, measured by the auditory evoked potential (AEP) technique. Sergeant fish produce sounds at close distances to the intended receiver (<or=1-2 body lengths), with different pulse characteristics that are associated primarily with aggression, nest preparation and courtship-female-visit behaviors. Energy peaks of all sounds were between 90 and 380 Hz, whereas courtship-visit sounds had a pulse repetition rate of 125 Hz with harmonic intervals up to 1 kHz. AEP threshold, which is probably higher than the behavioral threshold, indicates best sensitivity at low frequencies (95-240 Hz), with the lowest threshold at 125 Hz (123-127 dB(rms) re: 1 microPa). Thus, sound production and hearing in A. abdominalis are closely matched in the frequency domain and are useful for courtship and mating at close distances. Measured hearing thresholds did not differ among males and females during spawning or non-spawning periods, which indicates a lack of sex differences and seasonal variation in hearing capabilities. These data provide the first evidence that Abudefduf uses true acoustic communication on a level similar to that of both more derived (e.g. Dascyllus, Chromis) and more basal (e.g. Stegastes) soniferous pomacentrids. This correlation between sound production and hearing ability is consistent with the sensory drive model of signal evolution in which the sender and receiver systems coevolve within the constraints of the environment to maximize information transfer of acoustic signals.

Information maximization in face processing

This perspective paper explores principles of unsupervised learning and how they relate to face recognition. Dependency coding and information maximization appear to be central principles in neural coding early in the visual system. These principles may be relevant to how we think about higher visual processes such as face recognition as well. The paper first reviews examples of dependency learning in biological vision, along with principles of optimal information transfer and information maximization. Next, we examine algorithms for face recognition by computer from a perspective of information maximization. The eigenface approach can be considered as an unsupervised system that learns the first- and second-order dependencies among face image pixels. Eigenfaces maximize information transfer only in the case where the input distributions are Gaussian. Independent component analysis (ICA) learns high-order dependencies in addition to first- and second-order relations, and maximizes information transfer for a more general set of input distributions. Face representations based on ICA gave better recognition performance than eigenfaces, supporting the theory that information maximization is a good strategy for high level visual functions such as face recognition. Finally, we review perceptual studies suggesting that dependency learning is relevant to human face perception as well, and present an information maximization account of perceptual effects such as the atypicality bias, and face adaptation aftereffects.

Effective Information Transfer from the α1b-Adrenoceptor to Gα11 Requires Both β/γ Interactions and an Aromatic Group Four Amino Acids from the C Terminus of the G Protein

Co-expression of the alpha(1b)-adrenoreceptor and Galpha(11) in cells derived from a Galpha(q)/Galpha(11) knock-out mouse allows agonist-mediated elevation of intracellular Ca(2+) levels that is transduced by beta/gamma released from the G protein alpha subunit. Mutation of Tyr(356) of Galpha(11) to Phe, within a receptor contact domain, had little effect on function but this was reduced greatly by alteration to Ser and virtually eliminated by conversion to Asp. This pattern was replicated following incorporation of each form of Galpha(11) into fusion proteins with the alpha(1b)-adrenoreceptor. Following a [(35)S]guanosine 5'-O-(3-thiotriphosphate) (GTPgammaS) binding assay, immunoprecipitation of the wild type alpha(1b)-adrenoreceptor-Galpha(11) fusion protein indicated that the agonist phenylephrine stimulated guanine nucleotide exchange on Galpha(11) more than 30-fold. Information transfer by agonist was controlled in residue 356 Galpha(11) mutants with rank order Tyr > Phe > Trp > Ile > Ala = Gln = Arg > Ser > Asp, although these alterations did not alter the binding affinity of either phenylephrine or an antagonist ligand. Mutation of a beta/gamma contact interface in the alpha(1b)-adrenoreceptor-Tyr(356) Galpha(11) fusion protein did not alter ligand binding affinity but did reduce greatly beta/gamma binding and phenylephrine stimulation of [(35)S]GTPgammaS binding. It also prevented agonist elevation of intracellular Ca(2+) levels, as did a mutation in Galpha(11) that prevents G protein subunit dissociation. These results indicate that a bulky aromatic group is required four amino acids from the C terminus of Galpha(11) to maximize information transfer from an agonist-occupied receptor and disprove the hypothesis that tyrosine phosphorylation of this residue is required for G protein activation (Umemori, H., Inoue, T., Kume, S., Sekiyama, N., Nagao, M., Itoh, H., Nakanishi, S., Mikoshiba, K., and Yamamoto, T. (1997) Science 276, 1878-1881). This is distinct from Galpha(i1), where hydrophobicity of the amino acid is the key determinant at this location. They also further demonstrate a key role for the beta/gamma complex in enhancing receptor to G protein alpha subunit information transfer.

Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control.

Marr [J. Physiol. (1969) 202, 437-470] and Albus [Math. Biosci. (1971) 10, 25-61] hypothesized that cerebellar learning is facilitated by a granule cell sparse code, i.e. a neural code in which the fraction of active neurons is low at any one time. In this paper, we re-examine this hypothesis in light of recent experimental and theoretical findings. We argue that cerebellar motor learning is enhanced by a sparse code that simultaneously maximizes information transfer between mossy fibers and granule cells, minimizes redundancies between granule cell discharges, and re-codes the mossy fiber inputs with an adaptive resolution such that inputs corresponding to large errors are finely encoded. We then propose that a set of biologically plausible unsupervised learning rules can produce such a code. To maintain a low mean firing rate compatible with a sparse code, an activity-dependent homeostatic mechanism sets the cells' thresholds. Then, to maximize information transfer, the mossy fiber--granule cell synapses are adjusted by a Hebbian rule. Furthermore, to minimize redundancies between granule cell discharges, the inhibitory Golgi cell--granule cell synapses are tuned by an anti-Hebbian rule. Finally, to allow adaptive resolution, a performance-based neuromodulator-like signal gates these three plastic processes. We integrate these gated learning rules into a simplified model of the cerebellum for arm movement control, and show that unsupervised learning of granule cell sparse codes greatly improves cerebellar adaptive motor control in comparison to a "fixed" Marr--Albus-type model. Until recently, activity-dependent cerebellar plasticity was thought to be largely confined to the granule cell--Purkinje cell synapses. This static view of the cerebellum is, however, quickly being replaced by an extremely dynamic view in which plasticity is omnipresent. The present theoretical study shows how several forms of plasticity in the granular layer of the cerebellum can produce fast, accurate and stable cerebellar learning.

How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate.

Information from the senses must be compressed into the limited range of responses that spiking neurons can generate. For optimal compression, the neuron's response should match the statistics of stimuli encountered in nature. Given a maximum firing rate, a nerve cell should learn to use each available firing rate equally often. Given a set mean firing rate, it should self-organize to respond with high firing rates only to comparatively rare events. Here we derive an unsupervised learning rule that continuously adapts membrane conductances of a Hodgkin-Huxley model neuron to optimize the representation of sensory information in the firing rate. Maximizing information transfer between the stimulus and the cell's firing rate can be interpreted as a non-Hebbian developmental mechanism.

Development of low entropy coding in a recurrent network

In this paper we present an unsupervised neural network which exhibits competition between units via inhibitory feedback. The operation is such as to minimize reconstruction error, both for individual patterns, and over the entire training set. A key difference from networks which perform principal components analysis, or one of its variants, is the ability to converge to non-orthogonal weight values. We discuss the network's operation in relation to the twin goals of maximizing information transfer and minimizing code entropy, and show how the assignment of prior probabilities to network outputs can help to reduce entropy. We present results from two binary coding problems, and from experiments with image coding.

Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer

We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focusing on the case of nonlinear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker's infomax principle) leads to a factorial code-hence to the same solution as required by the redundancy-reduction principle of Barlow. We also show that this result is valid for linear and, more generally, unbounded, transfer functions, provided optimization is performed under an additive constraint, i.e. which can be written as a sum of terms, each one being specific to one output neuron. Finally, we study the effect of a non-zero input noise. We find that, to first order in the input noise, assumed to be small in comparison with the (small) output noise, the above results are still valid, provided the output noise is uncorrelated from one neuron to the other.

Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer

We investigate the consequences of maximizing information transfer in a simple neural network (one input layer, one output layer), focusing on the case of nonlinear transfer functions. We assume that both receptive fields (synaptic efficacies) and transfer functions can be adapted to the environment. The main result is that, for bounded and invertible transfer functions, in the case of a vanishing additive output noise, and no input noise, maximization of information (Linsker's infomax principle) leads to a factorial code-hence to the same solution as required by the redundancy-reduction principle of Barlow. We also show that this result is valid for linear and, more generally, unbounded, transfer functions, provided optimization is performed under an additive constraint, i.e. which can be written as a sum of terms, each one being specific to one output neuron. Finally, we study the effect of a non-zero input noise. We find that, to first order in the input noise, assumed to be small in comparison with th...

Automatic target recognition using a passive multisensor suite

An ongoing study of automatic target recognition using a passive multisensor suite is presented. The multisensor suite consists of electro-optical and acoustic sensors, a radio frequency interferometer, and meteorological sensors. Three feature analysis tools-a genetic algorithm, a branch-and-bound algorithm, and a new class overlap region partitioning scheme (CORPS)-are employed for assessing the effectiveness of features extracted by several data and feature fusion concepts. Dedicated hardware has been configured for laboratory and field testing. A high-resolution graphics display has been designed for monitoring the target-acquisition-related processes and maximizing information transfer to the operator.

Writing memorable geophysical papers: The need for proper author coalitions

A primary function of Eos is to serve the geophysical community. It does this by publishing meeting announcements, book reviews, advertisements for jobs, scientific news items, and the like. Recent articles have helped the membership assess the stage of their careers (Eos, 60, 1024, 1979), informed them of the advantages of having names near the beginning of the alphabet (Eos, 59, 118, 1978), and helped them maximize information transfer during scientific meetings (Eos, 62, 179, 1981). However, no one has dealt with the very difficult problem of making papers memorable. Some techniques, such as long author lists, are now passé. Everyone is doing it. Other techniques, such as writing a very short paper or a humorous paper, are beyond the ken of most AGU members. Fortunately, there remains one technique that can be used by a surprisingly large number of AGU members.