Neural Cliques Research Articles

A Vector Space Model for Neural Network Functions: Inspirations From Similarities Between the Theory of Connectivity and the Logarithmic Time Course of Word Production.

The present report examines the coinciding results of two study groups each presenting a power-of-two function to describe network structures underlying perceptual processes in one case and word production during verbal fluency tasks in the other. The former is theorized as neural cliques organized according to the function N = 2i − 1, whereas the latter assumes word conglomerations thinkable as tuples following the function N = 2i. Both theories assume the innate optimization of energy efficiency to cause the specific connectivity structure. The vast resemblance between both formulae motivated the development of a common formulation. This was obtained by using a vector space model, in which the configuration of neural cliques or connected words is represented by a N-dimensional state vector. A further analysis of the model showed that the entire time course of word production could be derived using basically one single minimal transformation-matrix. This again seems in line with the principle of maximum energy efficiency.

Neural Code-Neural Self-information Theory on How Cell-Assembly Code Rises from Spike Time and Neuronal Variability.

A major stumbling block to cracking the real-time neural code is neuronal variability - neurons discharge spikes with enormous variability not only across trials within the same experiments but also in resting states. Such variability is widely regarded as a noise which is often deliberately averaged out during data analyses. In contrast to such a dogma, we put forth the Neural Self-Information Theory that neural coding is operated based on the self-information principle under which variability in the time durations of inter-spike-intervals (ISI), or neuronal silence durations, is self-tagged with discrete information. As the self-information processor, each ISI carries a certain amount of information based on its variability-probability distribution; higher-probability ISIs which reflect the balanced excitation-inhibition ground state convey minimal information, whereas lower-probability ISIs which signify rare-occurrence surprisals in the form of extremely transient or prolonged silence carry most information. These variable silence durations are naturally coupled with intracellular biochemical cascades, energy equilibrium and dynamic regulation of protein and gene expression levels. As such, this silence variability-based self-information code is completely intrinsic to the neurons themselves, with no need for outside observers to set any reference point as typically used in the rate code, population code and temporal code models. Moreover, temporally coordinated ISI surprisals across cell population can inherently give rise to robust real-time cell-assembly codes which can be readily sensed by the downstream neural clique assemblies. One immediate utility of this self-information code is a general decoding strategy to uncover a variety of cell-assembly patterns underlying external and internal categorical or continuous variables in an unbiased manner.

Twin Neurons for Efficient Real-World Data Distribution in Networks of Neural Cliques: Applications in Power Management in Electronic Circuits.

Associative memories are data structures that allow retrieval of previously stored messages given part of their content. They, thus, behave similarly to the human brain's memory that is capable, for instance, of retrieving the end of a song, given its beginning. Among different families of associative memories, sparse ones are known to provide the best efficiency (ratio of the number of bits stored to that of the bits used). Recently, a new family of sparse associative memories achieving almost optimal efficiency has been proposed. Their structure, relying on binary connections and neurons, induces a direct mapping between input messages and stored patterns. Nevertheless, it is well known that nonuniformity of the stored messages can lead to a dramatic decrease in performance. In this paper, we show the impact of nonuniformity on the performance of this recent model, and we exploit the structure of the model to improve its performance in practical applications, where data are not necessarily uniform. In order to approach the performance of networks with uniformly distributed messages presented in theoretical studies, twin neurons are introduced. To assess the adapted model, twin neurons are used with the real-world data to optimize power consumption of electronic circuits in practical test cases.

On the capacity of an associative memory model based on neural cliques

Based on recent work by Gripon and Berrou (2011), we introduce a new model of an associative memory. We give upper and lower bounds on the memory capacity of the model.

Huffman Coding for Storing Non-Uniformly Distributed Messages in Networks of Neural Cliques

Associative memories are data structures that allow retrieval of previously stored messages given part of their content. They thus behave similarly to human brain's memory that is capable for instance of retrieving the end of a song given its beginning. Among different families of associative memories, sparse ones are known to provide the best efficiency (ratio of the number of bits stored to that of bits used). Nevertheless, it is well known that non-uniformity of the stored messages can lead to dramatic decrease in performance. Recently, a new family of sparse associative memories achieving almost-optimal efficiency has been proposed. Their structure induces a direct mapping between input messages and stored patterns. In this work, we show the impact of non-uniformity on the performance of this recent model and we exploit the structure of the model to introduce several strategies to allow for efficient storage of non-uniform messages. We show that a technique based on Huffman coding is the most efficient.

Differential consolidation and pattern reverberations within episodic cell assemblies in the mouse hippocampus.

One hallmark feature of consolidation of episodic memory is that only a fraction of original information, which is usually in a more abstract form, is selected for long-term memory storage. How does the brain perform these differential memory consolidations? To investigate the neural network mechanism that governs this selective consolidation process, we use a set of distinct fearful events to study if and how hippocampal CA1 cells engage in selective memory encoding and consolidation. We show that these distinct episodes activate a unique assembly of CA1 episodic cells, or neural cliques, whose response-selectivity ranges from general-to-specific features. A series of parametric analyses further reveal that post-learning CA1 episodic pattern replays or reverberations are mostly mediated by cells exhibiting event intensity-invariant responses, not by the intensity-sensitive cells. More importantly, reactivation cross-correlations displayed by intensity-invariant cells encoding general episodic features during immediate post-learning period tend to be stronger than those displayed by invariant cells encoding specific features. These differential reactivations within the CA1 episodic cell populations can thus provide the hippocampus with a selection mechanism to consolidate preferentially more generalized knowledge for long-term memory storage.

Revenge of the “Neurds”: Characterizing Creative Thought in Terms of the Structure and Dynamics of Memory

There is cognitive, neurological, and computational support for the hypothesis that defocusing attention results in divergent or associative thought, conducive to insight and finding unusual connections, while focusing attention results in convergent or analytic thought, conducive to rule-based operations. Creativity appears to involve both. It is widely believed that it is possible to escape mental fixation by spontaneously and temporarily engaging in a more associative mode of thought. The resulting insight (if found) may be refined in a more analytic mode of thought. The questions addressed here are: (a) How does the architecture of memory support these two modes of thought, and (b) what is happening at the neural level when one shifts between them? Recent advances in neuroscience shed light on this. Activated cell assemblies are composed of multiple neural cliques, groups of neurons that respond differentially to general or context-specific aspects of a situation. I refer to neural cliques that would not be included in the assembly if one were in an analytic mode, but would be if one were in an associative mode, as neurds. It is posited that the shift to a more associative mode of thought is accomplished by recruiting neurds that respond to abstract or atypical microfeatures of the problem or task. Since memory is distributed and content-addressable, this fosters the forging of associations to potentially relevant items previously encoded in those neurons. Thus it is proposed that creative thought occurs not by searching a space of predefined alternatives and blindly tweaking those that hold promise, but by evoking remotely associated items through the recruitment of neurds in a distributed, content-addressable memory.

The organizing principles of real-time memory encoding: Neural clique assemblies and universal neural codes

The organizing principles of real-time memory encoding: Neural clique assemblies and universal neural codes

Blind separation of spatio-temporal Synfire sources and visualization of neural cliques

A dominating paradigm in neuroscience attributes components of perception and behavior to synchronous spatio-temporal activities of subsets of neurons within neural networks—the so-called Synfire chains. Synfire chains cohere to generate neural cliques within the simultaneously-active Synfires. The present study is concerned with blind separation of Synfire activities, and with identification and visualization of more complex yet coherent neural patterns of activity, to which we refer as neural cliques. Assuming stationarity and linearity, we extend the blind source separation (BSS) technique to the spatio-temporal domain, to deal with dynamic signals, and apply it in our analysis of neural networks. To demonstrate the concept of dynamic BSS, we first apply it in a relatively simple physical example of separation of dynamic reflections, where the physics of the problem is well understood and the assumptions of linearity and stationarity are valid. In the case of neural networks, the problem is much more involved, and the above-mentioned assumptions are not fully justified. Nevertheless, the basic approach yields interesting insight into the function of complex neural activity. To better approximate the function of neural activity, we also investigate the effect of non-linearities on blind separation of neural activity.

Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus

To examine the network-level organizing principles by which the brain achieves its real-time encoding of episodic information, we have developed a 96-channel array to simultaneously record the activity patterns of as many as 260 individual neurons in the mouse hippocampus during various startling episodes. We find that the mnemonic startling episodes triggered firing changes in a set of CA1 neurons in both startle-type and environment-dependent manners. Pattern classification methods reveal that these firing changes form distinct ensemble representations in a low-dimensional encoding subspace. Application of a sliding window technique further enabled us to reliably capture not only the temporal dynamics of real-time network encoding but also postevent processing of newly formed ensemble traces. Our analyses revealed that the network-encoding power is derived from a set of functional coding units, termed neural cliques, in the CA1 network. The individual neurons within neural cliques exhibit "collective cospiking" dynamics that allow the neural clique to overcome the response variability of its members and to achieve real-time encoding robustness. Conversion of activation patterns of these coding unit assemblies into a set of real-time digital codes permits concise and universal representation and categorization of discrete behavioral episodes across different individual brains.