Sparse Distributed Representation Research Articles

Hierarchical intrinsically motivated agent planning behavior with dreaming in grid environments

Biologically plausible models of learning may provide a crucial insight for building autonomous intelligent agents capable of performing a wide range of tasks. In this work, we propose a hierarchical model of an agent operating in an unfamiliar environment driven by a reinforcement signal. We use temporal memory to learn sparse distributed representation of state–actions and the basal ganglia model to learn effective action policy on different levels of abstraction. The learned model of the environment is utilized to generate an intrinsic motivation signal, which drives the agent in the absence of the extrinsic signal, and through acting in imagination, which we call dreaming. We demonstrate that the proposed architecture enables an agent to effectively reach goals in grid environments.

Memory Model for Morphological Semantics of Visual Stimuli Using Sparse Distributed Representation

Recent achievements on CNN (convolutional neural networks) and DNN (deep neural networks) researches provide a lot of practical applications on computer vision area. However, these approaches require construction of huge size of training data for learning process. This paper tries to find a way for continual learning which does not require prior high-cost training data construction by imitating a biological memory model. We employ SDR (sparse distributed representation) for information processing and semantic memory model, which is known as a representation model of firing patterns on neurons in neocortex area. This paper proposes a novel memory model to reflect remembrance of morphological semantics of visual input stimuli. The proposed memory model considers both memory process and recall process separately. First, memory process converts input visual stimuli to sparse distributed representation, and in this process, morphological semantic of input visual stimuli can be preserved. Next, recall process can be considered by comparing sparse distributed representation of new input visual stimulus and remembered sparse distributed representations. Superposition of sparse distributed representation is used to measure similarities. Experimental results using 10,000 images in MNIST (Modified National Institute of Standards and Technology) and Fashion-MNIST data sets show that the sparse distributed representation of the proposed model efficiently keeps morphological semantic of the input visual stimuli.

Anomalous Behavior Detection Framework Using HTM-Based Semantic Folding Technique

Upon the working principles of the human neocortex, the Hierarchical Temporal Memory model has been developed which is a proposed theoretical framework for sequence learning. Both categorical and numerical types of data are handled by HTM. Semantic Folding Theory (SFT) is based on HTM to represent a data stream for processing in the form of sparse distributed representation (SDR). For natural language perception and production, SFT delivers a solid structural background for semantic evidence description to the fundamentals of the semantic foundation during the phase of language learning. Anomalies are the patterns from data streams that do not follow the expected behavior. Any stream of data patterns could have a number of anomaly types. In a data stream, a single pattern or combination of closely related patterns that diverges and deviates from standard, normal, or expected is called a static (spatial) anomaly. A temporal anomaly is a set of unexpected changes between patterns. When a change first appears, this is recorded as an anomaly. If this change looks a number of times, then it is set to a “new normal” and terminated as an anomaly. An HTM system detects the anomaly, and due to continuous learning nature, it quickly learns when they become the new normal. A robust anomalous behavior detection framework using HTM-based SFT for improving decision-making (SDR-ABDF/P2) is a proposed framework or model in this research. The researcher claims that the proposed model would be able to learn the order of several variables continuously in temporal sequences by using an unsupervised learning rule.

A Fast Spatial Pool Learning Algorithm of Hierarchical Temporal Memory Based on Minicolumn's Self-Nomination.

As a new type of artificial neural network model, HTM has become the focus of current research and application. The sparse distributed representation is the basis of the HTM model, but the existing spatial pool learning algorithms have high training time overhead and may cause the spatial pool to become unstable. To overcome these disadvantages, we propose a fast spatial pool learning algorithm of HTM based on minicolumn's nomination, where the minicolumns are selected according to the load-carrying capacity and the synapses are adjusted using compressed encoding. We have implemented the prototype of the algorithm and carried out experiments on three datasets. It is verified that the training time overhead of the proposed algorithm is almost unaffected by the encoding length, and the spatial pool becomes stable after fewer iterations of training. Moreover, the training of the new input does not affect the already trained results.

Hierarchical Temporal Memory-Based One-Pass Learning for Real-Time Anomaly Detection and Simultaneous Data Prediction in Smart Grids

A neuro-cognitive inspired architecture based on the Hierarchical Temporal Memory (HTM) is proposed for anomaly detection and simultaneous data prediction in real-time for smart grid <inline-formula><tex-math notation="LaTeX">$\mu$</tex-math><alternatives><mml:math><mml:mi>&#x03BC;</mml:mi></mml:math><inline-graphic xlink:href="barua-ieq1-3037054.gif"/></alternatives></inline-formula>PMU data. The key technical idea is that the HTM learns a <i>sparse distributed temporal representation</i> of sequential data that turns out to be very useful for anomaly detection and simultaneous data prediction in real-time. Our results show that the proposed HTM can predict anomalies within 83&#x2013;90 percent accuracy for three different application profiles, namely <i>Standard, Reward Few False Positive, Reward Few False Negative</i> for two different datasets. We show that the HTM is competitive to five state-of-the-art algorithms for anomaly detection. Moreover, for the multi-step prediction in the online setting, the same HTM achieves a low 0.0001 normalized mean square error, a low negative log-likelihood score of 1.5 and is also competitive to six state-of-the-art prediction algorithms. We demonstrate that the same HTM model can be used for <i>both the tasks</i> and can learn online in one-pass, in an unsupervised fashion and adapt to changing statistics. <i>The other state-of-the-art algorithms are either less accurate or are limited to one of the tasks or cannot learn online in one-pass, and adapt to changing statistics.</i>

View and Clothing Invariant Gait Recognition via 3D Human Semantic Folding

A novel 3-dimensional (3D) human semantic folding is introduced to provide a robust and efficient gait recognition method which is invariant to camera view and clothing style. The proposed gait recognition method comprises three modules: (1) 3D body pose, shape and viewing data estimation network (3D-BPSVeNet); (2) gait semantic parameter folding model; and (3) gait semantic feature refining network. First, 3D-BPSVeNet is constructed based on a convolution gated recurrent unit (ConvGRU) to extract 2-dimensional (2D) to 3D body pose and shape semantic descriptors (2D-3D-BPSDs) from a sequence of gait parsed RGB images. A 3D gait model with virtual dressing is then constructed by morphing the template of 3D body model using the estimated 2D-3D-BPSDs and the recognized clothing styles. The more accurate 2D-3D-BPSDs without clothes are then obtained by using the silhouette similarity function when updating the 3D body model to fit the 2D gait. Second, the intrinsic 2D-3D-BPSDs without interference from clothes are encoded by sparse distributed representation (SDR) to gain the binary gait semantic image (SD-BGSI) in a topographical semantic space. By averaging the SD-BGSIs in a gait cycle, a gait semantic folding image (GSFI) is obtained to give a high-level representation of gait. Third, a gait semantic feature refining network is trained to refine the semantic feature extracted directly from GSFI using three types of prior knowledge, i.e., viewing angles, clothing styles and carrying condition. Experimental analyses on CMU MoBo, CASIA B, KY4D, OU-MVLP and OU-ISIR datasets show a significant performance gain in gait recognition in terms of accuracy and robustness.

Neuromemrisitive Architecture of HTM with On-Device Learning and Neurogenesis

Hierarchical temporal memory (HTM) is a biomimetic sequence memory algorithm that holds promise for invariant representations of spatial and spatio-temporal inputs. This article presents a comprehensive neuromemristive crossbar architecture for the spatial pooler (SP) and the sparse distributed representation classifier, which are fundamental to the algorithm. There are several unique features in the proposed architecture that tightly link with the HTM algorithm. A memristor that is suitable for emulating the HTM synapses is identified and a new Z-window function is proposed. The architecture exploits the concept of synthetic synapses to enable potential synapses in the HTM. The crossbar for the SP avoids dark spots caused by unutilized crossbar regions and supports rapid on-chip training within two clock cycles. This research also leverages plasticity mechanisms such as neurogenesis and homeostatic intrinsic plasticity to strengthen the robustness and performance of the SP. The proposed design is benchmarked for image recognition tasks using Modified National Institute of Standards and Technology (MNIST) and Yale faces datasets, and is evaluated using different metrics including entropy, sparseness, and noise robustness. Detailed power analysis at different stages of the SP operations is performed to demonstrate the suitability for mobile platforms.

Academic Collaboration Recommendation Based on Sparse Distributed Representation

The researchers with similar research contents are helpful for understanding the relevant field and the promotion of further exchanges and collaborations. Find proper researchers is a complex task, we proposed an innovative approach of collaboration recommendation based on sparse distributed representation (SDR). According to the sparse distributed representation theory, the text contents of authors' thesis are characterized to vectors, then recommend collaborations based on the similarity of vectors. We also validate our approach using the data set of NIPS, the experimental results show that the method we proposed is effectively used for collaborative recommendation.

Spatial-Pooling Memristor Crossbar Converting Sensory Information to Sparse Distributed Representation of Cortical Neurons

The spatial pooler in brain's neocortex models how the cortical neurons learn the feedforward connections and forms the cortical representation from various sensory information. To realize the spatial pooler's operation in hardware, we propose a new spatial-pooling memristor crossbar that converts the inputs from sensory organ into the sparse distributed representation (SDR) of cortical neurons, in this paper. The spatial pooling is composed of overlapping, inhibition, and learning steps. The proposed memristor crossbar can perform exactly the same functions of overlapping, inhibition, and learning with the spatial-pooling software algorithm. By converting the sensory information to SDR using the spatial-pooling memristor crossbar, we can have the following advantages for the cortical information processing: first, preserving the semantic similarity through the spatial pooling; second, changing synaptic weights continuously; third, keeping the sparsity of SDR as low as 2% like brain's neocortex; and fourth, improving the noise robustness. In this paper, these four properties of the spatial pooling in brain's neocortex have been simulated and verified in the memristor crossbar circuit. For verifying the spatial-pooling operation, the memristor crossbar has been tested for MNIST image recognition in this paper. The recognition rate is as high as 95% for the memristor crossbar with 4096 columns.

Toward navigation ability for autonomous mobile robots with learning from demonstration paradigm

Learning from demonstration, as an important component of imitation learning, is a paradigm for robot to learn new tasks. Considering the application of learning from demonstration in the navigation issue, the robot can also acquire the navigation task via the human teacher’s demonstration. Based on research of the human brain neocortex, in this article, we present a learning from demonstration navigation paradigm from the perspective of hierarchical temporal memory theory. As a type of end-to-end learning form, the demonstrated relationship between perception data and motion commands will be learned and predicted by using hierarchical temporal memory. This framework first perceives images to obtain the corresponding categories information; then the categories incorporated with depth and motion command data are encoded as a sequence of sparse distributed representation vectors. The sequential vectors are treated as the inputs to train the navigation hierarchical temporal memory. After the training, the navigation hierarchical temporal memory stores the transitions of the perceived images, depth, and motion data so that future motion commands can be predicted. The performance of the proposed navigation strategy is evaluated via the real experiments and the public data sets.

InvarNet-X: A Black-Box Invariant-Based Approach to Diagnosing Big Data Systems

As big data spreads rapidly, performance problems in these systems become common concerns. As the first line of defending these problems, performance diagnosis plays an essential role in big data systems. It is notoriously difficult to conduct performance diagnosis in large distributed systems. Previous work either pinpoint the root causes by instrumenting the applications or runtime systems in a white-box way, which leads to a considerable overhead, or just provide some hints to the hidden root causes in a black-box way. Very few works focus on pinpointing the real root causes in a black-box way. To address this problem, this paper proposes a black-box invariant-based diagnosing approach and implements a proof-of-concept system named InvarNet-X . In this paper, performance diagnosis is formalized as a pattern recognition problem, meaning that each performance problem is identified by a specific pattern. The rationale of InvarNet-X is that the unobservable root causes of performance problems always expose themselves through the violations of the associations among directly observable performance metrics. Such observable associations are called likely invariants calculated by the maximal information criterion, and each performance problem is signified by a sparse distributed representation. A problem signature database is constructed by training multiple real performance problems in advance. Once a performance anomaly is detected, the diagnosing procedure is triggered. The root cause is pinpointed by retrieving similar signatures in the signature database. The experimental evaluations in a controlled big data system show that InvarNet-X can achieve a high accuracy in diagnosing some real performance problems reported in software bug repositories, which is superior to several state-of-the-art approaches. Moreover, the light-weight property makes InvarNet-X easily facilitated in large-scale big data systems in real time.

Skill Learning for Intelligent Robot by Perception-Action Integration: A View from Hierarchical Temporal Memory

Skill learning autonomously through interactions with the environment is a crucial ability for intelligent robot. A perception-action integration or sensorimotor cycle, as an important issue in imitation learning, is a natural mechanism without the complex program process. Recently, neurocomputing model and developmental intelligence method are considered as a new trend for implementing the robot skill learning. In this paper, based on research of the human brain neocortex model, we present a skill learning method by perception-action integration strategy from the perspective of hierarchical temporal memory (HTM) theory. The sequential sensor data representing a certain skill from a RGB-D camera are received and then encoded as a sequence of Sparse Distributed Representation (SDR) vectors. The sequential SDR vectors are treated as the inputs of the perception-action HTM. The HTM learns sequences of SDRs and makes predictions of what the next input SDR will be. It stores the transitions of the current perceived sensor data and next predicted actions. We evaluated the performance of this proposed framework for learning the shaking hands skill on a humanoid NAO robot. The experimental results manifest that the skill learning method designed in this paper is promising.

Pattern separation, completion, and categorisation in the hippocampus and neocortex

The mechanisms for pattern completion and pattern separation are described in the context of a theory of hippocampal function in which the hippocampal CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial, associations between any spatial location (place in rodents, or spatial view in primates) and an object or reward, and to provide for completion of the whole memory during recall from any part. The factors important in the pattern completion in CA3 and also a large number of independent memories stored in CA3 include: a sparse distributed representation, representations that are independent due to the randomizing effect of the mossy fibres, heterosynaptic long-term depression as well as long-term potentiation in the recurrent collateral synapses, and diluted connectivity to minimize the number of multiple synapses between any pair of CA3 neurons which otherwise distort the basins of attraction. Recall of information from CA3 is implemented by the entorhinal cortex perforant path synapses to CA3 cells, which in acting as a pattern associator allow some pattern generalization. Pattern separation is performed in the dentate granule cells using competitive learning to convert grid-like entorhinal cortex firing to place-like fields, and in the dentate to CA3 connections that have diluted connectivity. Recall to the neocortex is achieved by a reverse hierarchical series of pattern association networks implemented by the hippocampo-cortical backprojections, each one of which performs some pattern generalization, to retrieve a complete pattern of cortical firing in higher-order cortical areas. New results on competitive networks show which factors contribute to their ability to perform pattern separation, pattern clustering, and pattern categorisation, and how these apply in different hippocampal and neocortical systems.

Enhancement of Classifiers in HTM-CLA Using Similarity Evaluation Methods

Abstract The recent development in the theory of Hierarchical Temporal Memory (HTM) - Cortical Learning Algorithms (CLA) which models the structural and algorithmic properties of neocortex has brought in a new paradigm in the field of machine intelligence. As the theory of HTM-CLA continues to evolve, the ways of inferring the patterns and structures recognized by the HTM-CLA algorithm are still a big challenge. Moreover, the existing methods used to infer the classification output from HTM-CLA are far from satisfactory. In this paper, we propose two new classifiers using similarity evaluation methods based on dot similarity (H-DS) and mean-shift clustering (H-MSC) to obtain classification from the sparse distributed representation (SDR) output of HTM-CLA. We validate and benchmark the performance of our proposed classifiers using three datasets from the UCI machine learning repository. The results show that the proposed classifiers enhance the classification performance of HTM-CLA and their performance is also comparable to other traditional machine learning technique such as decision tree.

The mechanisms for pattern completion and pattern separation in the hippocampus.

The mechanisms for pattern completion and pattern separation are described in the context of a theory of hippocampal function in which the hippocampal CA3 system operates as a single attractor or autoassociation network to enable rapid, one-trial, associations between any spatial location (place in rodents, or spatial view in primates) and an object or reward, and to provide for completion of the whole memory during recall from any part. The factors important in the pattern completion in CA3 together with a large number of independent memories stored in CA3 include a sparse distributed representation which is enhanced by the graded firing rates of CA3 neurons, representations that are independent due to the randomizing effect of the mossy fibers, heterosynaptic long-term depression as well as long-term potentiation in the recurrent collateral synapses, and diluted connectivity to minimize the number of multiple synapses between any pair of CA3 neurons which otherwise distort the basins of attraction. Recall of information from CA3 is implemented by the entorhinal cortex perforant path synapses to CA3 cells, which in acting as a pattern associator allow some pattern generalization. Pattern separation is performed in the dentate granule cells using competitive learning to convert grid-like entorhinal cortex firing to place-like fields. Pattern separation in CA3, which is important for completion of any one of the stored patterns from a fragment, is provided for by the randomizing effect of the mossy fiber synapses to which neurogenesis may contribute, by the large number of dentate granule cells each with a sparse representation, and by the sparse independent representations in CA3. Recall to the neocortex is achieved by a reverse hierarchical series of pattern association networks implemented by the hippocampo-cortical backprojections, each one of which performs some pattern generalization, to retrieve a complete pattern of cortical firing in higher-order cortical areas.

Representation and processing of structures with binary sparse distributed codes

The schemes for compositional distributed representations include those allowing on-the-fly construction of fixed dimensionality codevectors to encode structures of various complexity. Similarity of such codevectors takes into account both structural and semantic similarity of represented structures. We provide a comparative description of sparse binary distributed representation developed in the framework of the associative-projective neural network architecture and the more well known holographic reduced representations of T.A. Plate (1995) and binary spatter codes of P. Kanerva (1996). The key procedure in associative-projective neural networks is context-dependent thinning which binds codevectors and maintains their sparseness. The codevectors are stored in structured memory array which can be realized as distributed auto-associative memory. Examples of distributed representation of structured data are given. Fast estimation of the similarity of analogical episodes by the overlap of their codevectors is used in the modeling of analogical reasoning both for retrieval of analogs from memory and for analogical mapping.

Modelling multiple-cause structure using rectification constraints

We present an artificial neural network which self-organizes in an unsupervised manner to form a sparse distributed representation of the underlying causes in data sets. This coding is achieved by introducing several rectification constraints to a PCA network, based on our prior beliefs about the data. Through experimentation we investigate the relative performance of these rectifications on the weights and/or outputs of the network. We find that use of an exponential function on the output to the network is most reliable in discovering all the causes in data sets even when the input data are strongly corrupted by random noise. Preprocessing our inputs to achieve unit variance on each is very effective in helping us to discover all underlying causes when the power on each cause is variable. Our resulting network methodologies are straightforward yet extremely robust over many trials.

Modelling multiple-cause structure using rectification constraints

We present an artificial neural network which self-organizes in an unsupervised manner to form a sparse distributed representation of the underlying causes in data sets. This coding is achieved by introducing several rectification constraints to a PCA network, based on our prior beliefs about the data. Through experimentation we investigate the relative performance of these rectifications on the weights and/or outputs of the network. We find that use of an exponential function on the output to the network is most reliable in discovering all the causes in data sets even when the input data are strongly corrupted by random noise. Preprocessing our inputs to achieve unit variance on each is very effective in helping us to discover all underlying causes when the power on each cause is variable. Our resulting network methodologies are straightforward yet extremely robust over many trials.