Hierarchical Temporal Memory Algorithm Research Articles

An ontological approach to the detection of anomalies in vehicular ad hoc networks

In a vehicular environment, by becoming connected, vehicles are subject to more threats in comparison to traditional information systems, with the difference that, as a cyber-physical system, anomalies and intrusions could have repercussions in the physical world. In this work, we have developed an ontological anomaly-detection approach (OADA). The anomalies studied in this work mainly concern: network scans, DNS tunnel attacks, and telemetry data anomalies. Our contribution relates to a study of the attributes of interest for the algorithm used during the detection phase, namely the hierarchical temporal memory algorithm (HTM). The packets exchanged by the vehicle are grouped in instant description windows. These windows are then analyzed to extract a set of attributes. These are linked to the properties of network traffic, such as flow or latency. They are subject to the process of detecting anomalies and intrusions, carried out thanks to the algorithm with HTM. For each entry, the algorithm produces a score that allows us to decide if a window is abnormal and to lift an alert if that is the case. We evaluated our system using a communications and anomalies emulation tool. We use the corpus of data produced thanks to Autobot. We seek to determine from among the best scores of Matthews correlation coefficient (MCC) and Detection efficiency score (DES) which were the parameters for which HTM detects all anomalies with the greatest possible coverage. The obtained results prove that HTM can detect all anomalies for each window duration.

Sequence learning, prediction, and replay in networks of spiking neurons

Sequence learning, prediction and replay have been proposed to constitute the universal computations performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes these forms of computation. It learns sequences in an unsupervised and continuous manner using local learning rules, permits a context specific prediction of future sequence elements, and generates mismatch signals in case the predictions are not met. While the HTM algorithm accounts for a number of biological features such as topographic receptive fields, nonlinear dendritic processing, and sparse connectivity, it is based on abstract discrete-time neuron and synapse dynamics, as well as on plasticity mechanisms that can only partly be related to known biological mechanisms. Here, we devise a continuous-time implementation of the temporal-memory (TM) component of the HTM algorithm, which is based on a recurrent network of spiking neurons with biophysically interpretable variables and parameters. The model learns high-order sequences by means of a structural Hebbian synaptic plasticity mechanism supplemented with a rate-based homeostatic control. In combination with nonlinear dendritic input integration and local inhibitory feedback, this type of plasticity leads to the dynamic self-organization of narrow sequence-specific subnetworks. These subnetworks provide the substrate for a faithful propagation of sparse, synchronous activity, and, thereby, for a robust, context specific prediction of future sequence elements as well as for the autonomous replay of previously learned sequences. By strengthening the link to biology, our implementation facilitates the evaluation of the TM hypothesis based on experimentally accessible quantities. The continuous-time implementation of the TM algorithm permits, in particular, an investigation of the role of sequence timing for sequence learning, prediction and replay. We demonstrate this aspect by studying the effect of the sequence speed on the sequence learning performance and on the speed of autonomous sequence replay.

A New Hierarchical Temporal Memory Algorithm Based on Activation Intensity.

As a human-cortex-inspired computing model, hierarchical temporal memory (HTM) has shown great promise in sequence learning and has been applied to various time-series applications. HTM uses the combination of columns and neurons to learn the temporal patterns within the sequence. However, the conventional HTM model compacts the input into two naive column states—active and nonactive, and uses a fixed learning strategy. This simplicity limits the representation capability of HTM and ignores the impacts of active columns on learning the temporal context. To address these issues, we propose a new HTM algorithm based on activation intensity. By introducing the column activation intensity, more useful and fine-grained information from the input is retained for sequence learning. Furthermore, a self-adaptive nonlinear learning strategy is proposed where the synaptic connections are dynamically adjusted according to the activation intensity of columns. Extensive experiments are carried out on two real-world time-series datasets. Compared to the conventional HTM and LSTM model, our method achieved higher accuracy and less time overhead.

UNSUPERVISED REAL-TIME DIAGNOSIS SYSTEM FOR ECG STREAMING DAT

Detecting anomalies in time series data plays a vital role in the various applications of diagnosis systems. The importance of anomaly detection is increased by its ability to detect abnormalities in Electrocardiogram (ECG) signals to generate alerts for cardiac health problems. An ECG is a time series that provides essential information about the electrical activity of the heart and is used in the diagnosis of numerous heart diseases. An accurate ECG streaming analytics approach requires continuous learning and adaptation in changing data behaviors. We aim to diagnose ECG by investigating healthy ECG and ECG with cardiological disorders by detecting anomalies in ECG signals. The main objective of this paper is to develop an efficient unsupervised diagnosing system for ECG streaming data based on an online sequence memory algorithm called Hierarchical Temporal Memory (HTM). The HTM is based on neural network and machine learning algorithm for continuous learning tasks. The proposed customization of the HTM algorithm based on our problem domain provides a significant performance results of detection of anomalies in the ECG signals.

Hierarchical Temporal Memory Continuous Learning Algorithms for Fire State Determination

An ultimate goal of placing fire detection systems in buildings and structures is to allow for the rapid detection of fire and accurate faster than real time prediction of ensuing fire behavior so that relevant information can be delivered to the appropriate stakeholders. In the near-term, development of detection systems with decreased detection time, better discrimination against nuisance and false alarms, and real-time monitoring of the fire state is a critical interim step. Building comfort and efficiency systems are increasingly incorporating a greater quantity of sensors and these sensors are installed at a greater density than any fire sensor with the exception of the sprinkler. While currently used primarily for building management purposes, the application of these, or similar types of building sensors, for rapid fire detection, fire state determination, and fire forecasting offers great potential. This paper discusses the potential benefits of the application of Hierarchical Temporal Memory algorithms for fire state determination in a continuous learning environment based on its application to a series of live fire experiments.

Neuromorphic Architecture for the Hierarchical Temporal Memory

A biomimetic machine intelligence algorithm, that holds promise in creating invariant representations of spatiotemporal input streams is the hierarchical temporal memory (HTM). This unsupervised online algorithm has been demonstrated on several machine-learning tasks, including anomaly detection. Significant effort has been made in formalizing and applying the HTM algorithm to different classes of problems. There are few early explorations of the HTM hardware architecture, especially for the earlier version of the spatial pooler of HTM algorithm. In this article, we present a full-scale HTM architecture for both spatial pooler and temporal memory. Synthetic synapse design is proposed to address the potential and dynamic interconnections occurring during learning. The architecture is interweaved with parallel cells and columns that enable high processing speed for the HTM. The proposed architecture is verified for two different datasets: MNIST and the European number plate font (EUNF), with and without the presence of noise. The spatial pooler architecture is synthesized on Xilinx ZYNQ-7, with 91.16% classification accuracy for MNIST and 90\% accuracy for EUNF, with noise. For the temporal memory sequence prediction, first and second order predictions are observed for a 5-number long sequence generated from EUNF dataset and 95% accuracy is obtained. Moreover, the proposed hardware architecture offers 1364X speedup over the software realization. These results indicate that the proposed architecture can serve as a digital core to build the HTM in hardware and eventually as a standalone self-learning system.

시공간적 계층 메모리 학습 알고리즘을 이용한 근전도 패턴인식

This paper presents a new electromyogram (EMG) pattern recognition method based on the Hierarchical Temporal Memory (HTM) algorithm which is originally devised for image pattern recognition. In the modified HTM algorithm, a simplified two-level structure with spatial pooler, temporal pooler, and supervised mapper is proposed for efficient learning and classification of the EMG signals. To enhance the recognition performance, the category information is utilized not only in the supervised mapper but also in the temporal pooler. The experimental results show that the ten kinds of hand motion are successfully recognized.