Spatiotemporal Memory Research Articles

Dynamics and motion patterns of a k-capture game with attraction-repulsion interaction

In this paper, we propose and formulate a k-capture game, where a single intruder tries to reach a stationary target that is protected by a swarm of defenders, and the defenders try to intercept (capture) the intruder by a synchronous defense with k-capture (at least k defenders have to synchronously reach the intruder's location to capture it). Assume that the defenders have no explicit task collaboration, no spatio-temporal memory and no speed advantage. Three general collective motions with simple attraction-repulsion interaction between defender pairs are introduced to investigate the dynamics and motion patterns of the k-capture game. The results suggest that such system with merely local spacial attraction-repulsion interaction of defenders already shows very rich and even unexpected motion patterns, including the boundary defense motion pattern, the intruder-follower-and-rounded-up motion pattern, the periodic motion pattern, and the multiple small cohesive groups motion pattern. The results also show that the multiple small cohesive groups motion pattern seems to be the most effective defense motion since the power of the collective is better used. The study of k-capture game provides an insight into the defense-intrusion interaction in nature. This approach can also provide elegant insights for artificial systems, such as escort mission for aircraft, defense mission for military base or territorial boundaries by a swarm of autonomous vehicles.

A Hardware Implementation of SNN-Based Spatio-Temporal Memory Model.

Simulating human brain with hardware has been an attractive project for many years, since memory is one of the fundamental functions of our brains. Several memory models have been proposed up to now in order to unveil how the memory is organized in the brain. In this paper, we adopt spatio-temporal memory (STM) model, in which both associative memory and episodic memory are analyzed and emulated, as the reference of our hardware network architecture. Furthermore, some reasonable adaptations are carried out for the hardware implementation. We finally implement this memory model on FPGA, and additional experiments are performed to fine tune the parameters of our network deployed on FPGA.

EEG source localization using spatio-temporal neural network

Source localization of focal electrical activity from scalp electroencephalogram (sEEG) signal is generally modeled as an inverse problem that is highly ill-posed. In this paper, a novel source localization method is proposed to model the EEG inverse problem using spatio-temporal long-short term memory recurrent neural networks (LSTM). The network model consists of two parts, sEEG encoding and source decoding, to model the sEEG signal and receive the regression of source location. As there does not exist enough annotated sEEG signals correspond to specific source locations, simulated data is generated with forward model using finite element method (FEM) to act as a part of training signals. A framework for source localization is proposed to estimate the source position based on simulated training data. Experiments are done on simulated testing data. The results on simulated data exhibit good robustness on noise signal, and the proposed network solves the EEG inverse problem with spatio-temporal deep network. The result show that the proposed method overcomes the highly ill-posed linear inverse problem with data driven learning.

Predicting Fine-Grained Traffic Conditions via Spatio-Temporal LSTM

Predicting traffic conditions for road segments is the prelude of working on intelligent transportation. Many existing methods can be used for short-term or long-term traffic prediction, but they focus more on regions than on road segments. The lack of fine-grained traffic predicting approach hinders the development of ITS. Therefore, MapLSTM, a spatio-temporal long short-term memory network preluded by map-matching, is proposed in this paper to predict fine-grained traffic conditions. MapLSTM first obtains the historical and real-time traffic conditions of road segments via map-matching. Then LSTM is used to predict the conditions of the corresponding road segments in the future. Breaking the single-index forecasting, MapLSTM can predict the vehicle speed, traffic volume, and the travel time in different directions of road segments simultaneously. Experiments confirmed MapLSTM can not only achieve prediction for road segments based a large scale of GPS trajectories effectively but also have higher predicting accuracy than GPR and ConvLSTM. Moreover, we demonstrate that MapLSTM can serve various applications in a lightweight way, such as cognizing driving preferences, learning navigation, and inferring traffic emissions.

A Hippocampal Model for Behavioral Time Acquisition and Fast Bidirectional Replay of Spatio-Temporal Memory Sequences.

The hippocampus is known to play a crucial role in the formation of long-term memory. For this, fast replays of previously experienced activities during sleep or after reward experiences are believed to be crucial. But how such replays are generated is still completely unclear. In this paper we propose a possible mechanism for this: we present a model that can store experienced trajectories on a behavioral timescale after a single run, and can subsequently bidirectionally replay such trajectories, thereby omitting any specifics of the previous behavior like speed, etc, but allowing repetitions of events, even with different subsequent events. Our solution builds on well-known concepts, one-shot learning and synfire chains, enhancing them by additional mechanisms using global inhibition and disinhibition. For replays our approach relies on dendritic spikes and cholinergic modulation, as supported by experimental data. We also hypothesize a functional role of disinhibition as a pacemaker during behavioral time.

An Embodied Tutoring System for Literal vs. Metaphorical Concepts.

In this paper we combine motion captured data with linguistic notions (preliminary study) in a game-like tutoring system (study 1), in order to help elementary school students to better differentiate literal from metaphorical uses of motion verbs, based on embodied information. In addition to the thematic goal, we intend to improve young students’ attention and spatiotemporal memory, by presenting sensorimotor data experimentally collected from thirty two participants in our motion capturing labs. Furthermore, we examine the accomplishment of tutor’s goals and compare them to curriculum’s approach (study 2). Sixty nine elementary school students were randomly divided in two experimental groups (game-like and traditional) and one control group, which did not undergo an intervention. All groups were tested in pre and post-tests. Even though the diagnostic pretests present a uniform picture, two way analysis of variance suggests that the experimental groups showed progress in post-tests and, more specifically, game-like group showed less wrong answers in the linguistics task and higher learning achievements compared to the other two groups. Furthermore, in the game-like condition the participants needed gradually shorter period of time to identify the avatar’s actions. This finding was considered as a first indication of attentional and spatiotemporal memory’s improvement, while the tutor’s assistance features cultivated students’ metacognitive perception.

Temporal Self-Organization: A Reaction–Diffusion Framework for Spatiotemporal Memories

Self-organizing maps (SOMs) find numerous applications in learning, clustering, and recalling spatial input patterns. The traditional approach in learning spatiotemporal patterns is to incorporate time on the output space of a SOM along with heuristic update rules that work well in practice. Inspired by the pioneering work of Alan Turing, who used reaction-diffusion equations to explain spatial pattern formation, we develop an analogous theoretical model for a spatiotemporal memory to learn and recall temporal patterns. The contribution of the paper is threefold: 1) using coupled reaction-diffusion equations, we develop a theory from first principles for constructing a spatiotemporal SOM and derive an update rule for learning based on the gradient of a potential function; 2) we analyze the dynamics of our algorithm and derive conditions for optimally setting the model parameters; and 3) we mathematically quantify the temporal plasticity effect observed during recall in response to the input dynamics. The simulation results show that the proposed algorithm outperforms the SOM with temporal activity diffusion, neural gas with temporal activity diffusion and spatiotemporal map formation based on a potential function in the presence of correlated noise for the same data set and similar training conditions.

High experience levels delay recruitment but promote simultaneous time-memories in honey bee foragers.

Honey bee (Apis mellifera) foragers can remember both the location and time of day food is collected and, even in the absence of a reward, reconnoiter the food source at the appropriate time on subsequent days. This spatiotemporal memory (time-memory) is linked to the circadian clock and enables foragers to synchronize their behavior with floral nectar secretion rhythms, thus eliminating the need to rediscover productive food sources each day. Here, we asked whether the establishment of one time-memory influences the formation of another time-memory at the same time of day. In other words, can two time-place memories with the same 'time-stamp' co-exist? We simultaneously trained two groups of foragers from a single hive to two separate feeders at the same restricted time of day. After 5days of training, one feeder was shut off. The second feeder continued being productive 4 more days. Our results showed that (1) foragers with high experience levels at the first source were significantly more likely than low-experience foragers to maintain fidelity to their original source and resist recruitment to the alternative source, (2) nearly one-third of foragers demonstrated multiple, overlapping time-memories by visiting both feeders at the correct time and (3) significantly more high-experience than low-experience foragers exhibited this multitasking behavior. The ability to maintain and act upon two different, yet contemporaneous, time-memories gives the forager bee a previously unknown level of versatility in attending to multiple food sources. These findings have major implications for understanding the formation and management of circadian spatiotemporal memories.

An Improved Fractional-Order Optical Flow Model for Motion Estimation

The Horn and Schunck (HS) optical flow model cannot preserve discontinuity of motion estimation and has low accuracy especially for the image sequence, which includes complex texture. To address this problem, an improved fractional-order optical flow model is proposed. In particular, the fractional-order Taylor series expansion is applied in the brightness constraint equation of the HS model. The fractional-order flow field derivative is also used in the smoothing constraint equation. The Euler-Lagrange equation is utilized for the minimization of the energy function of the fractional-order optical flow model. Two-dimensional fractional differential masks are proposed and applied to the calculation of the model simplification. Considering the spatiotemporal memory property of fractional-order, the algorithm preserves the edge discontinuity of the optical flow field while improving the accuracy of the estimation of the dense optical flow field. Experiments on Middlebury datasets demonstrate the predominance of our proposed algorithm.

Network-based brain stimulation selectively impairs spatial retrieval

BackgroundDirect brain stimulation via electrodes implanted for intracranial electroencephalography (iEEG) permits the modulation of endogenous electrical signals with significantly greater spatial and temporal specificity than non-invasive approaches. It also allows for the stimulation of deep brain structures important to memory, such as the hippocampus, that are difficult, if not impossible, to target non-invasively. Direct stimulation studies of these deep memory structures, though, have produced mixed results, with some reporting improvement, some impairment, and others, no consistent changes. Objective/hypothesisWe hypothesize that to modulate cognitive function using brain stimulation, it is essential to modulate connected nodes comprising a network, rather than just alter local activity. MethodsiEEG data collected while patients performed a spatiotemporal memory retrieval task were used to map frequency-specific, coherent oscillatory activity between different brain regions associated with successful memory retrieval. We used these to identify two target nodes that exhibited selectively stronger coupling for spatial vs. temporal retrieval. In a subsequent session, electrical stimulation - theta-bursts with a fixed phase-lag (0° or 180°) – was applied to the two target regions while patients performed spatiotemporal retrieval. ResultsStimulation selectively impaired spatial retrieval while not affecting temporal retrieval, and this selective impairment was associated with theta decoupling of the spatial retrieval network. ConclusionThese findings suggest that stimulating tightly connected nodes in a functional network at the appropriate phase-lag may effectively modulate the network function, and while in this case it impaired memory processes, it sets a foundation for further network-based perturbation studies.

A fully subordinated linear flow model for hillslope subsurface stormflow

AbstractHillslope subsurface stormflow exhibits complex patterns when natural soils with multiscale heterogeneity impart a spatiotemporally nonlocal memory on flow dynamics. To efficiently quantify such nonlocal flow responses, this technical note proposes a fully subordinated flow (FSF) equation where the time‐ and flow‐subordination capture the temporal and spatial memory, respectively. Results show that the time‐subordination component of the FSF model captures a wide range of delayed flow response due to various degrees of soil heterogeneity (especially for low‐conductivity zones), while the model's flow‐subordination term accounts for the rapid flow responses along preferential flow paths. In the FSF model, parameters defining spatiotemporal memory functions may be related to soil properties, while other parameters such as scalar factors controlling the overall advection and diffusion are difficult to predict and can be estimated from subsurface stormflow hydrographs. These parameters can be constants at the hillslope scale because the spatiotemporal subordination, an upscaling technique, can capture the impact of system heterogeneity on flow dynamics, leading to a linear FSF model that might be applicable for various slopes. Valid scale, limitation and extension of the FSF model, and modification of the model for other complex hydrological dynamics are also discussed.

Neural Mechanism to Simulate a Scale-Invariant Future.

Predicting the timing and order of future events is an essential feature of cognition in higher life forms. We propose a neural mechanism to nondestructively translate the current state of spatiotemporal memory into the future, so as to construct an ordered set of future predictions almost instantaneously. We hypothesize that within each cycle of hippocampal theta oscillations, the memory state is swept through a range of translations to yield an ordered set of future predictions through modulations in synaptic connections. Theoretically, we operationalize critical neurobiological findings from hippocampal physiology in terms of neural network equations representing spatiotemporal memory. Combined with constraints based on physical principles requiring scale invariance and coherence in translation across memory nodes, the proposition results in Weber-Fechner spacing for the representation of both past (memory) and future (prediction) timelines. We show that the phenomenon of phase precession of neurons in the hippocampus and ventral striatum correspond to the cognitive act of future prediction.

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier

Memory is a complex process across different brain regions and a fundamental function for many cognitive behaviors. Emerging experimental results suggest that memories are represented by populations of neurons and organized in a categorical and hierarchical manner. However, it is still not clear how the neural mechanisms are emulated in computational models. In this paper, we present a spatio-temporal memory (STM) model using spiking neurons to explore the memory formulation and organization in the brain. Unlike previous approaches, this model employs temporal population codes as the neural representation of information and spiketiming-based learning methods to formulate the memory structure. It explicitly demonstrates that the complex spatio-temporal patterns are the internal neural representations of memory items. Two types of memory processes are analyzed and emulated: associative memory, i.e., spatio-temporal patterns driven by intra-assembly connections, and episodic memory, i.e., temporally separated spatio-temporal patterns linked by inter-assembly connections. Our model will provide a computational substrate based on lowlevel neural circuits for developing neuromorphic cognitive systems with wide applications.

A Self-Organizing Incremental Spatiotemporal Associative Memory Networks Model for Problems with Hidden State.

Identifying the hidden state is important for solving problems with hidden state. We prove any deterministic partially observable Markov decision processes (POMDP) can be represented by a minimal, looping hidden state transition model and propose a heuristic state transition model constructing algorithm. A new spatiotemporal associative memory network (STAMN) is proposed to realize the minimal, looping hidden state transition model. STAMN utilizes the neuroactivity decay to realize the short-term memory, connection weights between different nodes to represent long-term memory, presynaptic potentials, and synchronized activation mechanism to complete identifying and recalling simultaneously. Finally, we give the empirical illustrations of the STAMN and compare the performance of the STAMN model with that of other methods.

Design and integration of a spatio-temporal memory with emotional influences to categorize and recall the experiences of an autonomous mobile robot

Autonomous robots cohabiting with humans will have to achieve recurring tasks while adapting to the changing conditions of the world. A spatio-temporal memory categorizes the experiences of a robot to improve its ability to adapt to its environment. In this paper, we present a spatio-temporal (ST) memory model consisting of a cascade of two adaptive resonance theory (ART) networks: one to categorize spatial events and the other to extract temporal episodes from the robot's experiences. Artificial emotions are used to dynamically modulate learning and recall of the ART networks based on how the robot is able to carry its task, using a simple model of artificial emotions. Once an episode is recalled, future events can be predicted and used to influence the intentions of the robot. Evaluation of our ST model is done using an autonomous robotic platform that has to deliver objects to people within an office area. Results demonstrate that our model can memorize and recall the experiences of a robot, and that emotional episodes are recalled more often, allowing the robot to use its memory of past experiences early on when repeating a task.

Spatiotemporal memory is an intrinsic property of networks of dissociated cortical neurons.

The ability to process complex spatiotemporal information is a fundamental process underlying the behavior of all higher organisms. However, how the brain processes information in the temporal domain remains incompletely understood. We have explored the spatiotemporal information-processing capability of networks formed from dissociated rat E18 cortical neurons growing in culture. By combining optogenetics with microelectrode array recording, we show that these randomly organized cortical microcircuits are able to process complex spatiotemporal information, allowing the identification of a large number of temporal sequences and classification of musical styles. These experiments uncovered spatiotemporal memory processes lasting several seconds. Neural network simulations indicated that both short-term synaptic plasticity and recurrent connections are required for the emergence of this capability. Interestingly, NMDA receptor function is not a requisite for these short-term spatiotemporal memory processes. Indeed, blocking the NMDA receptor with the antagonist APV significantly improved the temporal processing ability of the networks, by reducing spontaneously occurring network bursts. These highly synchronized events have disastrous effects on spatiotemporal information processing, by transiently erasing short-term memory. These results show that the ability to process and integrate complex spatiotemporal information is an intrinsic property of generic cortical networks that does not require specifically designed circuits.

Place cells, grid cells, and memory.

The hippocampal system is critical for storage and retrieval of declarative memories, including memories for locations and events that take place at those locations. Spatial memories place high demands on capacity. Memories must be distinct to be recalled without interference and encoding must be fast. Recent studies have indicated that hippocampal networks allow for fast storage of large quantities of uncorrelated spatial information. The aim of the this article is to review and discuss some of this work, taking as a starting point the discovery of multiple functionally specialized cell types of the hippocampal-entorhinal circuit, such as place, grid, and border cells. We will show that grid cells provide the hippocampus with a metric, as well as a putative mechanism for decorrelation of representations, that the formation of environment-specific place maps depends on mechanisms for long-term plasticity in the hippocampus, and that long-term spatiotemporal memory storage may depend on offline consolidation processes related to sharp-wave ripple activity in the hippocampus. The multitude of representations generated through interactions between a variety of functionally specialized cell types in the entorhinal-hippocampal circuit may be at the heart of the mechanism for declarative memory formation.

Sphingosin-1-phosphate-receptor modulation reduces hyperoxia mediated brain injury

Background: Clinical and experimental studies revealed that oxygen which is widely used in neonatal medicine for resuscitation and treatment of pulmonary hypertension triggers widespread neural apoptosis leading to gray and white matter damage. Within the last years it became evident that modulation of the sphingosine-1phosphate (S1P)-receptor system by the S1P analogue FTY720, shows immunomodulatory and neuroprotective capacities. Since we recently show a detrimental role of hyperoxia in white matter structures of the developing rodent brain, we investigated whether FTY720 might modulate cell death of primary oligodendrocytes and shows therapeutic potential in the developing white matter in vivo. Material and methods: Primary oligodendrocyte precursor cells (pOLN) were generated from mixed glia cultures, obtained from 2 days old Spraque–Dawley rat brains. pOLNs were cultivated for 3– 4 days and were treated with biological active FTY720. Cell death was assessed by flow cytometry (Annexin-V, 7AAD). In order to evaluate the therapeutic potential of FTY720 in the context of hyperoxia-mediated brain injury in vivo we administered 1 mg/kg FTY720 i.p. to six days old (P6) Wistar rats prior to hyperoxia (24 h, 80% oxygen) and analysed oligodendrocyte degeneration and differentiation via immunohistochemistry and western blot at P7 and P11. To detect potential long-term changes regarding motorcognitive development we performed animal behaviour studies focusing on general motor activity and spatio-temporal learning and memory function in adolescent and young adult rats. Summary and conclusion: In the context of hyperoxia-mediated brain injury we provide evidence for an anti-apoptotic capacity of FTY720. Furthermore, we demonstrate a recovery in the differentiation potential of developing oligodendrocytes and an improvement in the long-term neuro-developmental outcome of rats suffering from oxygen toxicity. Our data indicate that FTY20 may have beneficial effects in preterm infants which are exposed to non-physiological elevated oxygen concentrations.

Multiple interacting brain areas underlie successful spatiotemporal memory retrieval in humans.

Emerging evidence suggests that our memories for recent events depend on a dynamic interplay between multiple cortical brain regions, although previous research has also emphasized a primary role for the hippocampus in episodic memory. One challenge in determining the relative importance of interactions between multiple brain regions versus a specific brain region is a lack of analytic approaches to address this issue. Participants underwent neuroimaging while retrieving the spatial and temporal details of a recently experienced virtual reality environment; we then employed graph theory to analyze functional connectivity patterns across multiple lobes. Dense, large-scale increases in connectivity during successful memory retrieval typified network topology, with individual participant performance correlating positively with overall network density. Within this dense network, the hippocampus, prefrontal cortex, precuneus, and visual cortex served as “hubs” of high connectivity. Spatial and temporal retrieval were characterized by distinct but overlapping “subnetworks” with higher connectivity within posterior and anterior brain areas, respectively. Together, these findings provide new insight into the neural basis of episodic memory, suggesting that the interactions of multiple hubs characterize successful memory retrieval. Furthermore, distinct subnetworks represent components of spatial versus temporal retrieval, with the hippocampus acting as a hub integrating information between these two subnetworks.

Taï chimpanzees anticipate revisiting high-valued fruit trees from further distances.

The use of spatio-temporal memory has been argued to increase food-finding efficiency in rainforest primates. However, the exact content of this memory is poorly known to date. This study investigated what specific information from previous feeding visits chimpanzees (Pan troglodytes verus), in Taï National Park, Côte d'Ivoire, take into account when they revisit the same feeding trees. By following five adult females for many consecutive days, we tested from what distance the females directed their travels towards previously visited feeding trees and how previous feeding experiences and fruit tree properties influenced this distance. To exclude the influence of sensory cues, the females' approach distance was measured from their last significant change in travel direction until the moment they entered the tree's maximum detection field. We found that chimpanzees travelled longer distances to trees at which they had previously made food grunts and had rejected fewer fruits compared to other trees. In addition, the results suggest that the chimpanzees were able to anticipate the amount of fruit that they would find in the trees. Overall, our findings are consistent with the hypothesis that chimpanzees act upon a retrieved memory of their last feeding experiences long before they revisit feeding trees, which would indicate a daily use of long-term prospective memory. Further, the results are consistent with the possibility that positive emotional experiences help to trigger prospective memory retrieval in forest areas that are further away and have fewer cues associated with revisited feeding trees.