Spatio-temporal Events Research Articles

Deep-prior ODEs augment fluorescence imaging with chemical sensors

To study biological signalling, great effort goes into designing sensors whose fluorescence follows the concentration of chemical messengers as closely as possible. However, the binding kinetics of the sensors are often overlooked when interpreting cell signals from the resulting fluorescence measurements. We propose a method to reconstruct the spatiotemporal concentration of the underlying chemical messengers in consideration of the binding process. Our method fits fluorescence data under the constraint of the corresponding chemical reactions and with the help of a deep-neural-network prior. We test it on several GCaMP calcium sensors. The recovered concentrations concur in a common temporal waveform regardless of the sensor kinetics, whereas assuming equilibrium introduces artifacts. We also show that our method can reveal distinct spatiotemporal events in the calcium distribution of single neurons. Our work augments current chemical sensors and highlights the importance of incorporating physical constraints in computational imaging.

A Study on a Spatiotemporal Entity-Based Event Data Model

An event is an important medium for recording, expressing, and understanding the real world. Additionally, a data model can provide a digital and structured description method for the real world. Therefore, studying event data models is highly important for describing the real world. By analyzing the representational categories of the existing event data models, the representation of existing event models was found to have different emphases and not be sufficiently balanced, and the universality and comprehensiveness need to be improved. Therefore, based on the advantages of the ontological event model in expressing semantic information and the advantages of the object-event-based spatiotemporal data model in expressing entity multidimensional characteristics and dynamic processes, a spatiotemporal entity-based event data model and the modeling method were designed to provide model support for event organization and processing. Additionally, the Long March and its important battles were selected as case studies to validate the proposed model. The validation shows that the proposed model performs well in terms of event dynamics, hierarchical structure, and complex interrelationships.

Urban Spatiotemporal Event Prediction Using Convolutional Neural Network and Road Feature Fusion Network

The security challenges faced by smart cities are attracting more attention from more people. Criminal activities and disasters can have a significant impact on the stability of a city, resulting in a loss of safety and property for its residents. Therefore, predicting the occurrence of urban events in advance is of utmost importance. However, current methods fail to consider the impact of road information on the distribution of cases and the fusion of information at different scales. In order to solve the above problems, an urban spatiotemporal event prediction method based on a convolutional neural network (CNN) and road feature fusion network (FFN) named CNN-rFFN is proposed in this paper. The method is divided into two stages: The first stage constructs feature map and structure of CNN then selects the optimal feature map and number of CNN layers. The second stage extracts urban road network information using multiscale convolution and incorporates the extracted road network feature information into the CNN. Some comparison experiments are conducted on the 2018–2019 urban patrol events dataset in Zhengzhou City, China. The CNN-rFFN method has an R2 value of 0.9430, which is higher than the CNN, CNN-LSTM, Dilated-CNN, ResNet, and ST-ResNet algorithms. The experimental results demonstrate that the CNN-rFFN method has better performance than other methods.

A Single-Pixel Event Photoactive Device for Real-Time, In-Sensor Spatiotemporal Optical Information Processing.

The increasing demand for energy-efficient, sophisticated optical sensing technologies in various applications, from machine vision to optical communication, highlights the necessity for innovations in spatiotemporal information sensing and processing at a nearly single-pixel scale. Traditional methods, including multi-pixel photodetector arrays and event-based camera systems, often fail to provide rapid, real-time detection and processing of dynamic events within the sensor. This shortcoming is particularly notable in handling high-dimensional spatiotemporal data, where the dependency on sequential data input and external processing tools leads to latency, reduced throughput, and heightened energy consumption, thereby impeding real-time parallel data processing capabilities. Here, a carrier-selective, single-pixel, position-sensitive planar photoactive device that integrates spatiotemporal event sensing with inherent short-term memory capabilities is introduced. The proof-of-concept single-pixel event photoactive device enables in-sensor spatiotemporal parallel optical information processing, efficiently managing multibit (>4 bit) data simultaneously and facilitating ultrafast (≈0.4µs) recognition of input patterns with low energy consumption (25 femtojoules). Additionally, by adjusting the operating speed from continuous to pulsed light illumination, the sensor array can detect trajectories and absolute position of events, offering in-sensor optical flow detection. This single-pixel event photodetector marks significant advancement toward developing compact, energy-efficient, ultrafast sensors suitable for a wide range of in sensor-based photonic applications.

An increase in the spatial extent of European floods over the last 70 years

Abstract. Floods regularly cause substantial damage worldwide. Changing flood characteristics, e.g., due to climate change, pose challenges to flood risk management. The spatial extent of floods is an important indicator of potential impacts, as consequences of widespread floods are particularly difficult to mitigate. The highly uneven station distribution in space and time, however, limits the ability to quantify flood characteristics and, in particular, changes in flood extents over large regions. Here, we use observation-driven routed runoff simulations over the last 70 years in Europe from a state-of-the-art hydrological model (the mesoscale Hydrologic Model – mHM) to identify large spatiotemporally connected flood events. Our identified spatiotemporal flood events compare well against an independent flood impact database. We find that flood extents increase by 11.3 % on average across Europe. This increase occurs over most of Europe, except for parts of eastern and southwestern Europe. Over northern Europe, the increase in flood extent is mainly driven by the overall increase in flood magnitude caused by increasing precipitation and snowmelt. In contrast, the increasing trend in flood extent over central Europe can be attributed to an increase in the spatial extent of heavy precipitation. Overall, our study illustrates the opportunities to combine long-term consistent regional runoff simulations with a spatiotemporal flood detection algorithm to identify large-scale trends in key flood characteristics and their drivers. The detected change in flood extent should be considered in risk assessments as it may challenge flood control and water resource management.

Stereotyped spatiotemporal dynamics of spontaneous activity in visual cortex prior to eye-opening.

Over the course of development, functional sensory representations emerge in the visual cortex. Prior to eye-opening, modular patterns of spontaneous activity form long-range networks that may serve as a precursor for mature network organization. Although the spatial structure of these networks has been well studied, their temporal features, which may contribute to their continued plasticity and development, remain largely uncharacterized. To address this, we imaged hours of spontaneous network activity in the visual cortex of developing ferrets of both sexes utilizing a fast calcium indicator (GCaMP8m) and widefield imaging at high temporal resolution (50Hz), then segmented out spatiotemporal events. The spatial structure of this activity was highly modular, exhibiting spatially segregated active domains consistent with prior work. We found that the vast majority of events showed a clear dynamic component in which modules activated sequentially across the field of view, but only a minority of events were well-fit with a linear traveling wave. We found that spatiotemporal events occur in repeated and stereotyped motifs, reoccurring across hours of imaging. Finally, we found that the most frequently occurring single-frame spatial activity patterns were predictive of future activity patterns over hundreds of milliseconds. Together, our results demonstrate that spontaneous activity in the early developing cortex exhibits a rich spatiotemporal structure, suggesting a potential role in the maturation and refinement of future functional representations.

Causal Event Graph-Guided Language-based Spatiotemporal Question Answering

Large Language Models have excelled at encoding and leveraging language patterns in large text-based corpora for various tasks, including spatiotemporal event-based question answering (QA). However, due to encoding a text-based projection of the world, they have also been shown to lack a full bodied understanding of such events, e.g., a sense of intuitive physics, and cause-and-effect relationships among events. In this work, we propose using causal event graphs (CEGs) to enhance language understanding of spatiotemporal events in language models, using a novel approach that also provides proofs for the model’s capture of the CEGs. A CEG consists of events denoted by nodes, and edges that denote cause and effect relationships among the events. We perform experimentation and evaluation of our approach for benchmark spatiotemporal QA tasks and show effective performance, both quantitative and qualitative, over state-of-the-art baseline methods.

Expert exploranation for communicating scientific methods — A case study in conflict research

Science communication aims at making key research insights accessible to the broad public. If explanatory and exploratory visualization techniques are combined to do so, the approach is also referred to as exploranation. In this context, the audience is usually not required to have domain expertise. However, we show that exploranation can not only support the communication between researchers and a broad audience, but also between researchers directly.With the goal of communicating an existing method for conducting causal inference on spatio-temporal conflict event data, we investigated how to perform exploranation for experts, i.e., expert exploranation. Based on application scenarios of the inference method, we developed three versions of an interactive visual story to explain the method to conflict researchers. We abstracted the corresponding design process and evaluated the stories both with experts who were unfamiliar with the explained method and experts who were already familiar with it.The positive and extensive feedback from the evaluation shows that expert exploranation is a promising direction for visual storytelling, as it can help to improve scientific outreach, methodological understanding, and accessibility for researchers new to a field.

A Sensor-Based Simulation Method for Spatiotemporal Event Detection

Human movements in urban areas are essential to understand human–environment interactions. However, activities and associated movements are full of uncertainties due to the complexity of a city. In this paper, we propose a novel sensor-based approach for spatiotemporal event detection based on the Discrete Empirical Interpolation Method. Specifically, we first identify the key locations, defined as “sensors”, which have the strongest correlation with the whole dataset. We then simulate a regular uneventful scenario with the observation data points from those key locations. By comparing the simulated and observation scenarios, events are extracted both spatially and temporally. We apply this method in New York City with taxi trip record data. Results show that this method is effective in detecting when and where events occur.

Automated Discovery of Successful Strategies in Association Football

Using automated data analysis to understand what makes a play successful in football can enable teams to make data-driven decisions that may enhance their performance throughout the season. Analyzing different types of plays (e.g., corner, penalty, free kicks) requires different considerations. This work focuses on the analysis of corner kick plays. However, the central ideas apply to analyzing all types of plays. While prior analyses (univariate, bivariate, multivariate) have explored the link between contextual factors (e.g., match period, type of defensive marking) and the level of success of a corner kick (e.g., shot, shot on goal, goal), there has been no attempt to combine spatiotemporal event data (sequences of ball movements through the field) and contextual information to determine when and how (strategy) a particular type of corner kick play (tactic) is more likely to succeed or not. To address this gap, we propose an approach that (1) transforms spatiotemporal data into an alternative representation suitable for mining sequential patterns, (2) identifies and characterizes the sequential patterns used by offensive teams to move the ball toward the scoring zone (tactics), and (3) extracts contrast patterns to identify under what conditions different tactics result in increased chances of success or failure; we call these conditions strategies. Our results suggest that favorable and unfavorable conditions for tactic application are not the same across different tactics, supporting the argument that there is a benefit in performing an analysis that treats different tactics separately, where spatiotemporal information plays a crucial role. Unlike prior works on the corner kick, our approach can capture how the interaction between multiple contextual factors impacts the outcome of a corner kick. At the same time, the results can be explained to others in natural languages.

Event encryption: rethinking privacy exposure for neuromorphic imaging

Bio-inspired neuromorphic cameras sense illumination changes on a per-pixel basis and generate spatiotemporal streaming events within microseconds in response, offering visual information with high temporal resolution over a high dynamic range. Such devices often serve in surveillance systems due to their applicability and robustness in environments with high dynamics and harsh lighting, where they can still supply clearer recordings than traditional imaging. In other words, when it comes to privacy-relevant cases, neuromorphic cameras also expose more sensitive data and pose serious security threats. Therefore, asynchronous event streams necessitate careful encryption before transmission and usage. This work discusses several potential attack scenarios and approaches event encryption from the perspective of neuromorphic noise removal, in which we inversely introduce well-crafted noise into raw events until they are obfuscated. Our evaluations show that the encrypted events can effectively protect information from attacks of low-level visual reconstruction and high-level neuromorphic reasoning, and thus feature dependable privacy-preserving competence. The proposed solution gives impetus to the security of event data and paves the way to a highly encrypted technique for privacy-protective neuromorphic imaging.

Multivariate Hawkes processes with spatial covariates for spatiotemporal event data analysis

Multivariate Hawkes processes with spatial covariates for spatiotemporal event data analysis

Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule

Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule

Event-Based Optical Flow via Transforming Into Motion-Dependent View.

Event cameras respond to temporal dynamics, helping to resolve ambiguities in spatio-temporal changes for optical flow estimation. However, the unique spatio-temporal event distribution challenges the feature extraction, and the direct construction of motion representation through the orthogonal view is less than ideal due to the entanglement of appearance and motion. This paper proposes to transform the orthogonal view into a motion-dependent one for enhancing event-based motion representation and presents a Motion View-based Network (MV-Net) for practical optical flow estimation. Specifically, this motion-dependent view transformation is achieved through the Event View Transformation Module, which captures the relationship between the steepest temporal changes and motion direction, incorporating these temporal cues into the view transformation process for feature gathering. This module includes two phases: extracting the temporal evolution clues by central difference operation in the extraction phase and capturing the motion pattern by evolution-guided deformable convolution in the perception phase. Besides, the MV-Net constructs an eccentric downsampling process to avoid response weakening from the sparsity of events in the downsampling stage. The whole network is trained end-to-end in a self-supervised manner, and the evaluations conducted on four challenging datasets reveal the superior performance of the proposed model compared to state-of-the-art (SOTA) methods.

Winning With Chaos in Association Football: Spatiotemporal Event Distribution Randomness Metric for Team Performance Evaluation

Winning With Chaos in Association Football: Spatiotemporal Event Distribution Randomness Metric for Team Performance Evaluation

Using semantic story maps to describe a territory beyond its map

The paper presents the Story Map Building and Visualizing Tool (SMBVT) that allows users to create story maps within a collaborative environment and a usable Web interface. It is entirely open-source and published as a free-to-use solution. It uses Semantic Web technologies in the back-end system to represent stories through a reference ontology for representing narratives. It builds up a user-shared semantic knowledge base that automatically interconnects all stories and seamlessly enables collaborative story building. Finally, it operates within an Open-Science oriented e-Infrastructure, which enables data and information sharing within communities of narrators, and adds multi-tenancy, multi-user, security, and access-control facilities. SMBVT represents narratives as a network of spatiotemporal events related by semantic relations and standardizes the event descriptions by assigning internationalized resource identifiers (IRIs) to the event components, i.e., the entities that take part in the event (e.g., persons, objects, places, concepts). The tool automatically saves the collected knowledge as a Web Ontology Language (OWL) graph and openly publishes it as Linked Open Data. This feature allows connecting the story events to other knowledge bases. To evaluate and demonstrate our tool, we used it to describe the Apuan Alps territory in Tuscany (Italy). Based on a user-test evaluation, we assessed the tool’s effectiveness at building story maps and the ability of the produced story to describe the territory beyond the map.

A deep marked graph process model for citywide traffic congestion forecasting

AbstractForecasting citywide traffic congestion on large road networks has long been a nontrivial research problem due to the challenge of modeling complex evolution patterns of congestion in highly stochastic traffic environments. Arguing that purely data‐driven methods may not perform well for congestion forecasting, we propose a deep marked graph process model for predicting the congestion indices and the occurrence time of traffic congestion events for complex signalized road networks. Traffic congestion is considered as a nonrigorous spatiotemporal extreme event. We extend the traditional point process model by integrating a specially designed spatiotemporal graph convolutional network. This hybrid strategy takes advantage of the simple form of the point process model as well as the ability of graph neural networks to emulate the evolution of congestion. Experiments on real‐world congestion data sets show that the proposed method outperforms state‐of‐the‐art baseline methods, yielding satisfactory prediction results on a large signalized road network with superior computational efficiency.

CCRG-04. AUTOMATED REAL TIME TRACKING OF GLIOBLASTOMA CELL CYCLE TRANSITIONS IN ASSEMBLOIDS LINKS G2 CELLS TO INFILTRATION

Abstract Glioblastoma (GB) is the most aggressive primary brain tumour in adults associated with survival rates below 5% despite treatment. Tumour repopulating residual cells in the surgical cavity resist treatment due to their molecular heterogeneity. Single-cell transcriptomic studies have linked GB heterogeneity to the cell cycle; however, it remains unclear to what extent cell cycle states affect the infiltrative capacity of GB. The “go or grow” paradigm predicts that migration and proliferation are separate spatiotemporal events. We used a brain tumour-cerebral organoid (assembloid) approach to test the migratory potential of GB cell cycle phases during assembloid infiltration. Human induced pluripotent cells were differentiated into cerebral organoids that self-assemble with spheroids of patient-derived GB cells, expressing a fluorescent cell cycle reporter (FUCCI). Utilising live-cell confocal imaging, the process of GB assembloid infiltration was captured. Resultant videos were analysed with automated image analysis capable of tracking cell cycle transitions and cellular migration in confocal microscopy planes, enabling the longitudinal quantification of GB migration and invasion. Our analysis revealed a subset of highly infiltrative GB cells, containing a significant number of cycling G2 phase cells that demonstrated a ~1.4-fold increase in directional infiltrative capacity compared to their non cycling G1 phase counterparts. To isolate and characterize the G2 migratory GB cell state, we developed a phenotypic separation assay using microenvironmental cues. Specifically, we found that these cues markedly affect GB migration routes enabling the isolation of the highly migratory GB cells for further investigation. Our results question the “go or grow” hypothesis at the level of GB cellular subpopulations. We hypothesise that the observed G2 migratory capacity of GB cells could be critical to tumour recurrence following surgery. Further research is underway to define the underlying mechanisms of G2 GB cell migration.

The life of the cell membrane: A paradigmatic reading from Deleuze and Guattari

While the Fluid Mosaic model (FMM) is widely accepted as an account of the cell membrane's structure-function, its inability to explain certain phenomena has led to the lipid rafts hypothesis (nanodomains) that spontaneous spatiotemporal enriched zones of sphingolipids-cholesterol-protein exist within the membrane. In this text, we propose a novel approach that conceives the cell membrane as a living entity. The questions regarding the FMM revolve around the fact that, although these molecular components are present in many cell types, the membrane does not react in the same way to every external agent; for example, a virus evokes a particular response: why is there some marked specificity of virus (or toxin) attack on one (or some) of these cell types and not to other cell types that nevertheless have a similar membrane protein constitution? The crucial question, to explain this selectivity, would be what determines the specificity of attack on some cells and not others? While FMN assumes a dynamism between macrostates at the intramolecular, intermolecular, and/or collective levels in the membrane, the approach of the lipid raft model presupposes a much greater and more complex dynamics of microstates (even nano-states) of these molecular components. In other words, it implies higher and instantaneous mobility as assemblages ("intentional") and thus, of the membrane itself (as a collective), in response to changes in the internal and external physicochemical environment over a broad spatiotemporal scale. This suggests a mechanism of membrane adaptation in the face of evolutionary constraints. In this text, we propose a paradigmatic approach, from Deleuze-Guattari's philosophy: to conceive the cell membrane as living and not as a mere molecular conglomerate with particular functions and mechanical processes between molecules. For this, we employ the functional concepts of territory and machinic assemblage, whence the vitality of the membrane would allow us to postulate instantaneous updates, within wider spatiotemporal scales in its composition in contrast with the model that dominates as a more plausible explanation nowadays, that does not include smaller spatiotemporal events. If we resort to the concept of territory and its different media components, we could offer a more plausible explanation of the vigorous dynamism in the composition of the cell membrane since it would allow more subtle and complex differentiations between media and thus make visible the constant and instant changes. We propose that the model of nanodomains, understood as a process of dynamic territorialization, offers a more complex and subtle explanation of the instantaneous changes in the cell membrane's composition. This approach expands the explanatory framework for cellular phenomena and reveals their spatiotemporal complexity in accordance with other research.

ST-GWLR: combining geographically weighted logistic regression and spatiotemporal hotspot trend analysis to explore the effect of built environment on traffic crash

ABSTRACT Road traffic crashes are becoming thorny issues being faced worldwide. Traffic crashes are spatiotemporal events and the research on the spatiotemporal patterns and variation trends of traffic crashes has been carried out. However, the impact of built environment on traffic crash spatiotemporal trends has not received much attention. Moreover, the spatial non-stationarity between the variation trends of traffic crashes and their influencing factors is usually neglected. To make up for the lack of analysis of built environment factors influencing spatiotemporal hotspot trends in traffic crashes, this paper proposed a method of “ST-GWLR” for analyzing the influence of built environment factors on spatiotemporal hotspot trends of traffic crashes by combining the spatiotemporal hotspot trend analysis and Geographically Weighted Logistic Regression (GWLR) modeling methods. Firstly, the traffic crash spatiotemporal hotspot trends were explored using the space-time cube model, hotspot analysis, and Mann-Kendall trend test. Then, the GWLR was introduced to capture the spatial non-stationarity neglected by the classic Global Logistic Regression (GLR) model, to improve the accuracy of the model estimation. GWLR model is used for the first time to analyze the significant local correlation between the traffic crash spatiotemporal hotspot trends and the built environment factors, to accurately and effectively identify the built environment factors that have significant influences on the hotspot trends of traffic crashes. The performance of the GWLR models and GLR models was examined and compared sufficiently. The results showed that the proposed ST-GWLR, which captured spatial non-stationarity, performed better than the classic GLR combined with spatiotemporal analysis, and improved the prediction accuracy of the models by 14.9%, 13.9%, and 15.1%, respectively. There were significant local correlations between intensifying hotspots and persistent hotspots of traffic crashes and the built environment factors. The findings of this paper have positive implications for traffic safety management and urban built environment planning.