Spatial-temporal Domain Research Articles

Spatial–temporal prediction model for unsteady near-wall flow around cylinder based on hybrid neural network

A hybrid neural network based on Densely Connected Convolutional Networks (DenseNet), Convolutional Long Short-Term Memory Neural Network (ConvLSTM), and Deconvolutional Neural Network (DeCNN) is employed to predict unsteady flow fields. The utilization of DenseNet makes the model more compact and makes the prediction of three-dimensional flow affordable. The ConvLSTM is implemented to predict multiple future time steps which improves prediction efficiency. The proposed model transforms the time sequences of velocity and pressure fields into uniform spatial–temporal topology as input and captures nonlinear feature information in the spatial–temporal domain. Numerical simulations are conducted for the flow around cylinder at different Reynolds numbers and the near-wall flow around cylinder with different gap ratios, and training samples for the neural network inputs are established. The predicted results are compared with the numerical simulation results, showing good agreement. From the prediction cycle, it can be seen that good prediction results can be maintained in the first three prediction cycles. The prediction results of the three-dimensional unsteady flow around a cylinder near a plane wall, exhibit remarkable accuracy, successfully capturing the evolution of turbulent vortex structures. This signifies that the prediction model is highly effective in capturing the spatial–temporal variations of complex unsteady flows.

Signatures of individuation across objects and events.

The physical world provides humans with continuous streams of experience in both space and time. The human mind, however, can parse and organize this continuous input into discrete, individual units. In the current work, we characterize the representational signatures of basic units of human experience across the spatial (object) and temporal (event) domains. We propose that there are three shared, abstract signatures of individuation underlying the basic units of representation across the two domains. Specifically, individuated entities in both the spatial domain (objects) and temporal domain (bounded events) resist restructuring, have distinct parts, and do not tolerate breaks; unindividuated entities in both the spatial domain (substances) and the temporal domain (unbounded events) lack these features. In three experiments, we confirm these principles and discuss their significance for cognitive and linguistic theories of objects and events. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Spreading Sea Clutter Suppression for High-Frequency Hybrid Sky-Surface Wave Radar Using Orthogonal Projection in Spatial–Temporal Domain

In recent years, the high-frequency hybrid sky-surface wave radar (HSSWR) has been increasingly used for target detection applications. Nevertheless, the specific bistatic system layout and the phase path disturbances induced by the ionospheric propagation channel may severely spread the sea clutter spectrum, thereby deteriorating the detection ability of the HSSWR for slow-moving targets. In this work, a novel subspace method based on the hybrid use of the amplitude and phase estimator (APES) and the orthogonal projection (OP) in the spatial–temporal domain, denoted as the APES-OP method, is proposed to suppress the spreading first-order sea clutter of the HSSWR. The distribution characteristics of targets and first-order sea clutter in the spatial–temporal domain were investigated, and a time-domain subspace signal model was adopted to describe targets perturbed by ionospheric phase path modulation. An APES filter was adopted to filter out the potential targets with a traversal approach to avoid attenuating desired signals while suppressing sea clutter. After that, sampling data from multi-channels and slow-time domains at the cell under test were employed to construct a spatial–temporal matrix, which was then utilized to obtain the sea clutter subspace by singular value decomposition. Simulation results indicate that the proposed algorithm can suppress sea clutter while retaining the target, even if the target is buried by sea clutter. The processing results of measured data further demonstrate the efficiency of the proposed algorithm. After sea clutter suppression, the target obscured by clutter can be revealed, and the signal-to-clutter ratio of the target is greatly improved.

Quantitative Assessment via Multi-Domain Fusion of Muscle Synergy Associated With Upper-Limb Motor Function for Stroke Rehabilitation.

Quantitative assessment of upper limb motor function aids therapists in providing appropriate rehabilitation strategies, which plays an essential role in post-stroke rehabilitation. Traditional assessments, relying on clinical scales or kinematic metrics, often involve subjective scores or are influenced by compensatory strategies. Recently, the use of muscle synergies, representing simplified neuromuscular control, has emerged as a promising approach for post-stroke assessment. In general, muscle synergies are decomposed into two components: synergy vectors and synergy activation. Synergy vectors represent the relative weighting of each muscle within each synergy, that is muscle coordination; synergy activation represents the recruitment of the muscle synergy over time, that is muscle activation strength. Both components are vital for adequately assessing patients' motor function. Therefore, we integrate the spatial domain and temporal domain features extracted from synergy vectors and synergy activation, constructing a multi-domain assessment system using a Random Forest classifier, which may provide great qualitative classification accuracy. Furthermore, a novel functional score is generated from the probabilities belonging to the pathological group. Finally, A study involving ten healthy subjects and ten post-stroke patients validates the proposed method. The experimental results show that the classification accuracy was enhanced to 98.56% by fusing the characteristics derived from different domains, which was higher than that based on spatial domain (94.90%) and temporal domain (91.08%), respectively. Furthermore, the assessment score generated by multi-domain fusion framework exhibited a significant correlation with the clinical score. These promising results show the potential of applying the proposed method to clinical assessments for post-stroke patients.

Multi-Subject 3D Human Mesh Construction Using Commodity WiFi

This paper introduces MultiMesh, a multi-subject 3D human mesh construction system based on commodity WiFi. Our system can reuse commodity WiFi devices in the environment and is capable of working in non-line-of-sight (NLoS) conditions compared with the traditional computer vision-based approach. Specifically, we leverage an L-shaped antenna array to generate the two-dimensional angle of arrival (2D AoA) of reflected signals for subject separation in the physical space. We further leverage the angle of departure and time of flight of the signal to enhance the resolvability for precise separation of close subjects. Then we exploit information from various signal dimensions to mitigate the interference of indirect reflections according to different signal propagation paths. Moreover, we employ the continuity of human movement in the spatial-temporal domain to track weak reflected signals of faraway subjects. Finally, we utilize a deep learning model to digitize 2D AoA images of each subject into the 3D human mesh. We conducted extensive experiments in real-world multi-subject scenarios under various environments to evaluate the performance of our system. For example, we conduct experiments with occlusion and perform human mesh construction for different distances between two subjects and different distances between subjects and WiFi devices. The results show that MultiMesh can accurately construct 3D human meshes for multiple users with an average vertex error of 4cm. The evaluations also demonstrate that our system could achieve comparable performance for unseen environments and people. Moreover, we also evaluate the accuracy of spatial information extraction and the performance of subject detection. These evaluations demonstrate the robustness and effectiveness of our system.

MVSTT: A Multiview Spatial-Temporal Transformer Network for Traffic-Flow Forecasting.

Accurate traffic-flow prediction remains a critical challenge due to complicated spatial dependencies, temporal factors, and unpredictable events. Most existing approaches focus on single-or dual-view learning and thus face limitations in systematically learning complex spatial-temporal features. In this work, we propose a novel multiview spatial-temporal transformer (MVSTT) network that can effectively learn complex spatial-temporal domain correlations and potential patterns from multiple views. First, we examine a temporal view and design a short-range gated convolution component from a short-term subview, and a long-range gated convolution component from a long-term subview. These two components effectively aggregate knowledge of the temporal domain at multiple granularities and mine patterns of node evolution across time steps. Meanwhile, in the spatial view, we design a dual-graph spatial learning module that captures fixed and dynamic spatial dependencies of nodes, as well as the evolution patterns of edges, from the static and dynamic graph subviews, respectively. In addition, we further design a spatial-temporal transformer to mine different levels of spatial-temporal features through multiview knowledge fusion. Extensive experiments on four real-world traffic datasets show that our method consistently outperforms the state-of-the-art baseline. The code of MVSTT is available at https://github.com/JianSoL/MVSTT.

Space–time recurrent memory network

Transformers have recently been popular for learning and inference in the spatial–temporal domain. However, their performance relies on storing and applying attention to the feature tensor of each frame in video. Hence, their space and time complexity increase linearly as the length of video grows, which could be very costly for long videos. We propose a novel visual memory network architecture for the learning and inference problem in the spatial–temporal domain. We maintain a fixed set of memory slots in our memory network and propose an algorithm based on Gumbel-Softmax to learn an adaptive strategy to update this memory. Finally, this architecture is benchmarked on the video object segmentation (VOS) and video prediction problems. We demonstrate that our memory architecture achieves state-of-the-art results, outperforming transformer-based methods on VOS and other recent methods on video prediction while maintaining constant memory capacity independent of the sequence length.

Video-based working condition recognition of fused magnesium furnace with stochastic configuration networks

Working condition recognition of a fused magnesium furnace (FMF) suffers from unbalanced under-burning condition samples, inconsistent quality of training samples and difficulty in characterizing dynamic production processes with images. This paper presents a novel approach to detect the under-burning working condition in FMF based on a 3D-cycle-generative adversarial network (3D-cycle-GAN) and Video Swin Transformer-Stochastic Configuration Networks (Video Swin-SCNs). Firstly, to resolve the temporal discontinuity defect caused by Cycle-GAN through the visual appearance of video composite frames, we construct a motion consistency-based 3D-Cycle-GAN model that considers the visual appearance and temporal continuity constraints of unpaired video transitions and is designed to generate video samples of under-burning working conditions. Secondly, a reinforcement learning approach is used to assess the value of the video quality and to filter out possible low-quality samples generated. Finally, the local attention is extended from the spatial domain to the spatial-temporal domain to solve the difficulty in characterizing the dynamic production process with the static images. The spatial-temporal features from the Video Swin Transformer are fed into SCNs to classify the working conditions. The experiment results indicate the effectiveness and feasibility of the proposed method.

Time Series Scattering Power Decomposition Using Ensemble Average in Temporal–Spatial Domains: Application to Forest Disturbance Detection

Time Series Scattering Power Decomposition Using Ensemble Average in Temporal–Spatial Domains: Application to Forest Disturbance Detection

PHYSICS-INFORMED ENCODER-DECODER GATED RECURRENT NEURAL NETWORK FOR SOLVING TIME-DEPENDENT PDES

The ability of deep-learning approaches to approximate complex functions offers a promising alternative for solving partial differential equations (PDEs). The methodology of incorporating the physics prior to the deep neural network can significantly reduce the requirement for labeled data. In this study, a novel physics-informed encoder-decoder gated recurrent units neural network is proposed to solve the time-dependent PDEs without using any observed data. The encoder is utilized to approximate the underlying patterns and structures of solutions over the entire spatial-temporal domain. The approximated solution is processed by the decoder, which is the gated recurrent units layer. We utilize the initial condition as the initial state of the gated recurrent units to retain critical information in the hidden states. The boundary conditions are enforced in the final prediction to enhance the model's performance. Then, we incorporate physical laws into the neural network during the training process. The effectiveness of this algorithm is demonstrated by solving Burgers-Fisher and coupled two-dimensional burgers' equations. The ability to identify unknown parameters is demonstrated through the solution of inverse problems.

Physics-informed neural network-based petroleum reservoir simulation with sparse data using domain decomposition

Recent advances in deep learning have expanded new possibilities for fluid flow simulation in petroleum reservoirs. However, the predominant approach in existing research is to train neural networks using high-fidelity numerical simulation data. This presents a significant challenge because the sole source of authentic wellbore production data for training is sparse. In response to this challenge, this work introduces a novel architecture called physics-informed neural network based on domain decomposition (PINN-DD), aiming to effectively utilize the sparse production data of wells for reservoir simulation with large-scale systems. To harness the capabilities of physics-informed neural networks (PINNs) in handling small-scale spatial-temporal domain while addressing the challenges of large-scale systems with sparse labeled data, the computational domain is divided into two distinct sub-domains: the well-containing and the well-free sub-domain. Moreover, the two sub-domains and the interface are rigorously constrained by the governing equations, data matching, and boundary conditions. The accuracy of the proposed method is evaluated on two problems, and its performance is compared against state-of-the-art PINNs through numerical analysis as a benchmark. The results demonstrate the superiority of PINN-DD in handling large-scale reservoir simulation with limited data and show its potential to outperform conventional PINNs in such scenarios.

Approximate Global Energy Management Based on Macro–Micro Mixed Traffic Model for Plug-in Hybrid Electric Vehicles

A multi-scale physical process management system is presented in this paper, taking the plug-in hybrid electric vehicle system as the physical interface connecting the macro traffic system to the micro energy conversion process, with the ultimate goal of global energy management in the full temporal–spatial domain for autonomous plug-in hybrid electric vehicles. This novel method adopts a macro traffic flow model at a large time scale, in which only the initial conditions and the traffic information of key road sections are required, and a car following model at the micro scale. Furthermore, local replanning of energy management is carried out by adjusting the power threshold and the efficiency weight through the type of reinforcement learning that is closest to human learning, once a short term speed disturbance is induced by unknown disturbances in the macro traffic flow. Due to the nonlinear relationship between speed fluctuation and power fluctuation, it is necessary to map the vehicle speed and acceleration characteristics to the power characteristics, instead of directly utilizing the traffic model characterized by the speed and acceleration characteristics. The results show that novel multi-scale physical management can achieve a smaller deviation from the global optimal solution and enhanced robustness of global energy management. Additionally, close coupling between the dynamic characteristics of vehicle components and speed fluctuation ensures correct tracking of the optimized target value.

Robust single-photon 3D imaging based on full-scale feature integration and intensity edge guidance

Active single-photon light detection and ranging (LiDAR) imaging technology adopting emerging single-photon avalanche diode (SPAD) has been used extensively for 3D imaging of complex scenes. However, the different signal-to-noise data acquired by SPAD under complex scenes and the larger pixel size of the SPAD pose a great challenge for robustly estimating the scene depth with rich detailed information. In this paper, we have designed a two-stage network including a multi-scale encoder-decoder sub-network and an intensity-guided edge refinement sub-network using a sensor fusion strategy to improve the robustness of the depth estimation network. In the first stage of the sub-network, we use the UNet3+ network to integrate fine-grained detail information and coarse-grained semantic information from full-scale through skip connections across scales on the decoder. In the meantime, the deep boosting network is added to the encoder to extract rich contextual information from the spatial-temporal domain to alleviate semantic gaps between the encoder and decoder. Moreover, in the second stage of the sub-network, we adopt an inverted y-shaped backbone network with the same structure between the encoder and decoder coupled with the same weight at both encoders. The purpose of this design is to extract structural similarity features with spatial consistency from intensity and depth to compensate for the missing detail of reconstructed depth by the first stage sub-network. The experimental results conducted on both simulated and real-world datasets demonstrate that our network outperforms previous networks. In addition, networks are trained on single or multiple signal-to-background ratio datasets to estimate the real-world data respectively, and the results show that our proposed network is more robust.

Eigenwave analysis of vortex-induced vibrations on long cylinders in sheared flows

The vortex-induced vibration of long cylinders in sheared flows is usually a mixture of standing and traveling waves. Understanding how the waveform is developed is of significant scientific and engineering interest. However, the interference between the traveling wave components makes the response patterns hard to interpret. Therefore, a spatial–temporal domain decomposition may offer a new perspective for the problem. This paper presents an eigenwave analysis method based on two-dimensional singular value decomposition, aiming to discover the hidden spatial–temporal modes. The application to experimental data shows its capability to separate the traveling and standing wave components, revealing the alternating dominance of these two waveforms along the span. It can also decompose the motion into two intrinsic traveling waves propagating in opposite directions, evolving and interacting in space and time. Based on the amplitude profiles of these waves, we locate the power-in/out regions and evaluate the lock-in range of reduced velocity as 4.73 to 7.68. It also reveals that the varying phase difference between multi-frequency modes, instead of the mode transition, is the main reason for the envelope fluctuation. The findings suggest that the proposed method is a useful data-driven technique extending the current toolkit for research on related topics.

Collaborative three-stream transformers for video captioning

As the most critical components in a sentence, subject, predicate and object require special attention in the video captioning task. To implement this idea, we design a novel framework, named COllaborative three-Stream Transformers (COST), to model the three parts separately and complement each other for better representation. Specifically, COST is formed by three branches of transformers to exploit the visual-linguistic interactions of different granularities in spatial–temporal domain between videos and text, detected objects and text, and actions and text. Meanwhile, we propose a cross-granularity attention module to align the interactions modeled by the three branches of transformers, then the three branches of transformers can support each other to exploit the most discriminative semantic information of different granularities for accurate predictions of captions. The whole model is trained in an end-to-end fashion. Extensive experiments conducted on three large-scale challenging datasets, i.e., YouCookII, ActivityNet Captions and MSVD, demonstrate that the proposed method performs favorably against the state-of-the-art methods.

The Extraction of Foreground Regions of the Moving Objects Based on Spatio-Temporal Information under a Static Camera

The rapid development of computer vision technology provides a basic guarantee for public security reliance on video surveillance. In current video surveillance based on static cameras, accurate and quick extractions of foreground regions of moving objects enable quicker analysis of the behavior of meaningful objects and thus improve the intelligent analysis level of video surveillance. However, there would always occur false detection in the extraction of foreground regions, because of the shaking of tree branches and leaves in the scene and the “ghosting” area caused by the delayed updating of the background model. To solve this problem, this paper proposes a method for the extraction of foreground regions by using spatio-temporal information. This method can accurately extract foreground regions of moving objects by utilizing the difference and complementarity between spatial domain methods and temporal domain methods and further in combination with image processing technology. Specifically, the foreground regions of moving objects can be extracted by the morphological processing of the combination of the spatial information and the morphologically processed temporal information in the video. The experimental results show that the proposed method for the extraction of foreground regions of moving objects in view of the spatio-temporal information can reduce false detections caused by the shaking of tree branches and leaves, and thus effectively extract foreground regions of moving objects.

Impacts of seasonally frozen soil hydrothermal dynamics on the watershed hydrological processes inferred from a spatially distributed numerical modelling approach

The freeze–thaw cycle over the surface seasonally frozen soil is an important soil hydrothermal dynamic process linking land surface processes and climatic changes in the cold regions. With the advancement of frozen soil hydrothermal dynamic studies and remote sensing technology, the simulation of frozen soil hydrological processes based on the distributed numerical model has become a hotspot to better understand the impact of frozen soil hydrothermal dynamics on the watershed hydrological processes in the cold regions over a large spatial scale. However, the quantitative analysis of the impact of seasonally frozen soil hydrothermal processes on watershed runoff at long-term time scales remained an unsolved issue in the field of frozen soil hydrology. Under the framework of the watershed distributed eco-hydrological model ESSI-3, a fully distributed frozen soil hydro-thermal processes integrated modeling system (FFIMS model) was established based on the coupled water and heat transferring mechanism for frozen soil hydro-thermal process simulations in the frozen surface or at a certain depth of a watershed. By coupling the FFIMS model with the distributed eco-hydrological model ESSI-3, the impacts of seasonally frozen soil hydrothermal processes on hydrological processes were investigated from the perspective of temporal-spatial domain with the simulated hydrothermal and hydrological processes for a long-term period from 2008 to 2016 over a watershed located in the cold region of Northeastern China. The results suggested that the soil freeze–thaw cycling posed different impacts with limited significance throughout the whole hydrological processes of the watershed in different seasons. Significant impacts on the hydrological processes were particularly observed in the thawing period of a year, when soil ice meltwater contributing to the discharge of the study watershed reached to about 35% in average in this period. ESSI-3 coupled with the FFIMS modelling system obviously improved the performance of the original ESSI-3 in cold region watershed simulations, and the averaged Nash efficiency coefficients obtained increased from almost 0 to 0.77 in the thawing period of a year. The study demonstrated the importance of application of spatially distributed numerical model with physical mechanism for seasonally frozen soil water and heat transfer process simulations.

Two-frame advanced iterative self-tuning algorithm for accurate phase retrieval

Two-shot random phase-shifting interferometry has been actively investigated and improved owing to more demanding requirements in investigating dynamic phenomena or sensing some transient events. Given the background intensity and phase step knowledge, a reconstruction algorithm can be used to recover a complex-valued signal from intensity-only measurements. Most algorithms assume the background intensity and modulation amplitude in a frame of interferogram are constant variables; however, in realistic systems, they vary laterally and axially across the field of view. The model can be used but often appears less precise. In this work, we propose an approach that leverages the iterative method to cope with spatially varying background intensity and modulation amplitude. Our approach, termed the two-frame advanced iterative self-tuning algorithm, uses the spatial mean algorithm paired with an iterative procedure to search the phase step from two captured fringe patterns. It represents a novel approach to reliable and practical two-step phase-shifting interferometry without pre-filtering due to the cancelation of simultaneously obtaining the phase shifts and measured phase through iterative operation in the spatial–temporal domain. Experimental results obtained using the proposed method indicate that it is a simple and robust solution for phase extraction from a two-frame unknown phase shift fringe pattern with spatially varying background intensity and modulation amplitude.

One Digital Health Intervention for Monitoring Human and Animal Welfare in Smart Cities: Viewpoint and Use Case.

Smart cities and digital public health are closely related. Managing digital transformation in urbanization and living spaces is challenging. It is critical to prioritize the emotional and physical health and well-being of humans and their animals in the dynamic and ever-changing environment they share. Human-animal bonds are continuous as they live together or share urban spaces and have a mutual impact on each other's health as well as the surrounding environment. In addition, sensors embedded in the Internet of Things are everywhere in smart cities. They monitor events and provide appropriate responses. In this regard, accident and emergency informatics (A&EI) offers tools to identify and manage overtime hazards and disruptive events. Such manifold focuses fit with One Digital Health (ODH), which aims to transform health ecosystems with digital technology by proposing a comprehensive framework to manage data and support health-oriented policies. We showed and discussed how, by developing the concept of ODH intervention, the ODH framework can support the comprehensive monitoring and analysis of daily life events of humans and animals in technologically integrated environments such as smart homes and smart cities. We developed an ODH intervention use case in which A&EI mechanisms run in the background. The ODH framework structures the related data collection and analysis to enhance the understanding of human, animal, and environment interactions and associated outcomes. The use case looks at the daily journey of Tracy, a healthy woman aged 27 years, and her dog Mego. Using medical Internet of Things, their activities are continuously monitored and analyzed to prevent or manage any kind of health-related abnormality. We reported and commented on an ODH intervention as an example of a real-life ODH implementation. We gave the reader examples of a "how-to" analysis of Tracy and Mego's daily life activities as part of a timely implementation of the ODH framework. For each activity, relationships to the ODH dimensions were scored, and relevant technical fields were evaluated in light of the Findable, Accessible, Interoperable, and Reusable principles. This "how-to" can be used as a template for further analyses. An ODH intervention is based on Findable, Accessible, Interoperable, and Reusable data and real-time processing for global health monitoring, emergency management, and research. The data should be collected and analyzed continuously in a spatial-temporal domain to detect changes in behavior, trends, and emergencies. The information periodically gathered should serve human, animal, and environmental health interventions by providing professionals and caregivers with inputs and "how-to's" to improve health, welfare, and risk prevention at the individual and population levels. Thus, ODH complementarily combined with A&EI is meant to enhance policies and systems and modernize emergency management.

LibCity-Dataset: a standardized and comprehensive dataset for urban spatial–temporal data mining

Abstract The LibCity-Dataset represents a significant contribution to the field of urban spatial-temporal data mining. This dataset uniquely integrates macro traffic state data with micro trajectory data, providing researchers with comprehensive and diverse urban spatial-temporal data. Specifically, we begin by collecting and processing existing open-source spatial-temporal data. Subsequently, we independently collected Beijing taxi trajectory data through third-party interfaces. This data bridges the gap in the scarcity of current open-source vehicle trajectory data. The distinctive aspect of the LibCity-Dataset lies in its innovative approach of standardizing the storage format, achieved through the implementation of atomic files. By adopting this standardized format, diverse data sources are harmonized, enabling effortless application of spatial-temporal prediction models across various datasets. The uniform storage format not only simplifies experimentation but also expedites the advancement of spatial-temporal prediction research, acting as a catalyst for further innovation. This Data Note provides a comprehensive insight into the creation methodology of the LibCity-Dataset, including data collection and processing methodology, data description, data validation, and usage notes. By facilitating open-source collaboration and setting a benchmark for standardization within the spatial-temporal prediction domain, this dataset aims to foster increased research cooperation and knowledge sharing. The open-source link of our dataset is https://drive.google.com/drive/folders/1g5v2Gq1tkOq8XO0HDCZ9nOTtRpB6-gPe.