Spatiotemporal Set Research Articles

A Bayesian spatio-temporal level set dynamic model and application to fire front propagation

A Bayesian spatio-temporal level set dynamic model and application to fire front propagation

Spatiotemporal knowledge teacher–student reinforcement learning to detect liver tumors without contrast agents

Detecting Liver tumors without contrast agents (CAs) has shown great potential to advance liver cancer screening. It enables the provision of a reliable liver tumor-detecting result from non-enhanced MR images comparable to the radiologists’ results from CA-enhanced MR images, thus eliminating the high risk of CAs, preventing an experience gap between radiologists and simplifying clinical workflows. In this paper, we proposed a novel spatiotemporal knowledge teacher–student reinforcement learning (SKT-RL) as a safe, speedy, and inexpensive contrast-free technology for liver tumor detection. Our SKT-RL builds a teacher–student framework to realize the exploring of explicit liver tumor knowledge from a teacher network on clear contrast-enhanced images to guide a student network to detect tumors from non-enhanced images directly. Importantly, our STK-RL enables three novelties in aspects of construction, transferring, and optimization to tumor knowledge to improve the guide effect. (1) A new spatiotemporal ternary knowledge set enables the construction of accurate knowledge that allows understanding of DRL’s behavior (what to do) and reason (why to do it) behind reliable detection within each state and between their related historical states. (2) A novel pixel momentum transferring strategy enables detailed and controlled knowledge transfer ability. It transfers knowledge at a pixel level to enlarge the explorable space of transferring and control how much knowledge is transferred to prevent over-rely of the student to the teacher. (3) A phase-trend reward function designs different evaluations according to different detection phases to optimal for each phase in high-precision but also allows reward trend to constraint the evaluation to improve stability. Comprehensive experiments on a generalized liver tumor dataset with 375 patients (including hemangiomas, hepatocellular carcinoma, and normal controls) show that our novel SKT-RL attains a new state-of-the-art performance (improved precision by at least 4% when comparing the six recent advanced methods) in the task of liver tumor detection without CAs. The results proved that our SKT-DRL has greatly promoted the development and deployment of contrast-free liver tumor technology.

Trajectory planning based on spatio-temporal reachable set considering dynamic probabilistic risk

Trajectory planning in complex traffic situations has always been a challenging task for intelligent vehicles. Comparing to the decoupling method, the spatio-temporal trajectory planning method owns more flexibility and reasonability due to the combination of lateral and longitudinal motion. However, it still has some shortcomings, such as unreasonable risk assessment, high computational complexity and heavy dependence on other models for generating target points. Therefore, a novel spatio-temporal based trajectory planning framework considering probability risk is proposed in this paper. Firstly, a GNN-LSTM based on trajectory prediction algorithm is presented in terms of risk analysis, and particularly the predicted trajectories and the vehicle dynamic model are combined for risk assessment. Secondly, a rough-fine hierarchical planning framework based on reachable set and dynamic programming is proposed. In this framework, the reachable set is taken as spatio-temporal node to reduce the computational costs and dynamic programming can help the algorithm to eliminate the dependence of other models in evaluating target points. Finally, a traffic scenario with random interactive obstacles is built and tested on a HIL platform. The experimental results show that compared with other typical algorithms, the average driving efficiency, driving risk behavior and driving comfort of the vehicle are significantly improved.

Epileptic prediction using spatiotemporal information combined with optimal features strategy on EEG.

Epilepsy is the second most common brain neurological disease after stroke, which has the characteristics of sudden and recurrence. Seizure prediction is seriously important for improving the quality of patients' lives. From the perspective of multiple dimensions including time-frequency, entropy and brain network, this paper proposed a novel approach by constructing the optimal spatiotemporal feature set to predict seizures. Based on strong independence and large information capabilities, the two-dimensional feature screening algorithm is performed to eliminate unnecessary redundant features. In order to verify the effectiveness of the optimal feature set, support vector machine (SVM) was used to classify the preictal and interictal states on both the Kaggle intracranial EEG and CHB-MIT scalp EEG dataset. This model achieved an average accuracy of 98.01%, AUC of 0.96, F-Score of 98.3% and FPR of 0.0383/h on the Kaggle dataset; On the CHB-MIT dataset, the average accuracy, AUC, F-score and FPR were 95.93%, 0.92, 94.97% and 0.0473/h, respectively. Further ablation experiments have confirmed that the temporal and spatial features fusion has better performance than the individual temporal or spatial features. Compared to the state-of-the-art methods, our approach outperforms most of these existing techniques. The results show that our approach can effectively extract the spatiotemporal information of epileptic EEG signals to predict epileptic seizures with high performance.

Kernel-based learning of birth process from evolving spatiotemporal RFS data stream in SMC[sbnd]CPHD filter for multi-target tracking

This paper presents a kernel-based framework to estimate the birth process in the sequential Monte Carlo cardinalized probability hypothesis density (SMCCPHD) filter from the evolving spatiotemporal birth random finite set data stream that can enhance the performance of the SMCCPHD based trackers when tracking targets with no a priori information on where and when targets can appear. The time-varying spatial distribution of birth events is modelled as a map with adaptive grid points and their corresponding density values. While the spatial distribution of birth density is estimated from the elements of the birth random finite sets (RFSs), the time-varying birth cardinality distribution is estimated from the cardinality of the birth RFSs. The corresponding estimations are eventually used by a SMCCPHD based tracker to adjust its filtering strength spatially and temporally for target tracking. Experimental results on both the synthetic and real video surveillance datasets show that the proposed algorithm can improve the tracking accuracy by more precisely eliminating clutters and more effectively capturing newborn targets.

Spatio-temporal analysis of chromospheric heating in a plage region

Context. Our knowledge of the heating mechanisms that are at work in the chromosphere of plage regions remains highly unconstrained from observational studies. While many heating candidates have been proposed in theoretical studies, the exact contribution from each of them is still unknown. The problem is rather difficult because there is no direct way of estimating the heating terms from chromospheric observations. Aims. The purpose of our study is to estimate the chromospheric heating terms from a multi-line high-spatial-resolution plage dataset, characterize their spatio-temporal distribution and set constraints on the heating processes that are at work in the chromosphere. Methods. We used nonlocal thermodynamical equilibrium inversions in order to infer a model of the photosphere and chromosphere of a plage dataset acquired with the Swedish 1-m Solar Telescope (SST). We used this model atmosphere to calculate the chromospheric radiative losses from the main chromospheric cooler from H I, Ca II, and Mg II atoms. In this study, we approximate the chromospheric heating terms by the net radiative losses predicted by the inverted model. In order to make the analysis of time-series over a large field of view computationally tractable, we made use of a neural network which is trained from the inverted models of two non-consecutive time-steps. We have divided the chromosphere in three regions (lower, middle, and upper) and analyzed how the distribution of the radiative losses is correlated with the physical parameters of the model. Results. In the lower chromosphere, the contribution from the Ca II lines is dominant and predominantly located in the surroundings of the photospheric footpoints. In the upper chromosphere, the H I contribution is dominant. Radiative losses in the upper chromosphere form a relatively homogeneous patch that covers the entire plage region. The Mg II also peaks in the upper chromosphere. Our time analysis shows that in all pixels, the net radiative losses can be split in a periodic component with an average amplitude of amp̅Q = 7.6 kW m−2 and a static (or very slowly evolving) component with a mean value of −26.1 kW m−2. The period of the modulation present in the net radiative losses matches that of the line-of-sight velocity of the model. Conclusions. Our interpretation is that in the lower chromosphere, the radiative losses are tracing the sharp lower edge of the hot magnetic canopy that is formed above the photosphere, where the electric current is expected to be large. Therefore, Ohmic current dissipation could explain the observed distribution. In the upper chromosphere, both the magnetic field and the distribution of net radiative losses are room-filling and relatively smooth, whereas the amplitude of the periodic component is largest. Our results suggest that acoustic wave heating may be responsible for one-third of the energy deposition in the upper chromosphere, whereas other heating mechanisms must be responsible for the rest: turbulent Alfvén wave dissipation or ambipolar diffusion could be among them. Given the smooth nature of the magnetic field in the upper chromosphere, we are inclined to rule out Ohmic dissipation of current sheets in the upper chromosphere.

Spatiotemporal Eye-Tracking Feature Set for Improved Recognition of Dyslexic Reading Patterns in Children.

Considering the detrimental effects of dyslexia on academic performance and its common occurrence, developing tools for dyslexia detection, monitoring, and treatment poses a task of significant priority. The research performed in this paper was focused on detecting and analyzing dyslexic tendencies in Serbian children based on eye-tracking measures. The group of 30 children (ages 7–13, 15 dyslexic and 15 non-dyslexic) read 13 different text segments on 13 different color configurations. For each text segment, the corresponding eye-tracking trail was recorded and then processed offline and represented by nine conventional features and five newly proposed features. The features were used for dyslexia recognition using several machine learning algorithms: logistic regression, support vector machine, k-nearest neighbor, and random forest. The highest accuracy of 94% was achieved using all the implemented features and leave-one-out subject cross-validation. Afterwards, the most important features for dyslexia detection (representing the complexity of fixation gaze) were used in a statistical analysis of the individual color effects on dyslexic tendencies within the dyslexic group. The statistical analysis has shown that the influence of color has high inter-subject variability. This paper is the first to introduce features that provide clear separability between a dyslexic and control group in the Serbian language (a language with a shallow orthographic system). Furthermore, the proposed features could be used for diagnosing and tracking dyslexia as biomarkers for objective quantification.

Epileptic electroencephalography classification using embedded dynamic mode decomposition

Objective. Seizure prediction devices for drug-resistant epileptic patients could lead to improved quality of life and new treatment options, but current approaches to classification of electroencephalography (EEG) segments for early identification of the pre-seizure state typically require many features and complex classifiers. We therefore propose a novel spatio-temporal EEG feature set that significantly aids in separation and easy classification of the interictal and preictal states. Approach. We derive key spectral features from the embedded dynamic mode decomposition (EmDMD) of the brain state system. This method linearizes the complex spatio-temporal dynamics of the system, describing the dynamics in terms of a spectral basis of modes and eigenvalues. The relative subband spectral power and mean phase locking values of these modes prove to be good indicators of the preictal state that precedes seizure onset. Main results. We analyze the linear separability and classification of preictal and interictal states based on our proposed features using seizure data extracted from the CHB-MIT scalp EEG and Kaggle American Epilepsy Society Seizure Prediction Challenge intracranial EEG databases. With a light-weight support vector machine or random forest classifier trained on these features, we classify the preictal state with a sensitivity of up to 92% and specificity of up to 89%. Significance. The EmDMD-derived features separate the preictal and interictal states, improving classification accuracy and motivating further work to incorporate them into seizure prediction algorithms.

PCT-TEE: Trajectory-based Private Contact Tracing System with Trusted Execution Environment

Existing Bluetooth-based private contact tracing (PCT) systems can privately detect whether people have come into direct contact with patients with COVID-19. However, we find that the existing systems lack functionality and flexibility , which may hurt the success of contact tracing. Specifically, they cannot detect indirect contact (e.g., people may be exposed to COVID-19 by using a contaminated sheet at a restaurant without making direct contact with the infected individual); they also cannot flexibly change the rules of “risky contact,” such as the duration of exposure or the distance (both spatially and temporally) from a patient with COVID-19 that is considered to result in a risk of exposure, which may vary with the environmental situation. In this article, we propose an efficient and secure contact tracing system that enables us to trace both direct contact and indirect contact. To address the above problems, we need to utilize users’ trajectory data for PCT, which we call trajectory-based PCT . We formalize this problem as a spatiotemporal private set intersection that satisfies both the security and efficiency requirements. By analyzing different approaches such as homomorphic encryption, which could be extended to solve this problem, we identify the trusted execution environment (TEE) as a candidate method to achieve our requirements. The major challenge is how to design algorithms for a spatiotemporal private set intersection under the limited secure memory of the TEE. To this end, we design a TEE-based system with flexible trajectory data encoding algorithms. Our experiments on real-world data show that the proposed system can process hundreds of queries on tens of millions of records of trajectory data within a few seconds.

Steel Wire Rope Surface Defect Detection Based on Segmentation Template and Spatiotemporal Gray Sample Set.

Machine-vision-based defect detection, instead of manual visual inspection, is becoming increasingly popular. In practice, images of the upper surface of cableway load sealing steel wire ropes are seriously affected by complex environments, including factors such as lubricants, adhering dust, natural light, reflections from metal or oil stains, and lack of defect samples. This makes it difficult to directly use traditional threshold-segmentation-based or supervised machine-learning-based defect detection methods for wire rope strand segmentation and fracture defect detection. In this study, we proposed a segmentation-template-based rope strand segmentation method with high detection accuracy, insensitivity to light, and insensitivity to oil stain interference. The method used the structural characteristics of steel wire rope to create a steel wire rope segmentation template, the best coincidence position of the steel wire rope segmentation template on the real-time edge image was obtained through multiple translations, and the steel wire rope strands were segmented. Aiming at the problem of steel wire rope fracture defect detection, inspired by the idea of dynamic background modeling, a steel wire rope surface defect detection method based on a steel wire rope segmentation template and a timely spatial gray sample set was proposed. The spatiotemporal gray sample set of each pixel in the image was designed by using the gray similarity of the same position in the time domain and the gray similarity of pixel neighborhood in the space domain, the dynamic gray background of wire rope surface image was constructed to realize the detection of wire rope surface defects. The method proposed in this paper was tested on the image set of Z-type double-layer load sealing steel wire rope of mine ropeway, and compared with the classic dynamic background modeling methods such as VIBE, KNN, and MOG2. The results show that the purposed method is more accurate, more effective, and has strong adaptability to complex environments.

Construction of the Long-Term Global Surface Water Extent Dataset Based on Water-NDVI Spatio-Temporal Parameter Set

Inland surface water is highly dynamic, seasonally and inter-annually, limiting the representativity of the water coverage information that is usually obtained at any single date. The long-term dynamic water extent products with high spatial and temporal resolution are particularly important to analyze the surface water change but unavailable up to now. In this paper, we construct a global water Normalized Difference Vegetation Index (NDVI) spatio-temporal parameter set based on the Moderate-resolution Imaging Spectroradiometer (MODIS) NDVI. Employing the Google Earth Engine, we construct a new Global Surface Water Extent Dataset (GSWED) with coverage from 2000 to 2018, having an eight-day temporal resolution and a spatial resolution of 250 m. The results show that: (1) the MODIS NDVI-based surface water mapping has better performance compared to other water extraction methods, such as the normalized difference water index, the modified normalized difference water index, and the OTSU (maximal between-cluster variance method). In addition, the water-NDVI spatio-temporal parameter set can be used to update surface water extent datasets after 2018 as soon as the MODIS data are updated. (2) We validated the GSWED using random water samples from the Global Surface Water (GSW) dataset and achieved an overall accuracy of 96% with a kappa coefficient of 0.9. The producer’s accuracy and user’s accuracy were 97% and 90%, respectively. The validated comparisons in four regions (Qinghai Lake, Selin Co Lake, Utah Lake, and Dead Sea) show a good consistency with a correlation value of above 0.9. (3) The maximum global water area reached 2.41 million km2 between 2000 and 2018, and the global water showed a decreasing trend with a significance of P = 0.0898. (4) Analysis of different types of water area change regions (Selin Co Lake, Urmia Lake, Aral Sea, Chiquita Lake, and Dongting Lake) showed that the GSWED can not only identify the seasonal changes of the surface water area and abrupt changes of hydrological events but also reflect the long-term trend of the water changes. In addition, GSWED has better performance in wetland areas and shallow areas. The GSWED can be used for regional studies and global studies of hydrology, biogeochemistry, and climate models.

Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks

This paper compares clinician hand motion for common suturing tasks across a range of experience levels and tissue types. Medical students (32), residents (41), attending surgeons (10), and retirees (2) were recorded on digital video while suturing on one of: foam, pig feet, or porcine bowel. Depending on time in position, each medical student, resident, and attending participant was classified as junior or senior, yielding six experience categories. This work focuses on trends associated with increasing tenure observed from those medical students (10), residents (15), and attendings (10) who sutured on foam, and draws comparison across tissue types where pertinent. Utilizing custom software, the two-dimensional location of each of the participant's hands were automatically recorded in every video frame, producing a rich spatiotemporal feature set. While suturing on foam, increasing clinician experience was associated with conserved path length per cycle of the non-dominant hand, significantly reducing from junior medical students (mean = 73.63 cm, sd = 33.21 cm) to senior residents (mean = 46.16 cm, sd = 14.03 cm, p = 0.015), and again between senior residents and senior attendings (mean = 30.84 cm, sd = 14.51 cm, p = 0.045). Despite similar maneuver rates, attendings also accelerated less with their non-dominant hand (mean = 16.27 cm/s2, sd = 81.12 cm/s2, p = 0.002) than senior residents (mean = 24.84 cm/s2, sd = 68.29 cm/s2, p = 0.002). While tying, medical students moved their dominant hands slower (mean = 4.39 cm/s, sd = 1.73 cm/s, p = 0.033) than senior residents (mean = 6.53 cm/s, sd = 2.52 cm/s). These results suggest that increased psychomotor performance during early training manifest through faster dominant hand function, while later increases are characterized by conserving energy and efficiently distributing work between hands. Incorporating this scalable video-based motion analysis into regular formative assessment routines may enable greater quality and consistency of feedback throughout a surgical career.

Low-Complexity and Hardware-Friendly H.265/HEVC Encoder for Vehicular Ad-Hoc Networks.

Real-time video streaming over vehicular ad-hoc networks (VANETs) has been considered as a critical challenge for road safety applications. The purpose of this paper is to reduce the computation complexity of high efficiency video coding (HEVC) encoder for VANETs. Based on a novel spatiotemporal neighborhood set, firstly the coding tree unit depth decision algorithm is presented by controlling the depth search range. Secondly, a Bayesian classifier is used for the prediction unit decision for inter-prediction, and prior probability value is calculated by Gibbs Random Field model. Simulation results show that the overall algorithm can significantly reduce encoding time with a reasonably low loss in encoding efficiency. Compared to HEVC reference software HM16.0, the encoding time is reduced by up to 63.96%, while the Bjontegaard delta bit-rate is increased by only 0.76–0.80% on average. Moreover, the proposed HEVC encoder is low-complexity and hardware-friendly for video codecs that reside on mobile vehicles for VANETs.

Modeling spatiotemporal topological relationships between moving object trajectories along road networks based on region connection calculus

ABSTRACTConsidering the attempts to model spatiotemporal topological relationships between moving object trajectories, the conceptual and computational framework for moving objects along a road network has not received much attention. This paper aims to draw an improved model based on Region Connection Calculus (RCC) theory to represent the spatiotemporal topological relationships between moving object trajectories along road networks. This paper first uses a dimension reduction method based on a linear-reference transformation to model the moving object trajectories segments, and then defines new time–connection and space–connection relations between two trajectory segments. On this basis, the paper proposes an extension to the RCC-based spatiotemporal binary relationship set so that the combined semantics of the spatiotemporal predicates can be described completely. A case study was carried out using Floating Car Data in Guangzhou city. The computational results show that in a real application, the occurrence frequencies of the RCC-based binary relationships are distributed nonuniformly and the semantics of some binary relationships with the highest occurrence are coarse. Therefore, the partition of the spatiotemporal connection relations and the finer aspects of the spatiotemporal relationship model may require further research work.

Spatio-Temporal Model for a Random Set Given by a Union of Interacting Discs

A spatio-temporal random set parametric model is defined based on the union of interacting discs. There are two types of parameters: those of the spatial part of the model and those of the state space model for temporal evolution. The simulation of the random set is available using a Markov chain Monte Carlo algorithm. Integral-geometric characteristics are evaluated and serve as an input to parameter estimation. We compare an MCMC maximum likelihood estimator with a particle filter estimator in a simulation study by drawing their temporal evolution and globally by means of the integrated mean square error. Interpretations of parameters and possible applications are discussed.

Automatic Generation of MPEG-7 Compliant XML Document for Motion Trajectory Descriptor in Sports Video

The MPEG-7 standard is a step towards standardizing the description of multimedia content so that quick and efficient identification of relevant content can be facilitated, together with efficient management of information. The description definition language (DDL) is a schema language to represent valid MPEG-7 descriptors and description schemes. MPEG-7 instances are XML documents that conform to a particular MPEG-7 schema, as expressed in the DDL and that describe audiovisual content. In this paper, we pick one of the visual descriptors related to motion in a video sequence, viz., motion trajectory. It describes the displacements of objects in time, where an object is defined as a spatiotemporal region or set of spatiotemporal regions. We present a method of automatically extracting trajectories from video sequences and generating an XML document that conforms to the MPEG-7 schema. We use sports videos in particular, because the trajectories are very random and the robustness of our algorithm can be demonstrated.

A STAP algorithm approach to fMRI: a simulation study.

To adapt the space-time adaptive processing (STAP) algorithm, previously developed in the field of sensor array processing and applied to radar signal processing, for use in construction of brain activation maps in functional magnetic resonance imaging (fMRI). STAP is a two-dimensional filter in which both the spatial and temporal responses are controlled adaptively. It processes space-time data as a complete spatiotemporal set. Unlike presently used fMRI techniques, STAP locates activated regions both spatially and in frequency. Computer simulations incorporating actual MRI noise indicate that STAP exhibits a high degree of accuracy in detecting the small signal intensity changes inherent in fMRI. Because STAP processes space-time data as a single data matrix, it exhibits potential over currently available fMRI methods in providing a measure of the full spatiotemporal extent of a task-related activity.

Improved image quality and computation reduction in 4-D reconstruction of cardiac-gated SPECT images

Spatiotemporal reconstruction of cardiac-gated SPECT images permits us to obtain valuable information related to cardiac function. However, the task of reconstructing this four-dimensional (4-D) data set is computation intensive. Typically, these studies are reconstructed frame-by-frame: a nonoptimal approach because temporal correlations in the signal are not accounted for. In this work, we show that the compression and signal decorrelation properties of the Karhunen-Loève (KL) transform may be used to greatly simplify the spatiotemporal reconstruction problem. The gated projections are first KL transformed in the temporal direction. This results in a sequence of KL-transformed projection images for which the signal components are uncorrelated along the time axis. As a result, the 4-D reconstruction task is simplified to a series of three-dimensional (3-D) reconstructions in the KL domain. The reconstructed KL components are subsequently inverse KL transformed to obtain the entire spatiotemporal reconstruction set. Our simulation and clinical results indicate that KL processing provides image sequences that are less noisy than are conventional frame-by-frame reconstructions. Additionally, by discarding high-order KL components that are dominated by noise, we can achieve savings in computation time because fewer reconstructions are needed in comparison to conventional frame-by-frame reconstructions.