Spatio-temporal Correlation Method Research Articles

SCGTracker: Spatio-temporal correlation and graph neural networks for multiple object tracking

Multiple Object Tracking is an important vitally important fundamental task in computer vision. Visual tracking becomes challenging when objects move in groups and are obscured from each other. There are two mainstream solution strategies for these group models. One is to transform the data association problem into a graph matching problem for solving, while the other is to apply the social power model as an advanced constraint for group tracking. In the former case, the solving difficulty geometric growth as the number of tracked objects increases, and the computing efficiency for real-time tracking demand cannot be met. The latter strategy tends to set up fixed-size groups or offline training rules, resulting in a lack of flexibility that limits their scenario generalization. According to the shortcomings of existing methods, this paper proposes a novel multiple object tracking method with spatio-temporal correlation and graph neural networks. Firstly, the relational features of the historical trajectories are extracted through the spatio-temporal relationship learning module, which models the spatio-temporal correlations of the objects and dynamically constructs the group structure online. Then, the graph neural network is combined with appearance and motion information, and the similarity between each detection and tracklet is used as a weight in node feature aggregation to make powerful distinctions between node features. Meanwhile, the spatio-temporal correlation method is also used to solve target loss issues caused by occlusion. Even collocated with linearly assigned data association method, good tracking results are still achieved, with a low computational complexity. Experiments on three challenging public datasets, namely MOT16, MOT17, and MOT20, validated the accuracy and efficiency of the proposed tracking method.

A Novel Approach for Mining Spatiotemporal Explicit and Implicit Information in Multiscale Spatiotemporal Data

In the era of big data, a significant volume of spatiotemporal data exists in a multiscale format, describing diverse phenomena in the objective world across different spatial and temporal scales. While existing methods focus on analyzing the features and connections of spatiotemporal data at various scales, they often overlook the consideration of uncertainty in spatiotemporal information within the context of multiscale meaning. To effectively harness the potential of spatiotemporal data, it becomes crucial to capture the fuzzy spatiotemporal information inherent in multiscale datasets. This paper proposes a novel multiscale spatiotemporal correlation method that accounts for and quantifies the uncertainty of spatiotemporal information. Spatiotemporal information is categorized into two types, explicit information and implicit information, based on respective levels of uncertainty. The method employs spatiotemporal cubes to interpret the spatiotemporal items within the data, followed by the introduction of a benchmark scale to determine the certainty of each spatiotemporal item based on its range and topological relationships. Subsequently, spatiotemporal confidence and correlation index are proposed to gauge the significance of geographical elements and their interrelationships. To validate the proposed method, a multiscale spatiotemporal transaction dataset is generated and utilized in the experiment. The experimental results demonstrate that the proposed method effectively captures spatiotemporal implicit information and enables better utilization of multiscale spatiotemporal data. Notably, the importance of each object of study varies when analyzed using different benchmark scales, providing valuable insights for professionals to identify novel objects and associations worthy of consideration. The obtained results can be used to construct spatiotemporal knowledge graphs.

Convection of multi-scale motions in turbulent boundary layer by temporal resolution particle image velocimetry

Experiments of particle image velocimetry (PIV) in the turbulent boundary layer at have been conducted to investigate the convection characteristic of turbulent structure and the validity of Taylor’s hypothesis. Views of (, the boundary layer thickness) were captured by four streamwise-arranged cameras. Distributions of streamwise turbulent kinetic energy on a streamwise scale were investigated by continuous-wave transform, and scales were found where the portion of streamwise turbulent kinetic energy approaches maximum. Fluctuating velocities (instant velocity minus average velocity on time dimension) were divided into large-scale motion (LSM) and small-scale motion (SSM) portions, bounded by . Convection velocities of LSM and SSM are determined by the spatiotemporal correlation method, and they are larger than local average velocities in near-wall regions, but smaller than local average velocities in wake regions. Statistical characteristics between velocity fields reconstructed by Taylor’s hypothesis and original fields were compared by the autocorrelation method, and the reconstructed field’s patterns are longer than original field’s patterns, while their heights do not have clear distinction. The correlation of original velocity fields and reconstructed fields shows that LSM can hold on over and SSM over in streamwise convection separation for regions of , given a threshold value (correlation coefficient C = 0.6).

Imputation of missing data from offshore wind farms using spatio-temporal correlation and feature correlation

Many industrial applications, such as fault diagnosis and remaining useful life prediction, require high-dimensional inputs to predict a reliable output. For offshore wind farm supervisory control and data acquisition (SCADA) systems, unfortunately, signal inputs are often missing due to harsh weather, resulting in the failure of network/sensors. It limits the accuracy of subsequent diagnostic or prognostic tasks. Although many methods have been proposed for imputing missing data, their applicability in offshore wind farms is still problematic because wind turbines (WTs) are time-varying systems, and conventional learning methods require high computational cost. To address this problem, we propose a learning framework containing two learning models, corresponding to two missing-data conditions. The framework imputes missing data by designing a spatio-temporal correlation method for entire feature-missing conditions and a feature-correlation method for partial feature-missing conditions, respectively. A real-world offshore wind farm dataset of a SCADA system with 33 WTs and 68 features, which was recorded over a one-month period, is used for experimental validation. We demonstrate that the proposed framework imputes the missing data with much smaller mean absolute error (MAE) and mean squared error (MSE) and requires less computational time, compared to the existing machine-learning methods for both imputation conditions.

Spatiotemporal correlation of optical coherence tomography in-vivo images of rabbit airway for the diagnosis of edema.

Detection of an early stage of subglottic edema is vital for airway management and prevention of stenosis, a life-threatening condition in critically ill neonates. As an observer for the task of diagnosing edema in vivo, we investigated spatiotemporal correlation (STC) of full-range optical coherence tomography (OCT) images acquired in the rabbit airway with experimentally simulated edema. Operating the STC observer on OCT images generates STC coefficients as test statistics for the statistical decision task. Resulting from this, the receiver operating characteristic (ROC) curves for the diagnosis of airway edema with full-range OCT in-vivo images were extracted and areas under ROC curves were calculated. These statistically quantified results demonstrated the potential clinical feasibility of the STC method as a means to identify early airway edema.

On the role of lagged exogenous variables and spatio–temporal correlations in improving the accuracy of solar forecasting methods

We propose and analyze a spatio–temporal correlation method to improve forecast performance of solar irradiance using gridded satellite-derived global horizontal irradiance (GHI) data. Forecast models are developed for seven locations in California to predict 1-h averaged GHI 1, 2 and 3 h ahead of time. The seven locations were chosen to represent a diverse set of maritime, mediterranean, arid and semi-arid micro-climates. Ground stations from the California Irrigation Management Information System were used to obtain solar irradiance time-series from the points of interest. In this method, firstly, we define areas with the highest correlated time-series between the satellite-derived data and the ground data. Secondly, we select satellite-derived data from these regions as exogenous variables to several forecast models (linear models, Artificial Neural Networks, Support Vector Regression) to predict GHI at the seven locations. The results show that using linear forecasting models and a genetic algorithm to optimize the selection of multiple time-lagged exogenous variables results in significant forecasting improvements over other benchmark models.

UTN-Model-Based Traffic Flow Prediction for Parallel-Transportation Management Systems

Aiming to comply with the requirement of parallel-transportation management systems (PtMS), this paper presents a short-term traffic flow prediction method for signal-controlled urban traffic networks (UTNs) based on the macroscopic UTN model. In contrast with other time-series-based or spatio-temporal correlation methods, the proposed method focuses more on using the substantial mechanism of traffic transmission in road networks and the topology model of the entire UTN. Furthermore, this approach employs a speed-density model based on the fundamental diagram (FD) to obtain more accurate travel times in links. In the comparison experiment, the microscopic traffic simulation software CORSIM is adopted to simulate the real urban traffic. The experiment results fully verify the outstanding performances of the proposed prediction method.

A proficient wavelet-based video encoder through the motion estimation algorithm

In several multimedia applications, image and video coding plays a significant role. In many applications, sceneries such as broadcast services over satellite and terrestrial channels, digital video storage, wires and wireless conversational services and digital video communication are used. It is not possible to store full digital video without processing. Video compression reduces the data used to represent digital video images and is a combination of spatial image compression and temporal motion compensation. The main aim of our research is to develop an efficient video compression system. The proposed system consists of three steps. At first, wavelet decomposition is applied to the I–frame and the resulting coefficients are quantised using the listless SPECK (LSK) algorithm. In the next step, motion estimation is done using adaptive rood search with the spatio–temporal correlation method (ARS–ST) and it calculates the distance between the P–frame and I–frame blocks. In the final step, the difference between the original and the predicted P–frame is evaluated and is known as residual. To produce good quality predictive frames, this residual is transmitted along with motion vectors (MVs) and to decrease its size, it is coded using LSK. The proposed video compression technique uses different videos for assessment and by determining the PSNR values, compression efficiency is estimated. By comparing the proposed system with existing systems, it is seen that our system effectively compresses videos with remarkable PSNR measurements.

Spatiotemporal Image Correlation Spectroscopy Reveals Arp2/3-Complex-Driven Cytoplasmic Streaming in Mouse Oocytes, Maintaining Meiotic Cortical Spindle Positioning

We use spatio-temporal correlation methods with transmitted light microscopy to demonstrate a novel pattern of internal-to-cortical circular cytoplasmic flow in meiosis II mouse oocytes. This flow is responsible for poising the spindle near the cortex for completion of meiosis II and successful oocyte fertilization. We show that this flow is driven by a balance of Arp2/3-complex-induced actin nucleation at the cortex and myosin-II-driven contraction of the cortex. STICS analysis of EGFP-Utrophin-labeled actin fibers shows cortical outflows with velocities consistent with those required to drive the cytoplasmic streaming.

Estimation of Epileptic Foci Localization by Plane-Based Spatio-Temporal Correlation Method

Estimation of Epileptic Foci Localization by Plane-Based Spatio-Temporal Correlation Method

Improved spatio-temporal correlation method for velocity field measurements

We have improved the spatio-temporal correlation method to obtain velocity field from moving images. The new method can specify a certain image frame out of many sequential frames of a moving object and determines its 2-D velocity field on the basis of the specified frame. In addition, it enables us to avoid an edge effect of velocity field that appears at both ends of image sequence. This improvement has enhanced reliability and reduced computational time, compared to the previous methods. As a further application, we will describe this method’s application to a non-solid moving object and a sport scene analysis.

3-D Velocity Measurement of Natural Convection Using Image Processing.

This paper describes quantitative three-dimensional measurement method for flow field of a rotating Rayleigh-Benard convection in a cylindrical cell heated below and cooled above. A correlation method for two-dimensional measurement was well advanced to a spatio-temporal correlation method. Erroneous vectors, often appeared in the correlation method, was successfully removed using Hopfield neural network. As a result, calculated 3-D velocity vector distribution well corresponded to the observed temperature distribution. Consequently, the simultaneous three-dimensional measurement system for temperature and flow field was developed.

3-D Velocity Measurement of Natural Convection using Image Processing.

This paper describes quantitative 3-D measurement method for flow field of a rotating Rayleigh-Benard convection in a cylindrical cell heated below and cooled above. A correlation method for 2-D measurement is well advanced to a spatio-temporal correlation method. Erroneous vectors, often appeared in the correlation method, is successfully removed using Hopfield network. As a result, calculated 3-D velocity vector distribution well corresponds to the temperature distribution.

Temperature and velocity measurement of a 3-D thermal flow field using thermo-sensitive liquid crystals

It is an important challenge to analyze a three-dimensional thermal flow field in engineering, science, and agriculture. For such an analysis, it is essential to measure physical quantities such as temperature and velocity over the entire thermal flow field. This paper presents a measurement system based on color image processing for temperature and velocity vector distributions in a three-dimensional thermal flow field. Flow visualization is accomplished by the use of thermo-sensitive liquid crystal tracers. An algorithm for the color-to-temperature transformation using a multi-layer feed-forward neural network is applied to three-dimensional natural convection in a rotating cylindrical cell. Two-dimensional temperature distributions in a slit plane are obtained by using the algorithm. A three-dimensional temperature distribution is consequently constructed by interpolating the two-dimensional distributions using the B-spline function. In addition, the Spatio-Temporal correlation method is applied to the natural convection to obtain a three-dimensional velocity vector distribution.

時空間相関に基づく粒子画像速度場計測法の解析性能改善

時空間相関に基づく粒子画像速度場計測法の解析性能改善

時空間相関法に基づく流れ場の3次元速度ベクトル計測

時空間相関法に基づく流れ場の3次元速度ベクトル計測

時空間相関法を用いた流れの三次元画像計測

A new measurement system of three-dimensional flow velocity vector was developed using a spatio-temporal correlation method. This method needs simultaneously two images observed at a short distance in the flow direction. Each of the two images consists of time series of the frames. A three-dimensional velocity vector is obtained by calculating spatio-temporal correlation between the two images.