Complex Temporal Correlations Research Articles

WALDATA: Wavelet transform based adversarial learning for the detection of anomalous trading activities

Detecting manipulative activities in stock market trading poses a significant challenge due to the complex temporal correlations inherent to the dynamically changing stock price data. This challenge is further exacerbated by the limited availability of labelled anomalous trading data instances. Stock price manipulations, which consist of infrequent anomalies in stock price trading data, are challenging to capture due to their sporadic occurrence and dynamically evolving nature. This scarcity and inherent complexity significantly complicate the creation of labelled datasets hence hinders the development of robust detection of different stock price manipulation schemes through supervised learning methods. Overcoming these challenges is crucial for enhancing our understanding of market dynamics and implementing robust market surveillance systems. To address these challenges, we introduce a novel stock price manipulation detection approach called WALDATA (Wavelet Transform based Adversarial Learning for the Detection of Anomalous Trading Activities). We leverage the Wavelet Transform (WT) to decompose non-stationary stock price time series into informative features and capture multi-scale dynamics within the data. We encode stock price data by transforming it into scalogram images through the Continuous Wavelet Transform, effectively converting stock price time series data into a 2D image representation. Subsequently, we employ a Generative Adversarial Network (GAN) architecture, originally applied to computer vision, to learn the underlying distribution of normal trading behaviour from the encoded images. We then train the discriminator as an anomaly detector for identifying manipulative trading activities in the stock market. The efficacy of WALDATA is rigorously evaluated on diverse real-world stock datasets using 1-level tick data from the LOBSTER project and the experimental results demonstrate the significant performance of our approach achieving an average AUC of 0.99 while maintaining low false alarm rates across various market conditions. These findings not only validate the effectiveness of the proposed WALDATA approach in accurately identifying stock price manipulations but also provide investors and regulators alike with valuable insights for the development of advanced market surveillance systems. This research demonstrates the promising potential of combining wavelet-based feature extraction and stock price time series to image representation with generative adversarial learning frameworks for anomaly detection in financial time series data. The successful implementation of WALDATA contributes to the development of advanced market surveillance systems and paves the way for further advancements in market surveillance, contributing towards a more efficient and robust financial system and a fair market environment.

PGNM: Using Physics-Informed Gated Recurrent Units Network Method to capture the dynamic data feature propagation process of PDEs

The multi-layer perceptron architecture in PINNs model severely limits the model’s ability to learn the temporal evolution of equation features. Instead, the GRU network is capable of capturing these complex temporal correlation features. This paper replaces the multi-layer perceptron structure of the PINNs model with the GRU network and called the modified model as physics-informed gated recurrent units network method (PGNM model). The PGNM model exhibits enhanced performance in assimilating insights from historical data and providing accurate predictions for future data. This paper compares the predictive performance of the proposed PGNM model and the classical PINNs model using L2 error and mean squared error (MSE) as metrics. Additionally, it evaluates how altering various parameter settings, such as the number of neurons, hidden layers and iterations, affects the predictive capabilities of both models. In conclusion, PGNM model shows significant improvement in prediction accuracy compared to PINNs model. Furthermore, PGNM model achieves better long-range prediction results than PINNs model when the training data does not include samples from the time period to be predicted.

An unsupervised adversarial autoencoder for cyber attack detection in power distribution grids

Detection of cyber attacks in smart power distribution grids with unbalanced configurations poses challenges due to the inherent nonlinear nature of these uncertain and stochastic systems. It originates from the intermittent characteristics of the distributed energy resources (DERs) generation and load variations. Moreover, the unknown behavior of cyber attacks, especially false data injection attacks (FDIAs) in the distribution grids with complex temporal correlations and the limited amount of labeled data increases the vulnerability of the grids and imposes a high risk in the secure and reliable operation of the grids. To address these challenges, this paper proposes an unsupervised adversarial autoencoder (AAE) model to detect FDIAs in unbalanced power distribution grids integrated with DERs, i.e., PV systems and wind generation. The proposed method utilizes long short-term memory (LSTM) in the structure of the autoencoder to capture the temporal dependencies in the time-series measurements and leverages the power of generative adversarial networks (GANs) for better reconstruction of the input data. The advantage of the proposed data-driven model is that it can detect anomalous points for the system operation without reliance on abstract models or mathematical representations and data labels. To evaluate the efficacy of the approach, it is tested on IEEE 13-bus and 123-bus systems with historical meteorological data (wind speed, ambient temperature, and solar irradiance) as well as historical real-world load data under three types of data falsification functions. The comparison of the detection results of the proposed model with other data-driven methods verifies its superior performance in detecting cyber attacks in unbalanced power distribution grids.

Reconstruction error‐based fault detection of time series process data using generative adversarial auto‐encoders

AbstractFaults in time series process data are typically difficult to detect due to the complex temporal correlations of data samples. In this context, traditional unsupervised machine learning algorithms, such as principal component analysis, independent component analysis, and so forth, would yield only limited performance. Deep learning‐based methods have been employed in recent years to address these problems. Recently, generative adversarial networks have emerged as a promising generative modelling approach for learning data distributions. Inspired by the above, in this study, we present a novel reconstruction error‐based fault detection method for time series process data using generative adversarial auto‐encoder (GAAE) with Wasserstein loss, cycle consistency loss, and gradient penalty methods. The proposed method is designed to detect abnormal patterns in the time series data by training a GAAE to learn the underlying normal behaviour of the data. GAAEs help in effectively capturing the data's hidden distribution, and Wasserstein loss with gradient penalty is used to improve the accuracy of the latent space representation of the data, while the cycle consistency loss ensures consistency between the input and output data during the reconstruction process. Based on the extent of the reconstruction error metric of the GAAEs, we identify the potential faults in the process data stream. The proposed method is evaluated on two data process sets, namely, the Tennessee Eastman benchmark process dataset and the nuclear power flux real‐time dataset from a pressurized heavy water nuclear reactor, to validate the efficacy of the proposed approach.

Towards integrated and fine-grained traffic forecasting: A Spatio-Temporal Heterogeneous Graph Transformer approach

Fine-grained traffic forecasting is crucial for the management of urban transportation systems. Road segments and intersection turns, as vital elements of road networks, exhibit heterogeneous spatial structures, yet their traffic states are interconnected due to spatial proximity. The heterogeneity and interrelationships arising from different road network elements pose major challenges to accurate traffic forecasting. However, existing forecasting studies focus solely on bidirectional road segments, disregarding the relationships between roads and turns. To achieve integrated traffic forecasting that considers both road segments and intersection turns, we propose a novel Spatio-Temporal Heterogeneous Graph Transformer (STHGFormer). For road network representation, we innovatively define a Heterogeneous Road network Graph (HRG), which provides a comprehensive depiction of the complete traffic network and emphasizes its inherent heterogeneity. Then, we propose a Heterogeneous Spatial Embedding (HSE) module to encode road network information, including heterogeneous attributes and interactions in the HRG. Based on the spatial information encoded by HSE, a unified SpaFormer, serving as the spatial module of STHGFormer, captures the interdependencies between roads and turns across the entire traffic network. To mitigate the impact of high temporal fluctuation, we embed the Adaptive Soft Threshold (AST) module into TempFormer, which dynamically adjusts the threshold to enhance the analysis capability of complex temporal correlations. Experiments conducted on a real-world dataset from Wuhan, China, demonstrate that STHGFormer outperforms state-of-the-art methods, achieving a 6.1 % improvement in road forecasting and an 8.5 % improvement in turn forecasting.

Reducing Mobile Web Latency Through Adaptively Selecting Transport Protocol

To improve the performance of mobile web services, a new transport protocol, QUIC, has been recently proposed as a substitute for TCP. However, with pros and cons of QUIC, it is challenging to decide whether and when to use QUIC in large-scale real-world mobile web services. Complex temporal correlation of network conditions, high user heterogeneity in a nationwide deployment, implementation diversity of QUIC variants limited, and resources on mobile devices all affect the selection of transport protocols. In this paper, we present WiseTrans, an adaptive transport protocol selection mechanism, to switch transport protocols for mobile web services online and improve the completion time of web requests. WiseTrans introduces machine learning techniques to deal with temporal heterogeneity, makes decisions with historical information to handle spatial heterogeneity, adopts an online learning method to keep pace with implementation variation, and switches transport protocols at the request level to reach high performance with acceptable overhead. We implement WiseTrans on two platforms (Android and iOS) in a popular mobile web service application of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Baidu</i> . Comprehensive experiments demonstrate that WiseTrans can reduce request completion time by up to 25.8% on average compared to the usage of a single protocol.

A data-driven model for production prediction of strongly heterogeneous reservoir under uncertainty

Production forecasting plays an important role in the reservoir development process. This task is challenging due to complex spatial and temporal correlations (e.g., inter-well interference and forward-backward linkages temporally). Most existing work tried to solve this problem by mining the impact of historical production conditions on the future production from the perspective of time series analysis. While such works gain satisfactory prediction accuracy, they cannot take the effect of well interference into account. In this study, we revisit the problem of production prediction from strongly heterogeneous reservoirs, aiming to improve the prediction accuracy by leveraging the breakthroughs of deep learning techniques. A new data-driven model is proposed to predict the production of each oil well, where the key novelty is to mine the inter-well interference relationships from the graph perspective. First, based on four constructed inter-well relationship graphs (well pattern structure graph, distance graph, similarity graph, and injection-production relationship graph), a Multi-Graph Convolutional Network (MGCN) is built to capture the spatial interactions of the wells. Then, a Long Short-Term Memory (LSTM) network is used to capture the complex temporal correlations. Finally, to make the model consider more production influencing factors, a Fully Connected Network (FCN) is utilized to fuse spatiotemporal correlations with other production influencing factors to predict production. Besides, considering the uncertainty of the data, an uncertainty evaluation method for predicting production is developed using Latin hypercube sampling and Monte Carlo methods. For evaluation and verification, the proposed model is applied to two case studies of numerical models and actual oilfields. The data of the actual oilfield are preprocessed by the proposed ensemble empirical mode decomposition (EEMD) component threshold denoising method. The optimal hyperparameters in the model are determined by Bayesian optimization algorithm. A series of experiments demonstrated that the proposed method has minimal prediction error and can predict the production with different probabilities, which can reduce the development risk.

Disentangled Dynamic Deviation Transformer Networks for Multivariate Time Series Anomaly Detection.

Graph neural networks have been widely used by multivariate time series-based anomaly detection algorithms to model the dependencies of system sensors. Previous studies have focused on learning the fixed dependency patterns between sensors. However, they ignore that the inter-sensor and temporal dependencies of time series are highly nonlinear and dynamic, leading to inevitable false alarms. In this paper, we propose a novel disentangled dynamic deviation transformer network (D3TN) for anomaly detection of multivariate time series, which jointly exploits multiscale dynamic inter-sensor dependencies and long-term temporal dependencies to improve the accuracy of multivariate time series prediction. Specifically, to disentangle the multiscale graph convolution, we design a novel disentangled multiscale aggregation scheme to better represent the hidden dependencies between sensors to learn fixed inter-sensor dependencies based on static topology. To capture dynamic inter-sensor dependencies determined by real-time monitoring situations and unexpected anomalies, we introduce a self-attention mechanism to model dynamic directed interactions in various potential subspaces influenced by various factors. In addition, complex temporal correlations across multiple time steps are simulated by processing the time series in parallel. Experiments on three real datasets show that the proposed D3TN significantly outperforms the state-of-the-art methods.

Multi-Channel Opportunistic Access for Heterogeneous Networks Based on Deep Reinforcement Learning

This paper investigates a new medium access control (MAC) protocol for multi-channel heterogeneous networks (HetNets) based on deep reinforcement learning (DRL), referred to as multi-channel deep-reinforcement learning multiple access (MC-DLMA). Specifically, we consider a HetNet where different radio networks adopt different MAC protocols to transmit data packets to a common access point on different wireless channels. Three key challenges for the MC-DLMA node are (i) no environmental knowledge is known in advance; (ii) the channels in HetNets are allocated to nodes using different MAC protocols; (iii) the capacities of different channels may be different. The main goal of MC-DLMA is to find an optimal access policy to transmit on those pre-allocated channels and expedite more efficient spectrum utilization. Due to the complex temporal correlation of spectrum states in HetNets, the traditional DRL technique, e.g., original deep Q-network (DQN) algorithm, is no longer applicable to our problem. In our MC-DLMA design, an advanced class of recurrent neural network, termed as Gated Recurrent Unit (GRU), is embedded into the original DQN technique to aggregate observations over time and reason the underlying temporal feature in multi-channel HetNets. Furthermore, we analytically give the optimal spectrum access patterns and derive the optimal throughputs in various HetNet scenarios. With judicious definitions of the state, action, and reward function in the parlance of the DRL framework, simulation results show that MC-DLMA can (i) find the optimal spectrum access strategies in various HetNets, (ii) outperform the random access policy, the whittle index policy, and the original DQN, (iii) perform cooperative transmission in a fully distributed manner in the presence of multiple agents, and (iv) adapt well to the environmental changes.

A Spatio-Temporal Multiscale Neural Network Approach for Wind Turbine Fault Diagnosis With Imbalanced SCADA Data

The supervisory control and data acquisition (SCADA) systems are widely installed on wind turbines (WTs) in major wind farms, which produce a large amount of sensory data that can be used for fault diagnosis of WTs. However, these SCADA data are naturally multivariate time series, which represent complex temporal correlations within each sensor variable and spatial correlations between different sensor variables. To effectively capture spatio-temporal correlations in SCADA data, we propose a new spatio-temporal multiscale neural network (STMNN). The proposed STMNN model contains two parallel feature extraction modules: first, a multiscale deep echo state network module to extract temporal multiscale features; second, a multiscale residual network module to extract spatial multiscale features. Additionally, as the WTs are in a normal working state most of the time, there are a large amount of normal SCADA data and few failure SCADA data. To address the data imbalance problem of SCADA data and enhance the fault diagnosis performance, instead of cross-entropy loss, the STMNN model adopts focal loss as loss function. Our proposed STMNN method can provide an end-to-end fault diagnosis solution with imbalanced SCADA data, and is evaluated through experiments on an SCADA dataset from a real wind farm. The experimental results and comparative analysis have proved the effectiveness of our proposed STMNN model in practical applications.

ADST: Forecasting Metro Flow Using Attention-Based Deep Spatial-Temporal Networks with Multi-Task Learning

Passenger flow prediction has drawn increasing attention in the deep learning research field due to its great importance in traffic management and public safety. The major challenge of this essential task lies in multiple spatiotemporal correlations that exhibit complex non-linear correlations. Although both the spatial and temporal perspectives have been considered in modeling, most existing works have ignored complex temporal correlations or underlying spatial similarity. In this paper, we identify the unique spatiotemporal correlation of urban metro flow, and propose an attention-based deep spatiotemporal network with multi-task learning (ADST-Net) at a citywide level to predict the future flow from historical observations. ADST-Net uses three independent channels with the same structure to model the recent, daily-periodic and weekly-periodic complicated spatiotemporal correlations, respectively. Specifically, each channel uses the framework of residual networks, the rectified block and the multi-scale convolutions to mine spatiotemporal correlations. The residual networks can effectively overcome the gradient vanishing problem. The rectified block adopts an attentional mechanism to automatically reweigh measurements at different time intervals, and the multi-scale convolutions are used to extract explicit spatial relationships. ADST-Net also introduces an external embedding mechanism to extract the influence of external factors on flow prediction, such as weather conditions. Furthermore, we enforce multi-task learning to utilize transition passenger flow volume prediction as an auxiliary task during the training process for generalization. Through this model, we can not only capture the steady trend, but also the sudden changes of passenger flow. Extensive experimental results on two real-world traffic flow datasets demonstrate the obvious improvement and superior performance of our proposed algorithm compared with state-of-the-art baselines.

Correspondence between temporal correlations in time series, inverse problems, and the spherical model.

In this paper we employ methods from statistical mechanics to model temporal correlations in time series. We put forward a methodology based on the maximum entropy principle to generate ensembles of time series constrained to preserve part of the temporal structure of an empirical time series of interest. We show that a constraint on the lag-one autocorrelation can be fully handled analytically and corresponds to the well-known spherical model of a ferromagnet. We then extend such a model to include constraints on more complex temporal correlations by means of perturbation theory, showing that this leads to substantial improvements in capturing the lag-one autocorrelation in the variance. We apply our approach on synthetic data and illustrate how it can be used to formulate expectations on the future values of a data-generating process.

Universal temporal characteristics and vanishing of multifractality in Barkhausen avalanches.

Barkhausen effect in ferromagnetic materials provides an excellent area for investigating scaling phenomena found in disordered systems exhibiting crackling noise. The critical dynamics is characterized by random pulses or avalanches with scale-invariant properties, power-law distributions, and universal features. However, the traditional Barkhausen avalanches statistics may not be sufficient to fully characterize the complex temporal correlation of the magnetic domain walls dynamics. Here we focus on the multifractal scenario to quantify the temporal scaling characteristics of Barkhausen avalanches in polycrystalline and amorphous ferromagnetic films with thicknesses from 50 to 1000 nm. We show that the multifractal properties are dependent on film thickness, although they seem to be insensitive to the structural character of the materials. Moreover, we observe for the first time the vanishing of the multifractality in the domain walls dynamics. As the thickness is reduced, the multifractal behavior gives place to a monofractal one over the entire range of time scales. This reorganization in the temporal scaling characteristics of Barkhausen avalanches is understood as a universal restructuring associated to the dimensional crossover, from three- to two-dimensional magnetization dynamics.

Generation of Self-Similar Gaussian Time Series by Means of the DWT and DWPT Variance Maps

Due to the several kinds of services that use the Internet and data networks infra-structures, the present networks are characterized by the diversity of types of traffic that have statistical properties as complex temporal correlation and nongaussian distribution. The networks complex temporal correlation may be characterized by the Short Range Dependence - (SRD) and the Long Range Dependence - (LRD). Models as the fGN (Fractional Gaussian Noise) may capture the LRD but not the SRD. This work presents two methods for traffic generation that synthesize approximate realizations of the self-similar fGN with SRD random process. The first one employs the IDWT (Inverse Discrete Wavelet Transform) and the second the IDWPT (Inverse Discrete Wavelet Packet Transform). It has been developed the variance map concept that allows to associate the LRD and SRD behaviors directly to the wavelet transform coefficients. The developed methods are extremely flexible and allow the generation of Gaussian time series with complex statistical behaviors.

Simulation of birdfoot delta formation with application to the Mississippi Delta

Recently, Seybold et al. (2007) proposed a reduced complexity model which simulates the process of delta formation on geological time scales. It includes subaerial and subaqueous growth in a three‐dimensional framework. In this paper we apply this model to the formation of a river‐dominated delta and compare the model dynamics with observations of the formation of the Balize Lobe of the Mississippi River Delta. The model generates both subaerial and subaqueous channels and lateral levee formations as well as a profile morphology with steep drop‐offs and a flat delta surface which is similar to natural ones. We show that the dimensionless parameters of the model may be consistently rescaled to match the Balize Lobe. This means that after rescaling the water flows, the subaerial geometry and time, the deposited (subaqueous) lobe volume, the sediment and water flows, the age, as well as the sediment capture ratio match the observed data. Finally, we use detrended fluctuation analysis to show that the modeled long‐term dynamics of the delta formation process shows a complex temporal correlation structure. A characteristic time scale separates periods of consistent delta growth by gradual sediment deposition at the mouths of distributary channels from periods during which random large‐scale channel avulsions lead to rapid change and the formation of new channels and subaqueous‐dominated deposition.

DDoS Attack Detection and Wavelets

This paper presents a systematic method for DDoS attack detection. DDoS attack can be considered a system anomaly or misuse from which abnormal behavior is imposed on network traffic. Attack detection can be performed via abnormal behavior identification. Network traffic characterization with behavior modeling could be a good indication of attack detection. Aggregated traffic has been found to be strong bursty across a wide range of time scales. Wavelet analysis is able to capture complex temporal correlation across multiple time scales with very low computational complexity. We utilize energy distribution based on wavelet analysis to detect DDoS attack traffic. Energy distribution over time will have limited variation if the traffic keeps its behavior over time (i.e. attack-free situation) while an introduction of attack traffic in the network will elicit significant energy distribution deviation in a short time period. Our experimental results with typical Internet traffic trace show that energy distribution variance markedly changes, causing a "spike" when traffic behaviors are affected by DDoS attack. In contrast, normal traffic exhibits a remarkably stationary energy distribution. In addition, this spike in energy distribution variance can be captured in the early stages of an attack, far ahead of congestion build-up, making it an effective detection of the attack.

Modeling heterogeneous network traffic in wavelet domain

Heterogeneous network traffic possesses diverse statistical properties which include complex temporal correlation and non-Gaussian distributions. A challenge to modeling heterogeneous traffic is to develop a traffic model which can accurately characterize these statistical properties, which is computationally efficient, and which is feasible for analysis. This work develops wavelet traffic models for tackling these issues. In specific, we model the wavelet coefficients rather than the original traffic. Our approach is motivated by a discovery that although heterogeneous network traffic has the complicated short- and long-range temporal dependence, the corresponding wavelet coefficients are all dependent. Therefore, a simple wavelet model may be able to accurately characterize complex network traffic. We first investigate what short-range dependence is important among wavelet coefficients. We then develop the simplest wavelet model, i.e., the independent wavelet model for Gaussian traffic. We define and evaluate the (average) autocorrelation function and the buffer loss probability of the independent wavelet model for Fractional Gaussian Noise (FGN) traffic. This assesses the performance of the independent wavelet model, and the use of which for analysis. We also develop (low-order) Markov wavelet models to capture additional dependence among wavelet coefficients. We show that an independent wavelet model is sufficiently accurate, and a Markov wavelet model only improves the performance marginally. We further extend the wavelet models to non-Gaussian traffic through developing a novel time-scale shaping algorithm. The algorithm is tested using real network traffic and shown to outperform FARIMA in both efficiency and accuracy. Specifically, the wavelet models are parsimonious, and have the computation complexity O(N) in developing a model from a training sequence of length N, and O (M) in generating a synthetic traffic trace of length M.