Multivariate Series Research Articles

Change point detection in high dimensional covariance matrix using Pillai’s statistics

AbstractThis research proposes a method to test and estimate change points in the covariance structure of high-dimensional multivariate series data. Our method uses the trace of the beta matrix, known as Pillai’s statistics, to test the change in covariance matrix at each time point. We study the asymptotic normality of Pillai’s statistics for testing the equality of two covariance matrices when both sample size and dimension increase at the same rate. We test the existence of a single change point in a given time period using Cauchy combination test, the test using an weighted sum of Cauchy transformed p-values, and estimate the change point as the point whose statistic is the greatest. To test and estimate multiple change points, we use the idea of the wild binary segmentation and repeatedly apply the procedure for a single change point to each segmented period until no significant change point exists. We numerically provide the size and power of our method. We finally apply our procedure to finding abnormal behavior in the investment of a private equity fund.

Multi-Step Temperature Prognosis of Lithium-Ion Batteries for Real Electric Vehicles Based on a Novel Bidirectional Mamba Network and Sequence Adaptive Correlation

The battery systems of electric vehicles (EVs) are directly impacted by battery temperature in terms of thermal runaway and failure. However, uncertainty about thermal runaway, dynamic conditions, and a dearth of high-quality data sets make modeling and predicting nonlinear multiscale electrochemical systems challenging. In this work, a novel Mamba network architecture called BMPTtery (Bidirectional Mamba Predictive Battery Temperature Representation) is proposed to overcome these challenges. First, a two-step hybrid model of trajectory piecewise–polynomial regression and exponentially weighted moving average is created and used to an operational dataset of EVs in order to handle the problem of noisy and incomplete time-series data. Each time series is then individually labeled to learn the representation and adaptive correlation of the multivariate series to capture battery performance variations in complex dynamic operating environments. Next, a prediction method with multiple steps based on the bidirectional Mamba is suggested. When combined with a failure diagnosis approach, this scheme can accurately detect heat failures due to excessive temperature, rapid temperature rise, and significant temperature differences. The experimental results demonstrate that the technique can accurately detect battery failures on a dataset of real operational EVs and predict the battery temperature one minute ahead of time with an MRE of 0.273%.

Robformer: A robust decomposition transformer for long-term time series forecasting

Transformer-based forecasting methods have been widely applied to forecast long-term multivariate time series, which achieves significant improvements on extending the forecasting time. However, their performance can degenerate terribly when abrupt trend shift and seasonal fluctuation arise in long-term time series. Hence, we identify two bottlenecks of previous Transformers architecture: (1) the robustless decomposition module and (2) trend shifting problem. These result in a different distribution between the trend prediction and ground truth in the long-term multivariate series forecasting. Towards these bottlenecks, we design Robformer as a novel decomposition-based Transformer, which consists of three new inner module to enhance the predictability of Transformers. Concretely, we renew the decomposition module and add a seasonal component adjustment module to tackle the unstationarized series. Further, we propose a novel inner trend forecasting architecture inspired by polynomial fitting method, which outperforms previous design in accuracy and robustness. Our empirical studies show that Robformer can achieve 17% and 10% relative improvements than state-of-the-art Autoformer and FEDformer baselines under the fair long-term multivariate setting on six benchmarks, covering five mainstream time series forecasting applications: energy, economics, traffic, weather, and disease. The code will be released upon publication.

Graph spatiotemporal process for multivariate time series anomaly detection with missing values

The detection of anomalies in multivariate time series data is crucial for various practical applications, including smart power grids, traffic flow forecasting, and industrial process control. However, real-world time series data is usually not well-structured, posting significant challenges to existing approaches: (1) The existence of missing values in multivariate time series data along variable and time dimensions hinders the effective modeling of interwoven spatial and temporal dependencies, resulting in important patterns being overlooked during model training; (2) Anomaly scoring with irregularly-sampled observations is less explored, making it difficult to use existing detectors for multivariate series without fully-observed values. In this work, we introduce a novel framework called GST-Pro, which utilizes a graph spatiotemporal process and anomaly scorer to tackle the aforementioned challenges in detecting anomalies on irregularly-sampled multivariate time series. Our approach comprises two main components. First, we propose a graph spatiotemporal process based on neural controlled differential equations. This process enables effective modeling of multivariate time series from both spatial and temporal perspectives, even when the data contains missing values. Second, we present a novel distribution-based anomaly scoring mechanism that alleviates the reliance on complete uniform observations. By analyzing the predictions of the graph spatiotemporal process, our approach allows anomalies to be easily detected. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods, regardless of whether there are missing values present in the data. Our code is available: https://github.com/huankoh/GST-Pro.

Coupling correlation adaptive detrended analysis for multiple nonstationary series

The interaction among objects leads to the complexity observed in real-world systems. Investigating the coupling behavior between multiple variables provides an effective means of understanding the dynamic mechanisms within a complex system. In this paper, we introduce a novel method called coupling correlation adaptive detrended analysis (CCADA), which enables rigorous and robust assessment of the long-range coupling properties in nonstationary multivariate series resulting from complex systems. By combining adaptive fractal analysis and coupling correlation detrended analysis (CCDA), CCADA inherits the merits of CCDA in overcoming the limitations of existing methods that lead to spurious coupling correlations. Furthermore, CCADA offers improved accuracy in estimating the coupling exponent for both monofractal and multifractal systems. Extensive numerical tests and a real-world case study have confirmed the effectiveness and practicality of the proposed method. An in-depth discussion on the comparison of existing coupling correlation analysis methods demonstrates CCADA exhibits the highest levels of accuracy and robustness.

Adapt to small-scale and long-term time series forecasting with enhanced multidimensional correlation

Multivariate time series forecasting aims to predict time series data comprising several linked variables or characteristicsand is frequently used in stock forecasting, energy forecasting, etc. The tough task is to acquire furtherhistorical data to forecast future values while boosting the capacity to mine relationships between and within sequences. The existing methods neglect the weighted influence of various neighbors and relationships on the sequence instance, as well as the semantic information of the instance itself, making the inter-instance correlation measure inaccurate. Meanwhile, the length of the sequence input is limited and the variable features in the multivariate sequence are treated equally, short-term and multi-feature interference cannot be eliminated, causing the learnt time series features to deviate from the real features. In this work, we provide a Multivariate Time Series Forecasting model that emphasizes Relationships between and within sequences (MTSFR). Use the BERT model to characterize the text attributes of instances and construct basic semantic embeddings. Meanwhile, instance-level and relation-level attention are used to model topological relations among different instances. Computes the cross-correlation of multivariate sequences within instances and performs attention weighting for multivariate sequence encoding. And choose the Transformer model to realize the trend prediction of long-term multivariate series. Experimental results show that the F1 value of our approach achieves 68.50% and 74.66% under the CSI300 and S&P500 data sets respectively, both of which are superior to the SOTA technique. Furthermore, the model is suited for small-scale sequence relationship modeling and is effective at handling long-term multivariate sequence forecasting problems.

Data Analytics Enables Reconstruction of Missing Segments in Well Data History

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 208137, “Reconstruction of Missing Segments in Well Data History Using Data Analytics,” by Yuanjun Li, SPE, and Roland Horne, SPE, Stanford University, and Ahmed Alshmakhy, SPE, ADNOC, et al. The paper has not been peer reviewed. _ The problem of missing data is a common one in well-production records. An incomplete data set is commonly simplified by omitting all observations with missing values, which can lead to significant information loss. In the complete paper, the authors developed an efficient procedure that enables fast reconstruction of the entire production data set with multiple missing sections in different variables. Ultimately, the complete information can support the reservoir history-matching process and production allocation and can develop models for reservoir performance prediction. Introduction Missing values are a prevalent issue in pressure transient analysis. Their occurrence is often unavoidable and is a serious problem because many data-mining algorithms cannot work with data sets that are missing values. Sources of missing field data include operational problems, changes in operating conditions, and variable sampling frequencies. Imputation Technique of Missing Values Case Deletion. If a missing data problem exists in a multivariate series, the time series may not be evaluated appropriately. In time-series data, each record is unique. Thus, deleting a record would result in a series with gaps, which would be unusable for many analyses. Furthermore, the time-series plot would be truncated and riddled with flaws. Thus, to conduct a time-series analysis, those missing observations must be estimated and reinserted. When the number of missing values in the data set is considerable, eliminating the associated cases might result in the loss of vital and valuable information. Researchers have found that case elimination is not appropriate when the missing rate is greater than 3%. Physics-Based Imputation. Any physics-driven diagnostic or model is based on fundamental assumptions that may or may not be correct in all situations but are intended to represent reservoir behavior. While data-driven models do not make any assumptions about underlying physics, they rely on training data. Therefore, data analytics and machine learning can be an ideal substitute. Local Statistical Approach. Local statistical methods include the K-nearest-neighbor (KNN) imputation and local-least-square imputation, two of the most well-known algorithms. To impute the incomplete data, the KNN imputation employs a pairwise correlation between the target gene with missing values and the K closest reference genes. For missing-value imputation, the techniques in this local method use local similarity patterns in the data sets or a specific column of data.

Under-reported time-varying MINAR(1) process for modeling multivariate count series

A time-varying multivariate integer-valued autoregressive of order one (tvMINAR(1)) model is introduced for the non-stationary time series of correlated counts when under-reporting is likely present. A non-diagonal autoregression probability network is structured to preserve the cross-correlation of multivariate series, provide a necessary condition to ease model-fittings computations, and derive the full likelihood using the Viterbi algorithm. The motivating construction applies to fully under-reported counts that rely on a mixture presentation of the random thinning operator. Simulation studies are conducted to examine the proposed model, and the analysis of COVID-19 daily cases is accomplished to highlight its usefulness in applications. Finally, the comparison of models is presented using the posterior predictive checking method.

Modification of a Marine Pine Kraft Lignin Sample by Enzymatic Treatment with a Pycnoporus cinnabarinus Laccase.

Here, we report work on developing an enzymatic process to improve the functionalities of industrial lignin. A kraft lignin sample prepared from marine pine was treated with the high-redox-potential laccase from the basidiomycete fungus Pycnoporus cinnabarinus at three different concentrations and pH conditions, and with and without the chemical mediator 1-hydroxybenzotriazole (HBT). Laccase activity was tested in the presence and absence of kraft lignin. The optimum pH of PciLac was initially 4.0 in the presence and absence of lignin, but at incubation times over 6 h, higher activities were found at pH 4.5 in the presence of lignin. Structural changes in lignin were investigated by Fourier-transform infrared spectroscopy (FTIR) with differential scanning calorimetry (DSC), and solvent-extractable fractions were analyzed using high-performance size-exclusion chromatography (HPSEC) and gas chromatography-mass spectrometry (GC-MS). The FTIR spectral data were analyzed with two successive multivariate series using principal component analysis (PCA) and ANOVA statistical analysis to identify the best conditions for the largest range of chemical modifications. DSC combined with modulated DSC (MDSC) revealed that the greatest effect on glass transition temperature (Tg) was obtained at 130 U g cm-1 and pH 4.5, with the laccase alone or combined with HBT. HPSEC data suggested that the laccase treatments led to concomitant phenomena of oligomerization and depolymerization, and GC-MS revealed that the reactivity of the extractable phenolic monomers depended on the conditions tested. This study demonstrates that P. cinnabarinus laccase can be used to modify marine pine kraft lignin, and that the set of analytical methods implemented here provides a valuable tool for screening enzymatic treatment conditions.

Exploring Dynamic Structures in Matrix-Valued Time Series via Principal Component Analysis

Time-series data are widespread and have inspired numerous research works in machine learning and data analysis fields for the classification and clustering of temporal data. While there are several clustering methods for univariate time series and a few for multivariate series, most methods are based on distance and/or dissimilarity measures that do not fully utilize the time-dependency information inherent to time-series data. To highlight the main dynamic structure of a set of multivariate time series, this study extends the use of standard variance–covariance matrices in principal component analysis to cross-autocorrelation matrices at time lags k=1,2,…. This results in “principal component time series”. Simulations and a sign language dataset are used to demonstrate the effectiveness of the proposed method and its benefits in exploring the main structural features of multiple time series.

Laplace Homotopy Perturbation Method (Lhpm) For Solving Systems Of N-Dimensional Non-Linear Partial Differential Equation

In this research, we proposed coupling the Laplace transform method and the homotopy Perturbation Method (LHPM). We employed the fusion of the Laplace method to make up for the shortcomings of other semi-analytical approaches like the homotopy perturbation method, variation iteration method, and the Adomian decomposition method. We aim to obtain an approximate and semi-analytic solution of the n-dimensional system of nonlinear partial differential equations. N-dimensional partial differential equations with nonlinear terms are known as nonlinear partial differential equations. They have been used to solve mathematical problems like the Poincaré conjecture and the Calabi-Yau conjecture and describe physical systems, from gravity to fluid dynamics. Therefore, we proffer a semi-analytic solution in the form of a Taylor multivariate series of displacements x, y, and time t using the proposed method. A side-by-side comparison was carried out to compare the exact solution with the new solution using 3-dimensional graphs, and thus the graph analysis followed. Results show excellent agreement, and the emergence of this method as a viable alternative demonstrates its viability by requiring fewer computations and being much easier and more convenient than others, making it suitable for widespread use in engineering as well.

Multiple Mellin-Barnes integrals with straight contours

We show how the conic hull method, recently developed for the analytic and non-iterative evaluation of multifold Mellin-Barnes (MB) integrals, can be extended to the case where these integrals have straight contours of integration parallel to the imaginary axes in the complex planes of the integration variables. MB integrals of this class appear, for instance, when one computes the $\epsilon$-expansion of dimensionally regularized Feynman integrals, as a result of the application of one of the two main strategies (called A and B in the literature) used to resolve the singularities in $\epsilon$ of MB representations. We upgrade the Mathematica package MBConicHulls.wl which can now be used to obtain multivariable series representations of multifold MB integrals with arbitrary straight contours, providing an efficient tool for the automatic computation of such integrals. This new feature of the package is presented, along with an example of application by calculating the $\epsilon$-expansion of the dimensionally regularized massless one-loop pentagon integral in general kinematics and $D=4-2\epsilon$.

Spatial-temporal fusion network for maximum ground surface settlement prediction during tunnel excavation

The maximum ground surface settlement prediction is a complex problem as the settlement depends on plenty of intrinsic and extrinsic factors. To obtain the approximate range of the settlement, a hybrid prediction dataset including the geological and construction parameters is built using spatial and temporal series according to the sampling methods. The settlement prediction task is transformed into a multi-modal and multi-variate series prediction task. Hence, a spatial-temporal fusion network (STF-Network) is proposed. The spatial-temporal fusion mechanism is firstly designed to establish the spatial-temporal fusion map, which makes spatial and temporal series interact earlier. Then, the 3D residual unit structure is designed to capture the features of temporal series and spatial-temporal fusion map, and two fully-connected layers are established to capture the spatial structural information. Finally, the final output is merged by the three components. The experimental results for STF-Network demonstrate the superiority over state-of-the-art methods.

A Wind Speed Combination Forecasting Method Based on Multifaceted Feature Fusion and Transfer Learning for Centralized Control Center

• A novel combined model based on transfer learning is proposed for wind speed forecasting in different data features and scenarios. • MVMD is used to decompose the multivariate series with the intention of considering the internal coupling relationship between wind speed and meteorological series. • TDCN-BiLSTM model is established for alleviating the interference between data, extracting multi-scales characteristics. • The IMOGOA is developed to improve the optimization ability of the algorithm. With the establishment of remote centralized control centers for wind farms, the combined model of multiple deep neural networks used in most wind speed prediction methods can no longer meet the requirements of centralized control centers for efficient and low-cost wind speed prediction in different locations. Thus, a model based on multi-layer feature fusion and transfer learning is proposed in this paper for multi-location wind speed forecasting. Under the idea of "offline training and online prediction", the wind speed characteristics of wind farms at typical locations are learned in the offline stage by fusing multifaceted features through a two-channel dilated convolutional network and a bi-directional long short-term memory network, transferring the well-trained models to any target wind farm to achieve online prediction, and integrating the prediction results of each offline model with an improved multi-objective grasshopper optimization algorithm to further enhance the prediction accuracy. Finally, three comparisons reveal that the proposed model outperforms other baseline models in adaptability and accuracy.

Deep Predictive Models Based on IoT and Remote Sensing Big Time Series for Precision Agriculture

Managing time series data generated by the intelligent objects around us requires cleaning and processing techniques, as well as a prediction model with high accuracy and low complexity. This study attempts to fuse climate time series data (Temperature, Humidity, Wind speed and Rainfall) and remote sensing data to predict rice yield. We performed statistical analysis of the data, additive decomposition, and measured the correlation between the different variables. We checked the stationarity of the data by using the Augmented Dickey-Fuller ADF test to apply statistical prediction models based on AutoRegression and moving averages for univariate series such as ARIMA (Autoregressive Integrated Moving Average) and for multivariate series such as Vector AutoRegression VAR and Vector Autoregressive Moving Average VARMAX. In addition, we applied a non-parametric model such as KNearest Neighbors KNN and Recurrent Neural Network models such as Long short-term memory LSTM and Gated recurrent unit GRU for rice yield prediction. The best yield estimation is achieved using LSTM with a Root Mean Square Error RSME error of 0.100. Keywords— Agriculture, ARIMA ,KNN, LSTM, Time series.

D-finite multivariate series with arithmetic restrictions on their coefficients

AbstractA multivariate, formal power series over a field K is a Bézivin series if all of its coefficients can be expressed as a sum of at most r elements from a finitely generated subgroup $G \le K^*$ ; it is a Pólya series if one can take $r=1$ . We give explicit structural descriptions of D-finite Bézivin series and D-finite Pólya series over fields of characteristic $0$ , thus extending classical results of Pólya and Bézivin to the multivariate setting.

Structure of cross-correlation between stock and oil markets

We displayed in this paper the structure of cross-correlation between the S&P 500 stock market and the Brent Oil market and its evolutionary behavior. Technically, the ensemble empirical mode decomposition is adopted to separate the two series into components. Let a window slide along the multi-variate series of the components, generating a series of segments. For each segment, one calculates the mutual entropies between the components to describe the coupling strengths, resulting into a network between/within the two markets. The networks corresponding to the successive segments form a temporal network. It is found that the characteristic period of intrinsic mode for each series grows exponentially from several days to more than ten years. The couplings between long-term components (with periods larger than one year) form the stable backbone of the network. The shocks of short-term events on the long-term components determine mainly the evolutionary behavior, especially the changes of the coupling structure. This method can be extended straightforwardly to display the cross-correlation structures and their evolutions for complex systems composed of multi-subsystems.

Statistically validated hierarchical clustering: Nested partitions in hierarchical trees

We develop an algorithm that is fast and scalable in the detection of a nested partition extracted from a dendrogram that is obtained from hierarchical clustering of a multivariate series. Our algorithm provides a p-value for each clade observed in the hierarchical tree. The p-value is obtained by computing many bootstrap replicas of the dissimilarity matrix and by performing a statistical test on each difference between the dissimilarity associated with a given clade and the dissimilarity of the clade of its parent node. We prove the efficacy of our algorithm with a set of benchmarks generated by a hierarchically nested factor model. We compare results obtained by our algorithm with those of Pvclust. Pvclust is a widely-used algorithm pursuing a global approach originally developed in the context of phylogenetic studies. In our numerical experiments, we focus on the role of multiple hypothesis test correction and the robustness of the algorithms to inaccuracies and errors of datasets. We verify that our algorithm is much faster than Pvclust algorithm and has a better scalability both in the number of elements and in the number of records of the investigated multivariate set. We also apply our algorithm to two empirical datasets, one related to a biological complex system and the other related to financial time-series. We prove that the clusters detected by our methodology are meaningful with respect to some consensus partitioning of the two datasets.

Imputation by feature importance (IBFI): A methodology to envelop machine learning method for imputing missing patterns in time series data.

A new methodology, imputation by feature importance (IBFI), is studied that can be applied to any machine learning method to efficiently fill in any missing or irregularly sampled data. It applies to data missing completely at random (MCAR), missing not at random (MNAR), and missing at random (MAR). IBFI utilizes the feature importance and iteratively imputes missing values using any base learning algorithm. For this work, IBFI is tested on soil radon gas concentration (SRGC) data. XGBoost is used as the learning algorithm and missing data are simulated using R for different missingness scenarios. IBFI is based on the physically meaningful assumption that SRGC depends upon environmental parameters such as temperature and relative humidity. This assumption leads to a model obtained from the complete multivariate series where the controls are available by taking the attribute of interest as a response variable. IBFI is tested against other frequently used imputation methods, namely mean, median, mode, predictive mean matching (PMM), and hot-deck procedures. The performance of the different imputation methods was assessed using root mean squared error (RMSE), mean squared log error (MSLE), mean absolute percentage error (MAPE), percent bias (PB), and mean squared error (MSE) statistics. The imputation process requires more attention when multiple variables are missing in different samples, resulting in challenges to machine learning methods because some controls are missing. IBFI appears to have an advantage in such circumstances. For testing IBFI, Radon Time Series Data (RTS) has been used and data was collected from 1st March 2017 to the 11th of May 2018, including 4 seismic activities that have taken place during the data collection time.

Complexity-Entropy Causality Plane Analysis of Air Pollution Series

In this study, we explore the subtle temporal structure of environmental data using symbolic information-theory approach. The newly developed multivariate multiscale permutation entropy and complexity-entropy causality plane methodology are applied to the six pollutants data recorded in Beijing during 2013–2016, which is a powerful tool to discriminate nonlinear deterministic and stochastic dynamics. The obtained results showed that pollutant series exhibit significant randomness and a lower level of predictability in spring and summer, and more temporal correlations in winter and fall. In addition, surrogate analysis is implemented to avoid biased conclusion. We also define the relative complexity measure of multivariate series to reflect the complexity of a system. The highest relative complexity in winter is in line with the physical behavior of the pollution phenomenon.