Stationary Time Series Research Articles

Predictive Maintenance in IoT Devices using Time Series Analysis and Deep Learning

The pervasive integration of Internet of Things (IoT) devices across industries has ushered in a new era of data-driven operational efficiency. However, the reliability and uninterrupted functionality of these interconnected devices necessitate innovative approaches to maintenance. This research focuses on the development and implementation of a predictive maintenance framework for IoT devices, leveraging the synergies between Time Series Analysis (TSA) and Deep Learning (DL) techniques. The primary objective of this study is to enhance the accuracy and efficiency of predictive maintenance processes, ultimately minimizing downtime and optimizing resource utilization. The research methodology involves the collection of diverse data types from IoT devices, encompassing sensor readings, error logs, and historical maintenance records. A meticulous data preprocessing stage follows, involving cleaning, normalization, and feature extraction to prepare the dataset for analysis. The core analytical components of the proposed framework include Time Series Analysis for uncovering temporal patterns in the IoT data. Statistical methods and time series decomposition are applied to identify trends and seasonality, providing valuable insights into the device's performance over time. Concurrently, Deep Learning models, specifically recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), are employed to predict maintenance needs based on historical patterns. Results obtained from the application of the predictive maintenance framework to real-world IoT datasets demonstrate promising accuracy and efficiency in anticipating maintenance requirements. The paper identifies existing challenges in predictive maintenance for IoT devices and suggests future research directions. These include the exploration of edge computing, federated learning, and the integration of explainable AI to enhance model interpretability. In conclusion, the study underscores the significance of predictive maintenance in ensuring the reliability of IoT devices, offering a roadmap for industries seeking to harness the full potential of data analytics and artificial intelligence for operational excellence.

Weak convergence of the conditional U-statistics for locally stationary functional time series

Weak convergence of the conditional U-statistics for locally stationary functional time series

Novel statistical time series data augmentation and machine learning based classification of unobtrusive respiration data for respiration Digital Twin model

Digital Twin (DT), a concept of Healthcare (4.0), represents the subject’s biological properties and characteristics in a digital model. DT can help in monitoring respiratory failures, enabling timely interventions, personalized treatment plans to improve healthcare, and decision-support for healthcare professionals. Large-scale implementation of DT technology requires extensive patient data for accurate monitoring and decision-making with Machine Learning (ML) and Deep Learning (DL). Initial respiration data was collected unobtrusively with the ESP32 Wi-Fi Channel State Information (CSI) sensor. Due to limited respiration data availability, the paper proposes a novel statistical time series data augmentation method for generating larger synthetic respiration data. To ensure accuracy and validity in the augmentation method, correlation methods (Pearson, Spearman, and Kendall) are implemented to provide a comparative analysis of experimental and synthetic datasets. Data processing methodologies of denoising (smoothing and filtering) and dimensionality reduction with Principal Component Analysis (PCA) are implemented to estimate a patient’s Breaths Per Minute (BPM) from raw respiration sensor data and the synthetic version. The methodology provided the BPM estimation accuracy of 92.3% from raw respiration data. It was observed that out of 27 supervised classifications with k-fold cross-validation, the Bagged Tree ensemble algorithm provided the best ML-supervised classification. In the case of binary-class and multi-class, the Bagged Tree ensemble showed accuracies of 89.2% and 83.7% respectively with combined real and synthetic respiration dataset with the larger synthetic dataset. Overall, this provides a blueprint of methodologies for the development of the respiration DT model.

Active sensing ultrasonic guided wave-based damage diagnosis via stochastic stationary time-series models

In the context of acousto-ultrasonic guided wave-based structural health monitoring, a statistical damage detection and identification (collectively referred to as damage diagnosis) framework for metallic and composite materials is proposed. Stochastic stationary time-series autoregressive (AR) models are used to model the ultrasonic wave propagation between piezoelectric actuator-sensor pairs on structural components and enable the damage diagnosis process via the use of the AR estimated parameters and corresponding covariance matrices. The proposed method exploits guided wave signals including the reflection parts, and thus the extraction of the S0 and/or A0 modes is not necessary, while the statistical properties and variation of estimated model parameters with respect to damage intersecting and non-intersecting wave propagation paths are presented and assessed. To investigate the method’s performance and robustness, two variations are proposed based on the singular value decomposition and principal component analysis. The obtained modified AR parameter vectors are then used to estimate appropriate statistical characteristic quantities used to enable the damage detection and identification tasks. The diagnosis is based on properly defined statistical hypotheses decision-making schemes and predetermined type I error probabilities. The performance and applicability of the method are explored experimentally via a series of tests on aluminum and composite coupons under various damage scenarios for damage intersecting and non-intersecting paths. The results of the present study demonstrated the effectiveness and robustness of the proposed modeling and diagnostic framework for guided wave-based damage diagnosis that can be implemented in a potentially automated way.

Sea clutter radar target detector based on autoregressive sieve bootstrap

The autoregressive sieve (ARS) bootstrap method can capture the correlation of time series and expand the original data set to the required volume. In this study, we introduce the ARS bootstrap into the matrix constant false alarm rate (CFAR) method to counter the inability of the Monte Carlo methods to extract the detection threshold due to the limited number of pulses under a certain false alarm probability. We assume the echo data of each range cell in a coherent processing interval (CPI) to be a stationary time series, and regard target detection as a two-sample hypothesis testing problem. We propose a statistic to test the equivalence of the autocovariance corresponding to two potential time series by using the correlation between pulses in a short pulse dataset, and analyze the validity of the ARS bootstrap for this statistic. The statistic distribution is obtained according to the bootstrap procedure and the threshold is obtained under fixed false alarm probabilities. Numerical experiments on the simulated and real sea clutter data were conducted to compare the detection results of the partial matrix CFAR detectors for the short and long pulse cases. With the introduction of the bootstrap method, we do not need to model the clutter amplitude distribution and the related statistical characteristics during target detection. To improve the detection probability with low signal-to-clutter ratio (SCR), we propose the two-sample autocovariance vector (ACV) detector based on the proposed statistic and use the correlation properties between pulses. We observed a 28% increase in detection probability when the SCR was 0 dB compared to other matrix CFAR methods, such as detectors based on Logarithmic Euclidean (LE) distance, Kernel Function-based Symmetric Kullback-Leibler (KSKL) divergence, and Total Bregman divergence (TBD) for the short pulse case.

Stages and main tasks of the century-long control theory and system identification development

The article provides a review of the mathematical description of the dynamics of continuous and discrete linear stationary systems and objects, used at the development stage of the classical theory of automatic control in the form of mathematical models of the «input-output» type. The time and frequency characteristics of continuous and discrete control systems are described, typical links of stationary systems are considered, parametric discrete models of objects as part of typical digital control loops are presented. Stochastic discrete autoregressive models of stationary time series used to describe the dynamic objects in the synthesis of digital control systems are considered. A review of standard control laws for the implementation of continuous and discrete controllers has been completed. A method for synthesizing discrete controllers for multidimensional controlled objects with different, unknown and changing delays is considered, through which variable delays are compensated in the characteristic equation of a closed-loop control system. A common technique for synthesizing one-dimensional and multidimensional controllers for stochastic objects with delays based on ARMAX models is considered. An analysis of approaches to identifying delays in controlled objects is carried out and a method for identifying delays when using input-output models is considered, based on the calculation and comparison of impulse responses for extended and non-extended models of the controlled object. An analysis of the advantages and disadvantages of «input-output» type models is given, as well as the possibilities of their application for solving various classes of control theory problems.

Burn-in selection in simulating stationary time series

Many time series models are defined in a recursive manner, which prohibits exact simulations. In practice, one appeals to simulating a long time series and discarding a large number of initial simulated observations, known as the burn-in. For autoregressive models where the dependence decays exponentially fast, the choice of the burn-in is not critical. However, for long-memory time series where the dependence from the remote past is strong, it is not clear how to select the burn-in number. By combining several samplers with randomized burn-in numbers, a method for exactly simulating the expectation of a statistic computed from a time series is developed. Moreover, with some suitably chosen statistics, the exact simulation method can be applied to quantify the effect of burn-in numbers on the simulated sample. Simulation studies are conducted to provide some practical guidances for burn-in selections.

Reactivation of an old landslide in north–central Iran following reservoir impoundment: Results from multisensor satellite time-series analysis

Water impoundment combined with more frequent precipitation extremes due to climate change increases landslide hazards on the slopes surrounding dam reservoirs. In situ monitoring systems in these potential landslide-prone areas are often unavailable, making landslide failures challenging to forecast. This paper describes a multisensor and multivariate remote sensing approach using data from Envisat, Sentinel-1, Landsat and PlanetScope satellites to reconstruct the spatiotemporal evolution of the mechanism and causes of the March 2019 landslide failure backside of the dam reservoir in Hoseynabad-e Kalpush village, north–central Iran. Statistical analysis and time series clustering are performed to derive the main landslide kinematic features from multitemporal interferometric synthetic aperture radar (MT-InSAR) analysis. We also exploit GIS and wavelet analysis to correlate potential external driving factors with landslide kinematics. Envisat and Sentinel-1 MT-InSAR analyses revealed that a previously stable old landslide was reactivated following reservoir impoundment in early 2013. As the reservoir water level rose during the following years up to 34 m in 2019, the landslide displacement rate gradually increased from 3.5 cm/yr to 8.4 cm/yr, and the destabilization gradually propagated upslope. At this stage, seasonal precipitation effects were detected only in the vertical component, indicating swelling and shrinkage movements of the shallower soil layer. The reactivated landslide accelerated and catastrophically failed following the exceptional precipitation in early 2019, producing a horizontal shift of >40 m, detected with optical image digital correlation. In the aftermath, the landslide continued to move with a decreasing trend until final stabilization in October 2021. Our study demonstrates how combined observations derived from multisensor satellite remote sensing data can be used to assess landslide precursors and kinematics, as well as the influence of climatic and anthropogenic factors on the instability of slopes surrounding water reservoirs. This is especially relevant in data-scarce areas.

Forecasting performance of machine learning, time series, and hybrid methods for low‐ and high‐frequency time series

One of the main objectives of the time series analysis is forecasting, so both Machine Learning methods and statistical methods have been proposed in the literature. In this study, we compare the forecasting performance of some of these approaches. In addition to traditional forecasting methods, which are the Naive and Seasonal Naive Methods, S/ARIMA, Exponential Smoothing, TBATS, Bayesian Exponential Smoothing Models with Trend Modifications and STL Decomposition, the forecasts are also obtained using seven different machine learning methods, which are Random Forest, Support Vector Regression, XGBoosting, BNN, RNN, LSTM, and FFNN, and the hybridization of both statistical time series and machine learning methods. The data set is selected proportionally from various time domains in M4 Competition data set. Thereby, we aim to create a forecasting guide by considering different preprocessing approaches, methods, and data sets having various time domains. After the experiment, the performance and impact of all methods are discussed. Therefore, most of the best models are mainly selected from machine learning methods for forecasting. Moreover, the forecasting performance of the model is affected by both the time frequency and forecast horizon. Lastly, the study suggests that the hybrid approach is not always the best model for forecasting. Hence, this study provides guidelines to understand which method will perform better at different time series frequencies.

Kernel-based joint independence tests for multivariate stationary and non-stationary time series.

Multivariate time-series data that capture the temporal evolution of interconnected systems are ubiquitous in diverse areas. Understanding the complex relationships and potential dependencies among co-observed variables is crucial for the accurate statistical modelling and analysis of such systems. Here, we introduce kernel-based statistical tests of joint independence in multivariate time series by extending the d-variable Hilbert-Schmidt independence criterion to encompass both stationary and non-stationary processes, thus allowing broader real-world applications. By leveraging resampling techniques tailored for both single- and multiple-realization time series, we show how the method robustly uncovers significant higher-order dependencies in synthetic examples, including frequency mixing data and logic gates, as well as real-world climate, neuroscience and socio-economic data. Our method adds to the mathematical toolbox for the analysis of multivariate time series and can aid in uncovering high-order interactions in data.

Getting to grips with Resilience: Towards large-scale phenotyping of this complex trait

The capacity of animals to cope with environmental perturbations, hereafter called resilience, is an increasingly important trait. Resilience at the level of the animal is an emergent property of multiple underlying mechanisms (physiological, immunological, behavioral). This means that there is no direct measure of resilience, no easy key traits. It is a latent variable that may be inferred from multivariate measures. Further, the flexibility that resilience provides is evidenced in the rate of response to, and rate of recovery from, the environmental perturbation. Thus, it requires time-series measurements. The increasing availability of on-farm precision livestock technologies, that are capable of providing time-series measures of performance and of various physiological and health biomarkers, offer the opportunity to move toward large-scale phenotyping of resilience. There have been numerous studies putting forward methods to quantify resilience. These methods can be classified as being data-driven or concept driven. However, new candidate resilience proxies need to be validated. This is tricky to do as there is no direct measure of resilience, no easy gold standard measure. Per definition, good resilience will benefit the animal. Thus, the accumulated consequences of resilience can be used to evaluate resilience proxies. All other things being equal, it is expected that good resilience will be associated with a longer functional longevity (longevity adjusted for production level), with more reproductive cycles, and with fewer disease events. Recent examples of this approach of evaluating resilience proxies against the accumulated consequences of resilience are discussed. They show clearly that operational resilience proxies that are heritable and have been validated against the consequences of good resilience can be derived from on-farm time-series data. With the aim of deriving more nuanced phenotypes, there are an increasing number of studies that have taken up the challenge of attempting to statistically combine the information coming from multiple time-series measures. These studies show how multivariate time-series statistics can be used to derive more nuanced resilience phenotypes that capture some of the underlying mechanisms of resilience. In conclusion, the recent studies reviewed here have shown that operational and heritable resilience proxies exist, that they can form the basis for selection for resilience, and that more nuanced phenotypes are attainable which will allow selection for resilience to be tailored according to prevailing environmental challenge types.

Employing Deep-Learning Approach for the Early Detection of Mild Cognitive Impairment Transitions through the Analysis of Digital Biomarkers.

Mild cognitive impairment (MCI) is the precursor to the advanced stage of Alzheimer's disease (AD), and it is important to detect the transition to the MCI condition as early as possible. Trends in daily routines/activities provide a measurement of cognitive/functional status, particularly in older adults. In this study, activity data from longitudinal monitoring through in-home ambient sensors are leveraged in predicting the transition to the MCI stage at a future time point. The activity dataset from the Oregon Center for Aging and Technology (ORCATECH) includes measures representing various domains such as walk, sleep, etc. Each sensor-captured activity measure is constructed as a time series, and a variety of summary statistics is computed. The similarity between one individual's activity time series and that of the remaining individuals is also computed as distance measures. The long short-term memory (LSTM) recurrent neural network is trained with time series statistics and distance measures for the prediction modeling, and performance is evaluated by classification accuracy. The model outcomes are explained using the SHapley Additive exPlanations (SHAP) framework. LSTM model trained using the time series statistics and distance measures outperforms other modeling scenarios, including baseline classifiers, with an overall prediction accuracy of 83.84%. SHAP values reveal that sleep-related features contribute the most to the prediction of the cognitive stage at the future time point, and this aligns with the findings in the literature. Findings from this study not only demonstrate that a practical, less expensive, longitudinal monitoring of older adults' activity routines can benefit immensely in modeling AD progression but also unveil the most contributing features that are medically applicable and meaningful.

Causality in extremes of time series

Consider two stationary time series with heavy-tailed marginal distributions. We aim to detect whether they have a causal relation, that is, if a change in one causes a change in the other. Usual methods for causal discovery are not well suited if the causal mechanisms only appear during extreme events. We propose a framework to detect a causal structure from the extremes of time series, providing a new tool to extract causal information from extreme events. We introduce the causal tail coefficient for time series, which can identify asymmetrical causal relations between extreme events under certain assumptions. This method can handle nonlinear relations and latent variables. Moreover, we mention how our method can help estimate a typical time difference between extreme events. Our methodology is especially well suited for large sample sizes, and we show the performance on the simulations. Finally, we apply our method to real-world space-weather and hydro-meteorological datasets.

The Principles of Modelling Causal Relationships in Financial Time Series on the Example of Stock Indices

In the article, the authors consider the issues of modeling causal relationships using the example of financial time series. Many endogenous indicators with tens and hundreds of feedbacks characterise any complex dynamic system. The links between the elements of the system determine the impact on the result of the system through a variety of parameters that can lead to a distortion of the effect of managerial influence and, as a result, to a distortion of the system's efficiency as a whole. Modelling is an effective tool for determining the considered system parameters under the influence of external forces and restrictions on it.In the activity of the control system, the analytical tools of mathematical modelling based on the developed modelallow to see how the system will change over time and what factors have the most significant impact on it. The system is changeable in time, which causes the difference between the initial state and the state at a certain point. That is explained by causal relationships between the elements of the systems and the level of influence of external factors, which can be referred to as economic or market ones. The latter, in turn, can be analyzed and, as a result, predicted using mathematical methods. Speaking of external factors, many of them are not amenable to precise analysis, and examples of these are natural and ecological ones.In this article, the authors define the principles of modelling causal relationships in financial time series using the example of stock indices. In the course of the study, the unsteadiness of time series was verified by Dickey-Fuller tests, as well as using the KPSS test solution, which confirmed the unsteadiness of stock indices in the selected time period. The calculations were carried out using the modern statistical package Eviews.As a consequence, the relevance of this article is based on the search for an answer to the question of stationary financial time series.

Consumption–Production Profile Categorization in Energy Communities

Energy Transition is changing the renewable energy participation in new distributed generation systems like the Local Energy Markets. Due to its inherent intermittent and variable nature, forecasting production and consumption load profiles will be more challenging and demand more complex predictive models. This paper analyzes the production, consumption load profile, and storage headroom% of the Cornwall Local Energy Market, using advanced statistical time series methods to optimize the opportunity market the storage units provide. These models also help the Energy Community storage reserves to meet contract conditions with the Distribution Network Operator. With this more accurate and detailed knowledge, all sites from this Local Energy Market will benefit more from their installation by optimizing their energy consumption, production, and storage. This better accuracy will make the Local Energy Market more fluid and safer, creating a flexible system that will guarantee the technical quality of the product for the whole community. The training of several SARIMAX, Exponential Smoothing, and Temporal Causal models improved the fitness of consumption, production, and headroom% time series. These models properly decomposed the time series in trend, seasonality, and stochastic dynamic components that help us to understand how the Local Energy Market consumes, produces, and stores energy. The model design used all power flows and battery energy storage system state-of-charge site characteristics at daily and hourly granularity levels. All model building follows an analytical methodology detailed step by step. A benchmark between these sequence models and the incumbent forecasting models utilized by the Energy Community shows a better performance measured with model error reduction. The best models present mean squared error reduction between 88.89% and 99.93%, while the mean absolute error reduction goes from 65.73% to 97.08%. These predictive models built at different prediction scales will help the Energy Communities better contribute to the Network Management and optimize their energy and power management performance. In conclusion, the expected outcome of these implementations is a cost-optimal management of the Local Energy Market and its contribution to the needed new Flexibility Electricity System Scheme, extending the adoption of renewable energies.

Evaluating the impact of private land conservation with synthetic control design.

Programs to protect biodiversity on private land are increasingly being used worldwide. To understand the efficacy of such programs, it is important to determine their impact: the difference between the program's outcome and what would have happened without the program. Typically, these programs are evaluated by estimating the average program-level impact, which readily allows comparisons between programs or regions, but masks important heterogeneity in impact across the individual conservation interventions. We used synthetic control design, statistical matching, and time-series data to estimate the impact of individual protected areas over time and combined individual-level impacts to estimate program-level impact with a meta-analytic approach. We applied the method to private protected areas governed by conservation covenants (legally binding on-title agreements to protect biodiversity) in the Goldfields region of Victoria, Australia using woody vegetation cover as our outcome variable. We compared our results with traditional approaches to estimating program-level impact based on a subset of covenants that were the same age. Our results showed an overall program-level impact of a 0.3-0.8% increase in woody vegetation cover per year. However, there was significant heterogeneity in the temporal pattern of impact for individual covenants, ranging from -4 to +7% change in woody vegetation cover per year. Results of our approach were consistent with results based on traditional approaches to estimating program-level impact. Our study provides a transparent and robust workflow to estimate individual and program-level impacts of private protected areas.

On-Board Random Vibration-Based Robust Detection of Railway Hollow Worn Wheels under Varying Traveling Speeds

The problem of the prompt detection of early-stage hollow worn wheels in railway vehicles via on-board random vibration measurements under normal operation and varying speeds is investigated. This is achieved based on two unsupervised statistical time series (STS) methods which are founded on a multiple-model (MM) framework for the representation of healthy vehicle dynamics. The unsupervised MM power spectral density (U-MM-PSD) method employs Welch-based PSD estimates for wheel wear detection and the unsupervised MM autoregressive (U-MM-AR) method for the parameter vectors of multiple AR models. Both methods are assessed via two case studies using thousands of test cases. The first case study includes Monte Carlo simulations using a SIMPACK-based detailed railway vehicle model, while the second is based on field tests with an Athens Metro train. Wheel wear detection is pursued using lateral or vertical vibration signals from the bogie or the carbody of a trailed vehicle traveling with three different speeds (60, 70, 80 km/h) using wheels under healthy conditions or with early stage hollow wear. Both methods exhibit remarkable performance with the U-MM-AR method to achieve the best overall results, reaching correct detection rates of even 100% with false alarm rates below 5% based on a single accelerometer either on the carbody or bogie.

Smoothness of time series: a new approach to estimation

The assessment of the risk of occurrence of extreme phenomena is inherently linked to the theory of extreme values. In the context of a time series, the analysis of its trajectory toward a greater or lesser smoothness, i.e. presenting a lesser or greater propensity for oscillations, respectively, constitutes another contribution in the assessment of the risk associated with extreme observations. For example, a financial market index with successive oscillations between high and low values shows investors a more unstable and uncertain behavior. In stationary time series, the upper tail smoothness coefficient is described by the tail dependence coefficient, a well-known concept first introduced by Sibuya. This work focuses on an inferential analysis of the upper tail smoothness coefficient, based on subsampling techniques for time series. In particular, we propose an estimator with reduced bias. We also analyze the estimation of confidence intervals through a block bootstrap methodology and a test procedure to prior detect the presence or absence of smoothness. An application to real data is also presented.

ANALISIS PERAMALAN EKSPOR BESI ATAU BAJA DARI INDONESIA KE SINGAPURA DENGAN METODE TIME SERIES

Exporting iron or steel is a crucial sector for Indonesia's economy, and Singapore is one of its main trading partners in terms of exporting these products. In order to optimize the performance of iron or steel exports to Singapore, a deep understanding of the trends and patterns present in export data is necessary. Therefore, forecasting analysis using time series methods is highly relevant and required. The methodology used in this study is a descriptive-analytical method with data collection from secondary sources. The population consists of iron or steel exports from Indonesia to Singapore, with a sample comprising export data from 2017 to 2021, and the sample determination technique uses the purposive sampling method. The data analysis techniques used are descriptive statistical methods and time series forecasting. The research results indicate that the export of iron or steel from Indonesia to Singapore has shown a positive trend over the past five years, influenced by factors such as strong demand, resource availability, developing infrastructure, product quality, and so on. Based on the forecasting analysis, it is estimated that the demand for iron or steel from Indonesia to Singapore in the coming years will continue to increase. The findings of this research provide valuable insights for the players in the iron or steel industry in Indonesia, especially exporters and stakeholders in the supply chain. With accurate forecasting, production and delivery planning can be optimized, market trends can be identified, and demand fluctuations in Singapore can be anticipated.

Estimation of prediction error in time series

Summary The accurate estimation of prediction errors in time series is an important problem, which has immediate implications for the accuracy of prediction intervals as well as the quality of a number of widely used time series model selection criteria such as the Akaike information criterion. Except for simple cases, however, it is difficult or even impossible to obtain exact analytical expressions for one-step and multi-step predictions. This may be one of the reasons that, unlike in the independent case (see Efron, 2004), up to now there has been no fully established methodology for time series prediction error estimation. Starting from an approximation to the bias-variance decomposition of the squared prediction error, a method for accurate estimation of prediction errors in both univariate and multivariate stationary time series is developed in this article. In particular, several estimates are derived for a general class of predictors that includes most of the popular linear, nonlinear, parametric and nonparametric time series models used in practice, with causal invertible autoregressive moving average and nonparametric autoregressive processes discussed as lead examples. Simulations demonstrate that the proposed estimators perform quite well in finite samples. The estimates may also be used for model selection when the purpose of modelling is prediction.