Periodic Time Series Research Articles

A novel multivariate grey model for forecasting periodic oscillation time series

To solve the problem that the grey multivariate prediction model cannot well simulate systems with periodic oscillations, a novel multivariate grey model called the GM(1,N|sin) power model is proposed. The power exponential term and dynamic sinusoidal function are developed to represent the nonlinear relationship and periodic oscillations of the independent and dependent variables in the proposed model, respectively. First, the discrete time response formula for generating the simulated values at each time point is given. Second, the nonlinear programming model based on the particle swarm optimization algorithm is established to solve the power exponential and periodic coefficients. In addition, to enhance the generalization ability of the parameters, an improved nonlinear programming model considering the accuracy of the validation set is constructed. Finally, in the case studies, the quarterly electricity consumption of Jiangsu Province and PM2.5 concentrations in Nanjing are adopted to test the effectiveness of this model, and the results are obtained respectively through the GM(1,N|sin) power model and alternative models. The results indicate that the accuracy of GM(1,N|sin) power model in the time series with periodic oscillations outperforms the other six models in this paper.

AC-LSTM: Anomaly State Perception of Infrared Point Targets Based on CNN+LSTM

Anomaly perception of infrared point targets has high application value in many fields, such as maritime surveillance, airspace surveillance, and early warning systems. This kind of abnormality includes the explosion of the target, the separation between stages, the disintegration caused by the abnormal strike, etc. By extracting the radiation characteristics of continuous frame targets, it is possible to analyze and warn the target state in time. Most anomaly detection methods adopt traditional outlier detection, which has the problems of poor accuracy and a high false alarm rate. Driven by data, this paper proposes a new network structure, called AC-LSTM, which combines Convolutional Neural Networks (CNN) with Long Short-Term Memory (LSTM), and embeds the Periodic Time Series Data Attention module (PTSA). The network can better extract the spatial and temporal characteristics of one-dimensional time series data, and the PTSA module can consider the periodic characteristics of the target in the process of continuous movement, and focus on abnormal data. In addition, this paper also proposes a new time series data enhancement method, which slices and re-amplifies the long time series data. This method significantly improves the accuracy of anomaly detection. Through a large number of experiments, AC-LSTM has achieved higher scores on our collected datasets than other methods.

A new distributional treatment for time series and an anomaly detection investigation

Time series is traditionally treated with two main approaches, i.e., the time domain approach and the frequency domain approach. These approaches must rely on a sliding window so that time-shift versions of a periodic subsequence can be measured to be similar. Coupled with the use of a root point-to-point measure, existing methods often have quadratic time complexity. We offer the third R domain approach. It begins with an insight that subsequences in a periodic time series can be treated as sets of independent and identically distributed (iid) points generated from an unknown distribution in R. This R domain treatment enables two new possibilities: (a) the similarity between two subsequences can be computed using a distributional measure such as Wasserstein distance (WD), kernel mean embedding or Isolation Distributional kernel (IDK); and (b) these distributional measures become non-sliding-window-based. Together, they offer an alternative that has more effective similarity measurements and runs significantly faster than the point-to-point and sliding-window-based measures. Our empirical evaluation shows that IDK and WD are effective distributional measures for time series; and IDK-based detectors have better detection accuracy than existing sliding-window-based detectors, and they run faster with linear time complexity.

Compression and regularized optimization of modules stacked residual deep fuzzy system with application to time series prediction

The double-input-rule-modules stacked deep fuzzy method (DIRM-DFM) has attracted much attention because of its interpretability and prediction accuracy. However, when confronted with high-dimensional data and large numbers of fuzzy rules, the original DIRM-DFM has three limitations. First, the identity mapping that appears in the depth layer causes degradation of the prediction performance. Second, redundant fuzzy rules increase the complexity of the structure. Third, the overfitting phenomenon in the parameter-learning process imposes restrictions on the generalization ability of the system. Therefore, achieving structural simplification and high-accuracy prediction remains challenging. In this paper, a compression and regularized optimization scheme for the modules stacked residual deep fuzzy system (CDIRM-RDFS) is presented to address the above-mentioned limitations. To overcome the first limitation, a residual approximation mechanism was developed to approximate the actual output layer by layer. To obtain more compact fuzzy modules, singular value decomposition was conducted to eliminate the unimportant fuzzy rules, thus surmounting the second limitation. Finally, to prevent overfitting, regularization terms combining the l1 norm with the l2 norm were added to the loss function to penalize the parameters in the learning process and thereby solve the third problem. The performance of the proposed CDIRM-RDFS was validated using various types of datasets, including seasonal and non-seasonal as well as cyclical and non-cyclical time series. Compared with some existing popular shallow and deep models, the experimental results indicate that the proposed methodology can simplify the system architecture while improving the accuracy, effectiveness, and robustness of the prediction model.

Fault Prediction of Rolling Element Bearings Using the Optimized MCKD–LSTM Model

The reliability and safety of rotating equipment depend on the performance of bearings. For complex systems with high reliability and safety needs, effectively predicting the fault data in the use stage has important guiding significance for reasonably formulating reliability plans and carrying out reliability maintenance activities. Many methods have been used to solve the problem of reliability prediction. Due to its convenience and efficiency, the data-driven method is increasingly widely used in practical reliability prediction. In order to ensure the reliability of bearing operation, the main objective of the present study is to establish a novel model based on the optimized maximum correlation kurtosis deconvolution (MCKD) and long short-term memory (LSTM) recurrent neural network to realize early bearing fault warnings by predicting bearing fault time series. The proposed model is based on the lifecycle vibration signal of the bearing. In the first step, the cuckoo search (CS) is utilized to optimize the parameter filter length and deconvolution period of MCKD, considering the influence of periodic bearing time series, and to improve the fault impact component of the optimized MCKD deconvolution time series. Then the LSTM learning rate is selected according to the deconvolution time series. Finally, the dataset obtained through various preprocessing approaches is used to train and predict the LSTM model. The analyses performed using the XJTU-SY bearing dataset demonstrate that the prediction results are in good consistency with real fault data, and the average prediction accuracy of the optimized MCKD–LSTM model is 26% higher than that of the original time series.

Spatiotemporal association of rapid urbanization and water-body distribution on hemorrhagic fever with renal syndrome: A case study in the city of Xi'an, China.

Hemorrhagic fever with renal syndrome (HFRS) is a zoonosis characterized by clinical features of high fever, hemorrhage, and renal damage. China has the largest number of HFRS cases worldwide, accounting for over 90% of the total reported cases. In this paper, we used surveyed HFRS data and satellite imagery to conduct geostatistical analysis for investigating the associations of rapid urbanization, water bodies, and other factors on the spatiotemporal dynamics of HFRS from year 2005 to 2018 in Xi’an City, Northwest China. The results revealed an evident epidemic aggregation in the incidence of HFRS within Xi’an City with a phenomenal fluctuation in periodic time series. Rapid urbanization was found to greatly affect the HFRS incidence in two different time phases. HFRS caused by urbanization influences farmers to a lesser extent than it does to non-farmers. The association of water bodies with the HFRS incidence rate was found to be higher within the radii of 696.15 m and 1575.39 m, which represented significant thresholds. The results also showed that geomatics approaches can be used for spatiotemporally investigating the HFRS dynamic characteristics and supporting effective allocations of resources to formulate strategies for preventing epidemics.

Identification of [formula omitted] oscillations with the 0-1 test for periodic and chaotic time-series: One- and two-parameter bifurcations

This paper examines the oscillatory responses (periodic and chaotic) of a biosystem store model for bursting and complex Ca2+ oscillations in which three compartments have been taken into consideration: the cytosol, endoplasmic reticulum (ER) and mitochondria. The oscillatory model is used to examine the reliability of the 0-1 test for chaos in the bifurcation analysis of continuous signals obtained when the frequencies of oscillatory responses vary significantly with a relatively small changes of the bifurcation parameters. The illustrative examples in both the one- and two-parameter cases are designed to show that for a periodic time-series the test’s reliability may be questioned when a periodic series is classified as a chaotic one - the ’false-positive’ case. To prevent the incorrect result an additional computational work is needed to examine the frequency spectrum of the periodic time-series. The illustrative examples utilize an autonomous dynamical model of cytosolic calcium oscillations with three dynamical variables and sixteen parameters. The dynamical model is such that the frequency of oscillations may change by the factor of about 200, when a certain dynamical system’s parameter changes from its minimum to maximum values, making selection of the parameters in the 0-1 test extremely difficult. The extra computational work improves the test’s reliability and eliminates the ’false-positive’ outcomes of the test. The paper is focused on the computational aspects of the 0-1 test for periodic and chaotic oscillations rather than on the properties of the store model for bursting and complex Ca2+ oscillations.

PAR(1) model analysis: a web-based shiny application for analysing periodic autoregressive models

In this paper, a web-based shiny application called the ‘PAR(1) Model Analysis’— that allows the modelling, estimation and prediction of a periodic autoregressive time series with scale mixtures of skew-normal innovations, a general and quite flexible class of error distributions—is presented. The class of scale mixtures of skew-normal distributions is often used for statistical procedures of analysing symmetrical and asymmetrical data. The formulation of the scale of a mixture of skew-normal periodic autoregressive models and the estimating methods, which will be applied in the web application, is briefly explained. The embedded tools in the web application and their applications on data analysis are fully described, and finally, a real data example is completely analysed by using this app to illustrate its handling.

Activation Function Modulation in Generative Triangular Recurrent Neural Networks

Autonomous generation of time series is challenging because the network must capture short-term features while tracking long-term time dependencies. This paper introduces the modulation of the activation function slopes of the upper-lower triangular recurrent neural networks (ULTRNNs) for dynamic variation of memory through a secondary recurrent network with its own independent states. A zigzag propagation algorithm for weight updates is proposed that accounts for the dynamic interaction of the states between the ULTRNN and the secondary network. A novel training method is proposed that distributes the eigenvalues of the closed-loop system around the unit circle in the complex z-plane to ensure that the network behaves as a nonlinear oscillator with an output that neither collapses nor saturates but continues to emulate the target. Examples encompassing the Lorenz series, Santa Fe laser data, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">kolam</i> patterns, electrocardiogram (ECG) signals, stock pricing data, and smart grid data are presented to demonstrate that the proposed approach is highly effective in the generative modeling of complex periodic, chaotic, and nonstationary time series. The qualitative and quantitative performance of the ULTRNN obtained with the proposed activation-function modulation technique is comparable to that of state-of-the-art techniques including feedforward networks and generative adversarial networks, but with far fewer trainable parameters and shorter computation times.

Asymptotic normality of simultaneous estimators of cyclic long-memory processes

Spectral singularities at non-zero frequencies play an important role in investigating cyclic or seasonal time series. The publication [2] introduced the generalized filtered method-of-moments approach to simultaneously estimate singularity location and long-memory parameters. This paper continues studies of these simultaneous estimators. A wide class of Gegenbauer-type semi-parametric models is considered. Asymptotic normality of several statistics of the cyclic and long-memory parameters is proved. New adjusted estimates are proposed and investigated. The theoretical findings are illustrated by numerical results. The methodology includes wavelet transformations as a particular case.

A Novel Model Based on DA-RNN Network and Skip Gated Recurrent Neural Network for Periodic Time Series Forecasting

Deep learning models are playing an increasingly important role in time series forecasting with their excellent predictive ability and the convenience of not requiring complex feature engineering. However, the existing deep learning models still have shortcomings in dealing with periodic and long-distance dependent sequences, which lead to unsatisfactory forecasting performance on this type of dataset. To handle these two issues better, this paper proposes a novel periodic time series forecasting model based on DA-RNN, called DA-SKIP. Using the idea of task decomposition, the novel model, based on DA-RNN, GRU-SKIP and autoregressive component, breaks down the prediction of periodic time series into three parts: linear forecasting, nonlinear forecasting and periodic forecasting. The results of the experiments on Solar Energy, Electricity Consumption and Air Quality datasets show that the proposed model outperforms the three comparison models in capturing periodicity and long-distance dependence features of sequences.

Visual Exploration of Anomalies in Cyclic Time Series Data with Matrix and Glyph Representations

The digitalization of manufacturing involves machines equipped with sensors that collect, produce, and exchange data machine-to-machine and machine-to-human in real-time. As the data generated within a production process can be massive and overwhelming for human users, support is needed to understand and explore this data, and drive decisions from it. First, the data has to be monitored and recorded using methods that can handle massive datasets. Next, the collected data has to be analyzed (often in real-time) to, e.g., (i) identify undetected process correlations, (ii) forecast the product quality, and (iii) perform root-cause analysis of failures or problems. The analysis becomes even more valuable when the production process is divided into repeating tasks, producing a vast amount of comparable data. For instance, in automotive durability tests, engineers investigate an engine's condition using multiple sensors, recording data from repeating test cycles. Tests can span dozens or hundreds of cycles, and thousands of runtime hours, making it difficult for engineers to collect and monitor each iteration's data to detect interesting data, such as anomalies. We propose an interactive visual analytics approach that displays the iterations of durability tests as a collection of color-encoded cycle glyphs to tackle this issue. With our approach, domain users including test engineers can readily monitor the test, detect potential anomalies, and intuitively analyze, report and document the detected anomalies. This research is conducted in close collaboration with our partner from the automotive sector and shows the effectiveness and efficiency of a prototype with a pair analytics evaluation study. We open up directions for future work, including a visual interactive labeling concept for anomaly classification.

Intelligent grey forecasting model based on periodic aggregation generating operator and its application in forecasting clean energy

AbstractAccurate prediction of long‐term and short‐term clean energy production is the basis for understanding short‐term clean energy supply capacity, long‐term clean energy development trend and evaluating the effect of energy policies. However, under the circumstances of the large time span, the insufficient data samples and the periodic characteristics of seasonal clean energy production make the traditional grey prediction model prone to produce forecasting deviations. Given this situation, a novel seasonal fractional‐order full‐order time power discrete grey prediction model is initially proposed to deal with long‐term clean energy production sequences featured with nonlinearity and periodicity. Based on the proposed model, we also propose a data‐based algorithm to select the model structure adaptively. To prove the practicability of the new model for nonlinear long‐term development trend, monthly periodic time series and quarterly periodic time series, this article uses the new model to predict annual hydropower capacity in North America, monthly natural gas production in China and quarterly solar power generation in China. And the prediction results are compared with the existing grey models and non‐grey prediction models. Different methods including GM (1,1), DGM (1,1), NGM (1,1), ARGM (1,1), ENGM (1,1), Verhulst, CCRGM (1,1), FOTP‐DGMr (1,1), PFSM (1,1), Holt‐winters model, SARIMA model, SGM, HP‐GM and DGGM are used as benchmarks. In experiments, the MAPE of the proposed model is 2.92%, 2.43%, and 7.87%, respectively. The results of empirical analysis indicate that the proposed model generally outperform the benchmark model as it can well capture nonlinear long‐term development trend and seasonal characteristics.

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks.

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice.

A Deep Learning and GIS Approach for the Optimal Positioning of Wave Energy Converters

Renewable Energy Sources provide a viable solution to the problem of ever-increasing climate change. For this reason, several countries focus on electricity production using alternative sources. In this paper, the optimal positioning of the installation of wave energy converters is examined taking into account geospatial and technical limitations. Geospatial constraints depend on Land Use classes and seagrass of the coastal areas, while technical limitations include meteorological conditions and the morphology of the seabed. Suitable installation areas are selected after the exclusion of points that do not meet the aforementioned restrictions. We implemented a Deep Neural Network that operates based on heterogeneous data fusion, in this case satellite images and time series of meteorological data. This fact implies the definition of a two-branches architecture. The branch that is trained with image data provides for the localization of dynamic geospatial classes in the potential installation area, whereas the second one is responsible for the classification of the region according to the potential wave energy using wave height and period time series. In making the final decision on the suitability of the potential area, a large number of static land use data play an important role. These data are combined with neural network predictions for the optimizing positioning of the Wave Energy Converters. For the sake of completeness and flexibility, a Multi-Task Neural Network is developed. This model, in addition to predicting the suitability of an area depending on seagrass patterns and wave energy, also predicts land use classes through Multi-Label classification process. The proposed methodology is applied in the marine area of the city of Sines, Portugal. The first neural network achieves 98.7% Binary Classification accuracy, while the Multi-Task Neural Network 97.5% in the same metric and 93.5% in the F1 score of the Multi-Label classification output.

New estimation method for periodic autoregressive time series of order 1 with additive noise

The periodic behavior of real data can be manifested in the time series or in its characteristics. One of the characteristics that often manifests the periodic behavior is the sample autocovariance function. In this case, the periodically correlated (PC) behavior is considered. One of the main models that exhibits PC property is the periodic autoregressive (PARMA) model that is considered as the generalization of the classical autoregressive moving average (ARMA) process. However, when one considers the real data, practically the observed trajectory corresponds to the “pure” model with the additional noise which is a result of the noise of the measurement device or other external forces. Thus, in this paper we consider the model that is a sum of the periodic autoregressive (PAR) time series and the additive noise with finite-variance distribution. We present the main properties of the considered model indicating its PC property. One of the main goals of this paper is to introduce the new estimation method for the considered model’s parameters. The novel algorithm takes under consideration the additive noise in the model and can be considered as the modification of the classical Yule–Walker algorithm that utilizes the autocovariance function. Here, we propose two versions of the new method, namely the classical and the robust ones. The effectiveness of the proposed methodology is verified by Monte Carlo simulations. The comparison with the classical Yule–Walker method is presented. The approach proposed in this paper is universal and can be applied to any finite-variance models with the additive noise.

A novel structural adaptive discrete grey prediction model and its application in forecasting renewable energy generation

The rapidly growing renewable energy generation instigates stochastic volatility of electricity supply that may compromise the power grid's stability and increase the grid imbalance cost. Therefore, accurate mid-to-long term renewable energy generation forecasting is of great significance for integrating renewable energy systems with smart grid and energy strategic planning. For this purpose, a new structural adaptive discrete grey prediction model is proposed. Overall, the proposed model possesses three main contributions. Firstly, the introduction of nonlinear term and periodic term strengthens the ability of the traditional DGM (1,1) model to capture the nonlinear and linear development trend of time series and improves the adaptability of the grey prediction model to arbitrary periodic time series. Secondly, the emerging coefficients are determined by the particle swarm optimization algorithm and hold-out cross-validation method, and the adaptive selection of the model structure is realized. From the perspective of expert system, it reduces the need for modeling knowledge. Thirdly, the consistency of stretching, unbiasedness, and compatibility with other grey models are discussed, which further verified the feasibility and practicability of the proposed model. Besides, the performance of the proposed model is compared with those of a series of grey prediction models and non-grey prediction methods to verify the feasibility and superiority of this new approach by three real cases. The results indicate that the proposed model benefits from its adaptive structure and produces reliable predictions.

Digital Surveillance Through an Online Decision Support Tool for COVID-19 Over One Year of the Pandemic in Italy: Observational Study.

BackgroundItaly has experienced severe consequences (ie, hospitalizations and deaths) during the COVID-19 pandemic. Online decision support systems (DSS) and self-triage applications have been used in several settings to supplement health authority recommendations to prevent and manage COVID-19. A digital Italian health tech startup, Paginemediche, developed a noncommercial, online DSS with a chat user interface to assist individuals in Italy manage their potential exposure to COVID-19 and interpret their symptoms since early in the pandemic.ObjectiveThis study aimed to compare the trend in online DSS sessions with that of COVID-19 cases reported by the national health surveillance system in Italy, from February 2020 to March 2021.MethodsWe compared the number of sessions by users with a COVID-19–positive contact and users with COVID-19–compatible symptoms with the number of cases reported by the national surveillance system. To calculate the distance between the time series, we used the dynamic time warping algorithm. We applied Symbolic Aggregate approXimation (SAX) encoding to the time series in 1-week periods. We calculated the Hamming distance between the SAX strings. We shifted time series of online DSS sessions 1 week ahead. We measured the improvement in Hamming distance to verify the hypothesis that online DSS sessions anticipate the trends in cases reported to the official surveillance system.ResultsWe analyzed 75,557 sessions in the online DSS; 65,207 were sessions by symptomatic users, while 19,062 were by contacts of individuals with COVID-19. The highest number of online DSS sessions was recorded early in the pandemic. Second and third peaks were observed in October 2020 and March 2021, respectively, preceding the surge in notified COVID-19 cases by approximately 1 week. The distance between sessions by users with COVID-19 contacts and reported cases calculated by dynamic time warping was 61.23; the distance between sessions by symptomatic users was 93.72. The time series of users with a COVID-19 contact was more consistent with the trend in confirmed cases. With the 1-week shift, the Hamming distance between the time series of sessions by users with a COVID-19 contact and reported cases improved from 0.49 to 0.46. We repeated the analysis, restricting the time window to between July 2020 and December 2020. The corresponding Hamming distance was 0.16 before and improved to 0.08 after the time shift.ConclusionsTemporal trends in the number of online COVID-19 DSS sessions may precede the trend in reported COVID-19 cases through traditional surveillance. The trends in sessions by users with a contact with COVID-19 may better predict reported cases of COVID-19 than sessions by symptomatic users. Data from online DSS may represent a useful supplement to traditional surveillance and support the identification of early warning signals in the COVID-19 pandemic.

Weighted multi-scale limited penetrable visibility graph for exploring atrial fibrillation rhythm

Atrial fibrillation (AF) is a common cause of serious diseases such as stroke, heart failure and coronary artery disease, and electrocardiogram (ECG) detection is an important means of identifying AF. However, the ECG signal is quite noisy, making it difficult to detect AF through this method. Removing noise interference in ECG signals is a challenging problem. Traditional methods usually adopt various filtering methods to tackle this problem. Inspired by complex network theory, in this paper we present an innovative denoising approach for ECG detection called weighted multi-scale limited penetrable visibility graph (WMS-LPVG), which allows us to detect the rhythms characterizing AF in noisy ECG signals. To our knowledge, this is the first model that represents the AF rhythm series from the perspective of multi-scale complex networks. Furthermore, our WMS-LPVG model characterizes the AF rhythms in more detail, enabling us to identify AF sufferers more accurately. To demonstrate the effectiveness of our WMS-LPVG method, we first propose a new concept, called local efficiency entropy (LEE), which is utilized to identify the dynamic characteristics of time series. We then study the LEE-fluctuation trend under different scale factors. The experimental results show that the proposed LEE criterion can identify four kinds of ECG waveforms at a large scale. We then fuse the extracted LEE features with the original sequential features of ECG signals to build a multi-model complex network and feed the fused features into an XGboost model to identify AF patients. To demonstrate the generality of our WMS-LPVG model, we construct complex networks with WMS-LPVG for periodic and chaotic time series, respectively, and further discuss their degree distributions. The results show that our WMS-LPVG method perfectly retains information about original sequences and offers good anti-noise ability.

Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning.

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, , of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, , generated with different values of . Then, the ordinal probabilities computed from the time series of interest, , are used as input features to the trained algorithm and that returns a value, , that contains meaningful information about the temporal correlations present in . We have also shown that the difference, , of the permutation entropy (PE) of the time series of interest, , and the PE of a FN time series generated with , , allows the identification of the underlying determinism in . Here, we apply our methodology to different datasets and analyze how and correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.