Noisy Time Series Research Articles

IT2CFNN: An interval type-2 correlation-aware fuzzy neural network to construct non-separable fuzzy rules with uncertain and adaptive shapes for nonlinear function approximation

In this paper, a new interval type-2 fuzzy neural network able to construct non-separable fuzzy rules with various shapes is introduced for function approximation problems. To reflect the uncertainty, the shape of fuzzy sets is considered to be uncertain. Therefore, a new form of shapeable interval type-2 fuzzy sets based on a general Gaussian model able to construct different shapes (including triangular, bell-shaped, trapezoidal) is proposed. To consider the interactions among input variables, input vectors are transformed to new feature spaces with uncorrelated variables proper for defining each fuzzy rule. Next, the new features are fed to a fuzzification layer using proposed interval type-2 fuzzy sets with adaptive shapes. Consequently, interval type-2 non-separable fuzzy rules with proper shapes, considering the local interactions of variables and the uncertainty are formed. For type reduction, the contribution of the upper and lower firing strengths of each fuzzy rule is adaptively selected separately. To train different parameters of the network, the Levenberg–Marquardt optimization method is utilized. The performance of the proposed method is investigated on clean and noisy datasets to show the ability to consider the uncertainty. Moreover, the proposed paradigm is successfully applied to real-world time-series predictions, regression problems, and nonlinear system identification. According to the experimental results, the performance of our proposed model outperforms other methods with a more parsimonious structure. Based on several experiments, the test RMSE of the proposed method is equal to 0.0243 for noisy McGlass time series prediction, 1.92 for Santa-Fe Laser prediction, 0.0301 for Box–Jenkins system identification, 0.0569 for Poland electricity load forecasting, 4.22 for Google stock price tracking, and 13.22 for Sydney stock price tracking.

Optimal Estimate of Global Biome—Specific Parameter Settings to Reconstruct NDVI Time Series with the Harmonic ANalysis of Time Series (HANTS) Method

Terrestrial remote sensing data products retrieved from radiometric measurements in the optical and thermal infrared spectrum such as vegetation spectral indices can be heavily contaminated by atmospheric conditions, including cloud and aerosol layers. This contamination results in gaps or noisy observations. The harmonic analysis of time series (HANTS) has been widely used for time series reconstruction of remote sensing imagery in recent decades. To use HANTS model, a series of parameters, such as number of frequencies (NF), fitting error tolerance (FET), degree of over-determinedness (DoD), and regularization factor (Delta), need to be defined by users. These parameters provide flexibilities, but also make it difficult for non-expert users to determine appropriate settings for specific applications. This study systematically evaluated the reconstruction performance of the model under different parameter setting scenarios by simulating pixel-wise reference and noisy NDVI time series. The results of these numerical experiments were further used to identify optimal settings and improve global NDVI reconstruction performance. The results suggested optimal settings for different areas (local optimization). If a user opts to use unique settings for global reconstruction, the setting NF = 4, FET = 0.05, DoD = 5, and Delta = 0.5 can produce the best performance across all setting scenarios (global optimization). In addition, several internal improvements, such as dynamic weighting scheme, polynomial and inter-annual harmonic components, and ancillary attributes of input data can be used to further improve the performance of reconstruction. With these results, future non-expert users can easily determine appropriate settings of HANTS for specific applications in different regions.

Unsupervised selection of optimal single-molecule time series idealization criterion

Single-molecule (SM) approaches have provided valuable mechanistic information on many biophysical systems. As technological advances lead to ever-larger data sets, tools for rapid analysis and identification of molecules exhibiting the behavior of interest are increasingly important. In many cases the underlying mechanism is unknown, making unsupervised techniques desirable. The divisive segmentation and clustering (DISC) algorithm is one such unsupervised method that idealizes noisy SM time series much faster than computationally intensive approaches without sacrificing accuracy. However, DISC relies on a user-selected objective criterion (OC) to guide its estimation of the ideal time series. Here, we explore how different OCs affect DISC’s performance for data typical of SM fluorescence imaging experiments. We find that OCs differing in their penalty for model complexity each optimize DISC’s performance for time series with different properties such as signal/noise and number of sample points. Using a machine learning approach, we generate a decision boundary that allows unsupervised selection of OCs based on the input time series to maximize performance for different types of data. This is particularly relevant for SM fluorescence data sets, which often have signal/noise near the derived decision boundary and include time series of nonuniform length because of stochastic bleaching. Our approach, AutoDISC, allows unsupervised per-molecule optimization of DISC, which will substantially assist in the rapid analysis of high-throughput SM data sets with noisy samples and nonuniform time windows.

A Chaos Analysis of the Dry Bulk Shipping Market

Finding low-dimensional chaos is a relevant issue as it could allow short-term reliable forecasting. However, the existence of chaos in shipping freight rates remains an open and outstanding matter as previous research used methodology that can produce misleading results. Using daily data, this paper aims to unveil the nonlinear dynamics of the Baltic Dry Index that has been proposed as a measure of the shipping rates for certain raw materials. We tested for the existence of nonlinearity and low-dimensional chaos. We have also examined the chaotic dynamics throughout three sub-sampling periods, which have been determined by the volatility pattern of the series. For this purpose, from a comprehensive view we apply several metric and topological techniques, including the most suitable methods for noisy time series analysis. The proposed methodology considers the characteristics of chaotic time series, such as nonlinearity, determinism, sensitivity to initial conditions, fractal dimension and recurrence. Although there is strong evidence of a nonlinear structure, a chaotic and, therefore, deterministic behavior cannot be assumed during the whole or the three periods considered. Our findings indicate that the generalized autoregressive conditional heteroscedastic (GARCH) model and exponential GARCH (EGARCH) model explain a significant part of the nonlinear structure that is found in the dry bulk shipping freight market.

Abrupt climate change as a rate-dependent cascading tipping point

Abstract. We propose a conceptual model comprising a cascade of tipping points as a mechanism for past abrupt climate changes. In the model, changes in a control parameter, which could for instance be related to changes in the atmospheric circulation, induce sequential tipping of sea ice cover and the ocean's meridional overturning circulation. The ocean component, represented by the well-known Stommel box model, is shown to display so-called rate-induced tipping. Here, an abrupt resurgence of the overturning circulation is induced before a bifurcation point is reached due to the fast rate of change of the sea ice. Because of the multi-scale nature of the climate system, this type of tipping cascade may also be a risk concerning future global warming. The relatively short timescales involved make it challenging to detect these tipping points from observations. However, with our conceptual model we find that there can be a significant delay in the tipping because the system is attracted by the stable manifold of a saddle during the rate-induced transition before escaping towards the undesired state. This opens up the possibility for an early warning of the impending abrupt transition via detection of the changing linear stability in the vicinity of the saddle. To do so, we propose estimating the Jacobian from the noisy time series. This is shown to be a useful generic precursor to detect rate-induced tipping.

Closeness-centrality-correlation for detecting interdependency between coupled systems

Generalized synchronization is a common interdependency between coupled systems which exists in many branches of life, social and physical science. In this paper, a novel method, called closeness-centrality-correlation is proposed for the detection of this interdependency. The proposed method is based on a global network measure (i.e., closeness centrality) of recurrence networks resulting from time series. We illustrate the feasibility of the proposed method using a paradigmatic coupled model and compare its performance to other commonly used interdependency methods. The numerical results show that the proposed method is quite satisfactory for detecting interdependency and outperforms the existing joint probability of recurrence method especially for the case that the dynamics of the two coupled subsystems are significantly different. Moreover, through analyzing the time series contaminated by white noise, we demonstrate that our method is robust against white noise. Finally, an application to recorded electroencephalogram data shows that the proposed measure is more reliable to detect the transitions of the interdependencies among the noisy electroencephalogram time series and thus provides longer pre-warning time for the onset of epilepsy.

Weighted recurrence network for characterizing continuous dynamical systems

Recurrence network (RN) is a powerful tool for the analysis of complex dynamical systems. It integrates complex network theory with the idea of recurrence of a trajectory, i.e. whether two state vectors are close neighbors in a phase space. However, the differences in proximity between connected state vectors are not considered in the RN construction. Here, we propose a weighted state vector recurrence network method which assigns weights to network links based on the proximity of the two connected state vectors. On the basis, we further propose a weighted data segment recurrence network that takes continuous data segments as nodes for the analysis of noisy time series. The feasibility of the proposed methods is illustrated based on the Lorenz system. Finally, an application to five types of EEG recordings is conducted to demonstrate the potentials of the proposed methods in the study of real-world data.

Long-Term Data Traffic Forecasting for Network Dimensioning in LTE with Short Time Series

Network dimensioning is a critical task in current mobile networks, as any failure in this process leads to degraded user experience or unnecessary upgrades of network resources. For this purpose, radio planning tools often predict monthly busy-hour data traffic to detect capacity bottlenecks in advance. Supervised Learning (SL) arises as a promising solution to improve predictions obtained with legacy approaches. Previous works have shown that deep learning outperforms classical time series analysis when predicting data traffic in cellular networks in the short term (seconds/minutes) and medium term (hours/days) from long historical data series. However, long-term forecasting (several months horizon) performed in radio planning tools relies on short and noisy time series, thus requiring a separate analysis. In this work, we present the first study comparing SL and time series analysis approaches to predict monthly busy-hour data traffic on a cell basis in a live LTE network. To this end, an extensive dataset is collected, comprising data traffic per cell for a whole country during 30 months. The considered methods include Random Forest, different Neural Networks, Support Vector Regression, Seasonal Auto Regressive Integrated Moving Average and Additive Holt–Winters. Results show that SL models outperform time series approaches, while reducing data storage capacity requirements. More importantly, unlike in short-term and medium-term traffic forecasting, non-deep SL approaches are competitive with deep learning while being more computationally efficient.

Intelligent Monitoring and Forecasting Using Machine Learning Techniques

Stock market prediction is a crucial task and a prominent research area in the financial domain as investing in the stock market involves greater risk. Various events may affect public sentiments and emotions differently, which may affect the trend of stock market prices. Because of dependency on numerous factors, the stock prices are dynamic, and not static, highly noisy and nonlinear time series data. Due to its great learning ability, machine learning has been applied to this research area. The enormous stock market volatility emphasizes the need to effectively assess the role of external factors such as news in stock prediction. Machine learning algorithms have made a great impact in predicting stocks accurately. Methods based on learning for stock price prediction are found to be popular and a lot of strategies have been used to improve the performance of the learning-based predictors. However, performing successful predictions in the stock market is still a challenge. In this paper, we use machine learning algorithms and predict the stock market trade to examine if it increases or decreases. Compared with existing learning-based methods, the effectiveness of this new enhanced learning-based method is demonstrated by using a Naïve Bayes classifier which is found to be consistent and gives the maximum efficiency.

“A new feature-based time series classification method by using scale-space extrema”

Time series data mining has received significant attention over the past decade, and many approaches have focused on classification tasks where the goal is to define the label of a test time series, given labeled training data. Time series classification approaches can be broadly grouped into two categories as instance-based and feature-based methods. Instance-based approaches utilize similarity information in a nearest-neighbor setting to classify time series data. Although approaches from this category provide accurate results, their performance degrades with long and noisy time series. On the other hand, feature-based approaches extract features to deal with the limitations of instance-based approaches; however, these approaches work with predefined features and may not be successful in certain classification problems. This study proposes a time series classification approach that benefits from both scale-space theory and bag-of-features technique. The method starts with finding the scale-space extrema points (i.e. key points) of each time series according to the SiZer (SIgnificant ZERo crossings of the derivatives) method, and then proceeds to create the local features set around these points. After extraction of the local features from each key point, a bag-of-features representation for each time series is constructed as the summary of the key point characteristics. We evaluate the success of the proposed representation on time series classification problems from various domains. Our experimental results show that our proposal provides competitive results compared to widely used approaches in the literature.

Forecasting the COVID-19 Pandemic in Saudi Arabia Using a Modified Singular Spectrum Analysis Approach: Model Development and Data Analysis

BackgroundInfectious disease is one of the main issues that threatens human health worldwide. The 2019 outbreak of the new coronavirus SARS-CoV-2, which causes the disease COVID-19, has become a serious global pandemic. Many attempts have been made to forecast the spread of the disease using various methods, including time series models. Among the attempts to model the pandemic, to the best of our knowledge, no studies have used the singular spectrum analysis (SSA) technique to forecast confirmed cases.ObjectiveThe primary objective of this paper is to construct a reliable, robust, and interpretable model for describing, decomposing, and forecasting the number of confirmed cases of COVID-19 and predicting the peak of the pandemic in Saudi Arabia.MethodsA modified singular spectrum analysis (SSA) approach was applied for the analysis of the COVID-19 pandemic in Saudi Arabia. We proposed this approach and developed it in our previous studies regarding the separability and grouping steps in SSA, which play important roles in reconstruction and forecasting. The modified SSA approach mainly enables us to identify the number of interpretable components required for separability, signal extraction, and noise reduction. The approach was examined using different levels of simulated and real data with different structures and signal-to-noise ratios. In this study, we examined the capability of the approach to analyze COVID-19 data. We then used vector SSA to predict new data points and the peak of the pandemic in Saudi Arabia.ResultsIn the first stage, the confirmed daily cases on the first 42 days (March 02 to April 12, 2020) were used and analyzed to identify the value of the number of required eigenvalues (r) for separability between noise and signal. After obtaining the value of r, which was 2, and extracting the signals, vector SSA was used to predict and determine the pandemic peak. In the second stage, we updated the data and included 81 daily case values. We used the same window length and number of eigenvalues for reconstruction and forecasting of the points 90 days ahead. The results of both forecasting scenarios indicated that the peak would occur around the end of May or June 2020 and that the crisis would end between the end of June and the middle of August 2020, with a total number of infected people of approximately 330,000.ConclusionsOur results confirm the impressive performance of modified SSA in analyzing COVID-19 data and selecting the value of r for identifying the signal subspace from a noisy time series and then making a reliable prediction of daily confirmed cases using the vector SSA method.

Bayesian detection of event spreading pattern from multivariate binary time series

This paper concerns the inference of the event spreading path from noisy event occurrence time series. We propose a probabilistic tree model for the event spreading path, and provide a full Bayesian approach with a Metropolis-Hastings (MH) algorithm to detect the tree path from multivariate binary time series. Our approach allows a node to spread the event in a probabilistic way to an arbitrary number of downstream nodes, the input data to contain an unknown number of nodes which are not inside the event spreading path, and the algorithm to sample only tree spreading structures. Efficient MH moves and iterative procedures guarantee the detection even if none or multiple event spreading paths are presented in the data. Simulations on synthesized data sets showed that our method can accurately detect the event spreading topologies, has good one-step-ahead prediction accuracy and outperforms existing approaches. We also analyzed the influence of model parameters on estimation accuracy in depth and obtained some interesting conclusions. Furthermore, we applied our method to the PM2.5 data in provincial capitals of north and east China. The inferred path is largely consistent with the known pollutant spreading trend.

A robust-filtering method for noisy non-stationary multivariate time series with econometric applications

We investigate a new filtering method to estimate the hidden states of random variables for multiple non-stationary time series data. This helps in analyzing small sample non-stationary macro-economic time series in particular and it is based on the frequency domain application of the separating information maximum likelihood (SIML) method, developed by Kunitomo et al. (Separating Information Maximum Likelihood Estimation for High Frequency Financial Data. Springer, New York, 2018), and Kunitomo et al. (Japan J Statistics Data Sci 2:73–101, 2020), and Nishimura et al. (Asic-Pacific Financial Markets, 2019). We solve the filtering problem of hidden random variables of trend-cycle, seasonal and measurement-errors components, and propose a method to handle macro-economic time series. We develop the asymptotic theory based on the frequency domain analysis for non-stationary time series. We illustrate applications, including some properties of the method of Müller and Watson (Econometrica 86-3:775–804, 2018), and analyses of some macro-economic data in Japan.

Pattern discovery in time series using autoencoder in comparison to nonlearning approaches

In technical systems the analysis of similar situations is a promising technique to gain information about the system’s state, its health or wearing. Very often, situations cannot be defined but need to be discovered as recurrent patterns within time series data of the system under consideration. This paper addresses the assessment of different approaches to discover frequent variable-length patterns in time series. Because of the success of artificial neural networks (NN) in various research fields, a special issue of this work is the applicability of NNs to the problem of pattern discovery in time series. Therefore we applied and adapted a Convolutional Autoencoder and compared it to classical nonlearning approaches based on Dynamic Time Warping, based on time series discretization as well as based on the Matrix Profile. These nonlearning approaches have also been adapted, to fulfill our requirements like the discovery of potentially time scaled patterns from noisy time series. We showed the performance (quality, computing time, effort of parametrization) of those approaches in an extensive test with synthetic data sets. Additionally the transferability to other data sets is tested by using real life vehicle data. We demonstrated the ability of Convolutional Autoencoders to discover patterns in an unsupervised way. Furthermore the tests showed, that the Autoencoder is able to discover patterns with a similar quality like classical nonlearning approaches.

Experimental design in complex model formulation for lightning prediction

Space launch operations at Kennedy Space Center and Cape Canaveral Space Force Station (KSC/CCSFS) are complicated by unique requirements for near-real time determination of risk from lightning. Weather sensor networks for lightning forecasting produce data that are noisy, high volume, and high frequency time series for which traditional forecasting methods are often ill-suited. Current approaches result in significant residual uncertainties and consequentially may result in forecasting operational policies that are excessively conservative or inefficient. This work first proposes a forecasting methodology using wavelet decomposition of chaotic weather sensor time series and semiparametric single-index models to mitigate the chaotic signal and any possible distributional misspecification. Then, a screening experiment with augmentations is used to demonstrate how to explore the complex factor space of model parameters, guiding decisions regarding model formulation and gaining insight for follow-on research. Results indicate a promising technique for operationally relevant lightning prediction from chaotic sensor measurements.

A Comprehensive Framework for Uncovering Non-Linearity and Chaos in Financial Markets: Empirical Evidence for Four Major Stock Market Indices.

The presence of chaos in the financial markets has been the subject of a great number of studies, but the results have been contradictory and inconclusive. This research tests for the existence of nonlinear patterns and chaotic nature in four major stock market indices: namely Dow Jones Industrial Average, Ibex 35, Nasdaq-100 and Nikkei 225. To this end, a comprehensive framework has been adopted encompassing a wide range of techniques and the most suitable methods for the analysis of noisy time series. By using daily closing values from January 1992 to July 2013, this study employs twelve techniques and tools of which five are specific to detecting chaos. The findings show no clear evidence of chaos, suggesting that the behavior of financial markets is nonlinear and stochastic.

Complex interplay between spectral harmonicity and different types of cross-frequency couplings in nonlinear oscillators and biologically plausible neural network models.

Cross-frequency coupling (CFC) refers to the nonlinear interaction between oscillations in different frequency bands, and it is a rather ubiquitous phenomenon that has been observed in a variety of physical and biophysical systems. In particular, the coupling between the phase of slow oscillations and the amplitude of fast oscillations, referred as phase-amplitude coupling (PAC), has been intensively explored in the brain activity recorded from animals and humans. However, the interpretation of these CFC patterns remains challenging since harmonic spectral correlations characterizing nonsinusoidal oscillatory dynamics can act as a confounding factor. Specialized signal processing techniques are proposed to address the complex interplay between spectral harmonicity and different types of CFC, not restricted only to PAC. For this, we provide an in-depth characterization of the time locked index (TLI) as a tool aimed to efficiently quantify the harmonic content of noisy time series. It is shown that the proposed TLI measure is more robust and outperforms traditional phase coherence metrics (e.g., phase locking value, pairwise phase consistency) in several aspects. We found that a nonlinear oscillator under the effect of additive noise can produce spurious CFC with low spectral harmonic content. On the other hand, two coupled oscillatory dynamics with independent fundamental frequencies can produce true CFC with high spectral harmonic content via a rectification mechanism or other post-interaction nonlinear processing mechanisms. These results reveal a complex interplay between CFC and harmonicity emerging in the dynamics of biologically plausible neural network models and more generic nonlinear and parametric oscillators. We show that, contrary to what is usually assumed in the literature, the high harmonic content observed in nonsinusoidal oscillatory dynamics is neither a sufficient nor necessary condition to interpret the associated CFC patterns as epiphenomenal. There is mounting evidence suggesting that the combination of multimodal recordings, specialized signal processing techniques, and theoretical modeling is becoming a required step to completely understand CFC patterns observed in oscillatory rich dynamics of physical and biophysical systems.

An Empirical Study of Neural Networks for Trend Detection in Time Series

Detecting structure in noisy time series is a difficult task. One intuitive feature is the notion of trend. From theoretical hints and using simulated time series, we empirically investigate the efficiency of standard recurrent neural networks (RNNs) to detect trends. We show the overall superiority and versatility of certain standard RNNs structures over various other estimators. These RNNs could be used as basic blocks to build more complex time series trend estimators.

Self-organization maps (SOM) in the definition of a “transfer function” for a diatoms-based climate proxy

This study utilizes self-organizing maps (SOM) —a particular type of artificial neural network (ANN) — as a nonlinear analysis tool for identifying species that can be used as proxies for climate variables. Instrumental, environmental, and biological time-series data from the Alfonso Basin (Gulf of California) were analyzed. The analysis comprised three stages: (1) standardization and selection of biological databases; (2) elimination of noise in the time series data; (3) definition of an ANN architecture and assessment of differences between models built using either noisy or noiseless time series. Noise reduction improved the SOM outputs. A global structuring index was estimated to define the associative linkage between diatoms and climate variables. The results showed that diatoms identified are the most suitable proxies for reconstructing the main climate variables including precipitation, atmospheric temperature, and sea surface temperature. The method developed has the potential to improve the hierarchization and selection of variables that can be potentially reconstructed without a significant loss of climate information.

Of power and despair in cetacean conservation: estimation and detection of trend in abundance with noisy and short time-series.

Many conservation instruments rely on detecting and estimating a population decline in a target species to take action. Trend estimation is difficult because of small sample size and relatively large uncertainty in abundance/density estimates of many wild populations of animals. Focusing on cetaceans, we performed a prospective analysis to estimate power, type-I, sign (type-S) and magnitude (type-M) error rates of detecting a decline in short time-series of abundance estimates with different signal-to-noise ratio. We contrasted results from both unregularized (classical) and regularized approaches. The latter allows to incorporate prior information when estimating a trend. Power to detect a statistically significant estimates was in general lower than 80%, except for large declines. The unregularized approach (status quo) had inflated type-I error rates and gave biased (either over- or under-) estimates of a trend. The regularized approach with a weakly-informative prior offered the best trade-off in terms of bias, statistical power, type-I, type-S and type-M error rates and confidence interval coverage. To facilitate timely conservation decisions, we recommend to use the regularized approach with a weakly-informative prior in the detection and estimation of trend with short and noisy time-series of abundance estimates.