Procedures for outlier detection in circular time series models

This PhD dissertation deals with the world of multivariate time series models where the behaviour of the observed process is described by using a time-varying parameter. In particular, this thesis explore three different dynamic multivariate nonlinear models which are able to deal with multivariate time series gathered from heavy-tailed phenomena. Although the popularity of linear and univariate time series models, empirical evidences have shown that variables generated from complex phenomena are typically inter-related both contemporaneously and across time. This is the case for several fields of science such as economics, finance, biology or physics, where it is widely accepted that with a univariate approach it is difficult to obtain a satisfactory representation of the reality or to make good predictions about the future. For these reasons, the literature of linear multivariate Gaussian time series models has received increasing attention. However, these models are known for their unsatisfactory performances when the collected data are contaminated by outliers, yielding biased estimates and unreliable forecasts. In fact, when departure from the hypothesis of normality is confirmed by the observed data, it is reasonable to switch into the realm of nonlinear or non-Gaussian time series models. Unfortunately, despite the development of recent technologies, the estimation of nonlinear time series models might be really challenging, since they require simulation-based and computer-intensive methods. In addition, statistical properties of such estimators are not always easy to be derived. This thesis contributes to the literature by defining dynamic multivariate and heavy-tailed models that are relatively simple. The emphasis is models which are analytically tractable and can be easily estimated by means of maximum likelihood. For each of the models, a very detailed statistical and asymptotic analysis it is provided. Their practical usefulness is highlighted with several simulation studies and empirical applications.

A new comprehensive approach to nonlinear time series analysis and modeling is developed in the present paper. We introduce novel data-specific mid-distribution-based Legendre Polynomial (LP)-like nonlinear transformations of the original time series {Y(t)} that enable us to adapt all the existing stationary linear Gaussian time series modeling strategies and make them applicable to non-Gaussian and nonlinear processes in a robust fashion. The emphasis of the present paper is on empirical time series modeling via the algorithm LPTime. We demonstrate the effectiveness of our theoretical framework using daily S&P 500 return data between 2 January 1963 and 31 December 2009. Our proposed LPTime algorithm systematically discovers all the ‘stylized facts’ of the financial time series automatically, all at once, which were previously noted by many researchers one at a time.

Pakistan is considered among the top five countries with the highest CO2 emissions globally. This calls for pragmatic policy implementation by all stakeholders to bring finality to this alarming situation since it contributes greatly to global warming, thereby leading to climate change. This study is an attempt to make a comparative analysis of the linear time series models with nonlinear time series models to study CO2 emission data in Pakistan. These linear and nonlinear time series models were used to model and forecast future values of CO2 emissions for a short period. To assess and select the best model among these linear and nonlinear time series models, we used the root mean square error (RMSE) and the mean absolute error (MAE) as performance indicators. The outputs showed that the nonlinear machine learning models are the best among all other models, having the lowest RMSE and MAE values. Based on the forecasted value of the nonlinear machine learning neural network autoregressive model, Pakistan's CO2 emissions will be 1.048 metric tons per capita by 2028. The increasing trend in emissions is a frightening and clear warning, suggesting that innovative policies must be initiated to reduce the trend. We encourage the Pakistan government to price CO2 emissions by companies and entities per ton, adapt electricity production from hydro, wind, and different sources with no emissions of CO2, initiate rigorous planting of more trees in the populated areas of Pakistan as forest covers, provide incentives to companies, organisations, institutions, and households to come out with clean technologies or use technologies with no CO2 emissions or those with lower ones, and fund more studies to develop clean and innovative technologies with less or no CO2 emissions.

The typical time series models such as ARMA, AR, and MA are founded on the normality and stationarity of a system and expressed by a linear difference equation; therefore, they are strictly limited to the linear system. However, some nonlinear factors are within the practical system; thus, it is difficult to fit the model for real systems with the above models. This paper proposes a general expression for linear and nonlinear auto-regressive time series models (GNAR). With the gradient optimization method and modified AIC information criteria integrated with the prediction error, the parameter estimation and order determination are achieved. The model simulation and experiments show that the GNAR model can accurately approximate to the dynamic characteristics of the most nonlinear models applied in academics and engineering. The modeling and prediction accuracy of the GNAR model is superior to the classical time series models. The proposed GNAR model is flexible and effective.

The performance of time series models is assessed using synthetic head series simulated with a numerical model that solves Richards' equation for variably saturated flow. Heads were simulated in a homogeneous unconfined aquifer between two parallel canals; measured daily precipitation and potential evaporation are specified at the land surface and root water uptake is simulated. The head response to a precipitation event is nonlinear and depends on the saturation degree and rainfall before and after the precipitation event while evaporation reduction occurs during summers. Synthetic series were generated for 27 years and three different soil types; the unsaturated zone thickness varies between 0 and >5 m. The synthetic head series were simulated with a linear and nonlinear time series model. Performance of a linear time series model with four parameters, using a scaled Gamma response, gave R2 values ranging from 0.67 to 0.96. The nonlinear time series model with five parameters simulates recharge using a root zone reservoir after which the head response to recharge is simulated with a scaled Gamma response function. The nonlinear time series model was able to simulate all synthetic head series very well with R2 values above 0.9 for almost all models. The head response of the nonlinear model to a single precipitation event compares well to the response of the variably saturated groundwater model. The provided scripts may be used to simulate synthetic head series for other climates or for systems with additional complexity to assess the performance of other data-driven models.

Gross domestic product (GDP) serves as a crucial economic indicator for measuring a country's economic growth, exhibiting both linear and non-linear trends. This study aims to analyze and propose an efficient and accurate time series approach for modeling and forecasting the GDP annual growth rate (%) of Saudi Arabia, a key financial indicator of the country. Stochastic linear and non-linear time series modeling, along with hybrid approaches, are employed and their results are compared. Initially, conventional linear and nonlinear methods such as ARIMA, Exponential smoothing, TBATS, and NNAR are applied. Subsequently, hybrid models combining these individual time series approaches are utilized. Model diagnostics, including mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE), are employed as criteria for model selection to identify the best-performing model. The findings demonstrated that the neural network autoregressive (NNAR) model, as a non-linear approach, outperformed all other models, exhibiting the lowest values of MAE, RMSE and MAPE. The NNAR(5,3) projected the GDP of 1.3% which is close to the projection of IMF benchmark (1.9) for the year 2023. The selected model can be employed by economists and policymakers to formulate appropriate policies and plans. This quantitative study provides policymakers with a basis for monitoring fluctuations in GDP growth from 2022 to 2029 and ensuring the sustained progression of GDP beyond 2029. Additionally, this study serves as a guide for researchers to test these approaches in different economic dynamics.

Chapter 3 - Introduction of multiple/multivariate linear and nonlinear time series models in forecasting streamflow process

Context. The practice of today’s problems actualizes the increase in requirements for the accuracy, reliability and completeness of the results of time series processing in many applied areas. One of the methods that provides high-precision processing of time series with the introduction of a stochastic model of measured parameters is statistical learning methods. However, modern approaches to statistical learning are limited, for the most part, to simplified polynomial models. Practice proves that real data most often have a complex form of a trend component, which cannot be reproduced by polynomials of even a high degree. Smoothing of nonlinear models can be implemented by various approaches, for example, by the method of determining the parameters of nonlinear models using the differential spectra balance (DSB) in the scheme of differential-non-Taylor transformations (DNT). The studies proved the need for its modification in the direction of developing a conditional approach to determining the structure of nonlinear mathematical models for processing time series with complex trend dynamics.Objective. The development of a method for determining the structure of nonlinear by mathematical models for processing time series using DSB in DNT transformations.Method. The paper develops a method for constructing nonlinear mathematical models in the DNT transformation scheme. The modification of the method consists in controlling the conditions for the formation of a certain system of equations in the DSB scheme to search for the parameters of a nonlinear model with its analytical solutions. If the system is indeterminate, the nonlinear model is supplemented by linear components. In the case of an overdetermined system, its solution is carried out using the least squares norm. A defined system is solved by classical approaches. These processes are implemented with the control of stochastic and dynamic accuracy of models in the areas of observation and extrapolation. If the results of statistical learning are unsatisfactory in accuracy, the obtained values of the nonlinear model are used as initial approximations of numerical methods.Result. Based on carried-out research, a method for determining the structure of nonlinear models for processing time series using BDS in the scheme of DNT transformations is proposed. Its application provides a conditional approach to determining the structure of models for processing time series and increasing the accuracy of estimation at the interval of observation and extrapolation.Conclusions. The application of the proposed method for determining the structure of nonlinear models for processing time series allows obtaining models with the best predictive properties in terms of accuracy

We propose a new diagnostic test for linear and nonlinear time series models, using a generalized spectral approach. Under a wide class of time series models that includes autoregressive conditional heteroskedasticity (ARCH) and autoregressive conditional duration (ACD) models, the proposed test enjoys the appealing "nuisance-parameter-free" property in the sense that model parameter estimation uncertainty has no impact on the limit distribution of the test statistic. It is consistent against any type of pairwise serial dependence in the model standardized residuals and allows the choice of a proper lag order via data-driven methods. Moreover, the new test is asymptotically more efficient than the correlation integral–based test of Brock, Hsieh, and LeBaron (1991, Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence) and Brock, Dechert, Scheinkman, and LeBaron (1996, Econometric Reviews 15, 197–235), the well-known BDS test, against a class of plausible local alternatives (not including ARCH). A simulation study compares the finite-sample performance of the proposed test and the tests of BDS, Box and Pierce (1970, Journal of the American Statistical Association 65, 1509–1527), Ljung and Box (1978, Biometrika 65, 297–303), McLeod and Li (1983, Journal of Time Series Analysis 4, 269–273), and Li and Mak (1994, Journal of Time Series Analysis 15, 627–636). The new test has good power against a wide variety of stochastic and chaotic alternatives to the null models for conditional mean and conditional variance. It can play a valuable role in evaluating adequacy of linear and nonlinear time series models. An empirical application to the daily S&P 500 price index highlights the merits of our approach.We thank the co-editor (Don Andrews) and two referees for careful and constructive comments that have lead to significant improvement over an earlier version. We also thank C.W.J. Granger, D. Tjøstheim, and Z. Xiao for helpful comments. Hong's participation is supported by the National Science Foundation via NSF grant SES–0111769. Lee thanks the UCR Academic Senate for research support.

In transportation applications such as real-time route guidance, ramp metering, congestion pricing and special events traffic management, accurate short-term traffic flow prediction is needed. For this purpose, this paper proposes several novel online Grey system models (GM): GM(1,1|cos(ωt)), GM(1,1|sin(ωt),cos(ωt)), and GM(1,1|e-at,sin(ωt),cos(ωt)). To evaluate the performance of the proposed models, they are compared against a set of benchmark models: GM(1,1) model, Grey Verhulst models with and without Fourier error corrections, linear time series model, and nonlinear time series model. The evaluation is performed using loop detector and probe vehicle data from California, Virginia, and Oregon. Among the benchmark models, the error corrected Grey Verhulst model with Fourier outperformed the GM(1,1) model, linear time series, and non-linear time series models. In turn, the three proposed models, GM(1,1|cos(ωt)), GM(1,1|sin(ωt),cos(ωt)), and GM(1,1|e-at,sin(ωt),cos(ωt)), outperformed the Grey Verhulst model in prediction by at least 65%, 16% and 11%, in terms of Root Mean Squared Error, and by 82%, 58% and 42%, in terms of Mean Absolute Percentage Error, respectively. It is observed that the proposed Grey system models are more adaptive to location (e.g., perform well for all roadway types) and traffic parameters (e.g., speed, travel time, occupancy, and volume), and they do not require as many data points for training (4 observations are found to be sufficient).

In this paper, we propose automatic procedures for the model selection of various univariate time series data. Automatic model selection is important, especially in data mining with large number of time series, for example, the number (in thousands) of signals accessing a web server during a specific time period. Several methods have been proposed for automatic model selection of time series. However, most existing methods focus on linear time series models such as exponential smoothing and autoregressive integrated moving average(ARIMA) models. The key feature that distinguishes the proposed procedures from previous approaches is that the former can be used for both linear time series models and nonlinear time series models such as threshold autoregressive(TAR) models and autoregressive moving average-generalized autoregressive conditional heteroscedasticity(ARMA-GARCH) models. The proposed methods select a model from among the various models in the prediction error sense. We also provide an R package autots that implements the proposed automatic model selection procedures. In this paper, we illustrate these algorithms with the artificial and real data, and describe the implementation of the autots package for R.

A method for parametric estimation of higher order spectra of time series using a nonlinear autoregressive model based on multi-layered neural networks (NNAR model) is presented. In real world problems there exist signals that can not be described sufficiently by linear time series models such as AR or ARMA models. In order to characterize such signals, several nonlinear time series models have been investigated in recent years. However, in contrast with the case of linear models, there are a few parametric approaches that estimate the higher order statistical characteristics of observed time series using such nonlinear time series models. It is very difficult to derive analytically explicit formulations of higher order spectra from the expressions of such nonlinear time series models. In this study, employing numerical techniques, the authors construct a parametric estimator of higher order spectra. It consists of the following steps: 1. training an NNAR model on the given time series, 2. iteration of numerical integrals for solving the joint probability density function, 3. calculation of higher order cumulant functions by renewal equations based on the joint probability density function solved in 2., and 4. multidimensional discrete Fourier transforms of higher order cumulant functions calculated in 3. The authors also show that any NNAR model with finite valued weights satisfies a sufficient condition of convergence.

Time series modelling approaches are useful tools for simulating and forecasting hydrological variables and their change through time. Although linear time series models are common in hydrology, the nonlinear time series model, the Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model, has rarely been used in hydrology and water resources engineering. The GARCH model considers the conditional variance remaining in the residuals of the linear time series models, such as an ARMA or an ARIMA model. In the present study, the advantages of a GARCH model against a linear ARIMA model are investigated using three classes of the GARCH approach, namely Power GARCH, Threshold GARCH and Exponential GARCH models. A daily streamflow time series of the Matapedia River, Quebec, Canada, is selected for this study. It is shown that the ARIMA (13,1,4) model is adequate for modelling streamflow time series of Matapedia River, but the Engle test shows the existence of heteroscedasticity in the residuals of the ARIMA model. Therefore, an ARIMA (13,1,4)-GARCH (3,1) error model is fitted to the data. The residuals of this model are examined for the existence of heteroscedasticity. The Engle test indicates that the GARCH model has considerably reduced the heteroscedasticity of the residuals. However, the Exponential GARCH model seems to completely remove the heteroscedasticity from the residuals. The multi-criteria evaluation for model performance also proves that the Exponential GARCH model is the best model among ARIMA and GARCH models. Therefore, the application of a GARCH model is strongly suggested for hydrological time series modelling as the conditional variance of the residuals of the linear models can be removed and the efficiency of the model will be improved. Editor D. Koutsoyiannis; Associate editor C. Onof Citation Modarres, R. and Ouarda, T.B.M.J., 2013. Modelling heteroscedasticty of streamflow times series. Hydrological Sciences Journal, 58 (1), 1–11.

BackgroundPrediction models have gained immense importance in various fields for decision-making purposes. In the context of tennis, relying solely on the probability of winning a single match may not be sufficient for predicting a player's future performance or ranking. The performance of a tennis player is influenced by the timing of their matches throughout the year, necessitating the incorporation of time as a crucial factor. This study aims to focus on prediction models for performance indicators that can assist both tennis players and sports analysts in forecasting player standings in future matches.MethodologyTo predict player performance, this study employs a dynamic technique that analyzes the structure of performance using both linear and nonlinear time series models. A novel approach has been taken, comparing the performance of the non-linear Neural Network Auto-Regressive (NNAR) model with conventional stochastic linear and nonlinear models such as Auto-Regressive Integrated Moving Average (ARIMA), Exponential Smoothing (ETS), and TBATS (Trigonometric Seasonal Decomposition Time Series).ResultsThe study finds that the NNAR model outperforms all other competing models based on lower values of Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). This superiority in performance metrics suggests that the NNAR model is the most appropriate approach for predicting player performance in tennis. Additionally, the prediction results obtained from the NNAR model demonstrate narrow 95% Confidence Intervals, indicating higher accuracy and reliability in the forecasts.ConclusionIn conclusion, this study highlights the significance of incorporating time as a factor when predicting player performance in tennis. It emphasizes the potential benefits of using the NNAR model for forecasting future player standings in matches. The findings suggest that the NNAR model is a recommended approach compared to conventional models like ARIMA, ETS, and TBATS. By considering time as a crucial factor and employing the NNAR model, both tennis players and sports analysts can make more accurate predictions about player performance.

Water evaporation process is one of the main components of the hydrological cycle, according to which the correct estimation of this phenomenon plays an important role in irrigation management and river flow forecasting. Statistical and random models in the form of time series analysis are commonly used methods of estimation and forecasting. In this study, ARMA (autoregressive moving average), ARIMA (autoregressive integrated moving average), PARMA (periodic autoregressive moving average), and BL (bilinear) time series models (TSMs) are used to predict the annual and monthly pan evaporation values from the evaporation stations in several provinces of Iran in a 20-year statistical period. Results showed that PARMA model’s accuracy in term of correlation coefficient and Nash–Sutcliffe efficiency for almost all 31 stations had a precise forecasting among other proposed TSMs. For PARMA model, the forecasting accuracy in term of NSE indicated that PARMA model almost was the promising model among other linear and nonlinear TSMs in prediction of Epan for all stations, except Arak (NSE = 0.94) and Qom (NSE = 0.93). The ARIMA model for Khorramabad and Bandar Abbas with NSE = 0.52 had the unreliable prediction for Epan compared with other stations. In addition, Arak station in term of RMSE had the least error, 23.79 mm/day and 10.90 mm/day, respectively, for training and testing stages among the other stations.