Autoregressive Model Fitting Research Articles

Detecting overfitting by examining residuals in autoregressive models

The aim of this study is to see how overfitting can be detected using non-rigorous analysis of residuals. The well-known statistical packages was used to simulate data from stationary autoregressive models with Gaussian white noise with mean 0 and variance one. In order to see the effect of realization size on our findings, the sample size 50 was used as an example of small realization and the sample size 500 was used as an example of large realization. The method of maximum likelihood was used in the fitting of autoregressive models to the simulated data which is available in the statistical package R. Interesting and promising results were obtained. Our study seems to suggest that comparing estimates with their standard errors is the only reliable criterion in spotting or detecting overfitting. To make sure that the defect in the behavior of the residuals is due only to the over, we used only the same class of models in the simulation and the fitting.

Sevoflurane and Hypercapnia Blunt the Physiological Variability of Spontaneous Breathing: A Comparative Interventional Study.

Background: Although spontaneous breathing is known to exhibit substantial physiological fluctuation that contributes to alveolar recruitment, changes in the variability of the respiratory pattern following inhalation of carbon dioxide (CO2) and volatile anesthetics have not been characterized. Therefore, we aimed at comparing the indices of breathing variability under wakefulness, sleep, hypercapnia and sedative and anesthetic concentrations of sevoflurane. Methods: Spontaneous breathing pattern was recorded on two consecutive days in six rabbits using open whole-body plethysmography under wakefulness and spontaneous sleep and following inhalation of 5% CO2, 2% sevoflurane (0.5 MAC) and 4% (1 MAC) sevoflurane. Tidal volume (VT), respiratory rate (RR), minute ventilation (MV), inspiratory time (TI) and mean inspiratory flow (VT/TI) were calculated from the pressure fluctuations in the plethysmograph. Means and coefficients of variation were calculated for each measured variable. Autoregressive model fitting was applied to estimate the relative contributions of random, correlated, and oscillatory behavior to the total variance. Results: Physiological sleep decreased MV by lowering RR without affecting VT. Hypercapnia increased MV by elevating VT. Sedative and anesthetic concentrations of sevoflurane increased VT but decreased MV due to a decrease in RR. Compared to the awake stage, CO2 had no effect on VT/TI while sevoflurane depressed significantly the mean inspiratory flow. Compared to wakefulness, the variability in VT, RR, MV, TI and VT/TI were not affected by sleep but were all significantly decreased by CO2 and sevoflurane. The variance of TI originating from correlated behavior was significantly decreased by both concentrations of sevoflurane compared to the awake and asleep conditions. Conclusions: The variability of spontaneous breathing during physiological sleep and sevoflurane-induced anesthesia differed fundamentally, with the volatile agent diminishing markedly the fluctuations in respiratory volume, inspiratory airflow and breathing frequency. These findings may suggest the increased risk of lung derecruitment during procedures under sevoflurane in which spontaneous breathing is maintained.

Normal inverse Gaussian autoregressive model using EM algorithm

In this article, normal inverse Gaussian (NIG) autoregressive model is introduced. The parameters of the model are estimated using expectation maximization (EM) algorithm. The efficacy of the EM algorithm is shown using simulated and real-world financial data. It is shown that NIG autoregressive model fit very well on the considered financial data and hence could be useful in modelling of various real-life time-series data.

Closed-form expression for finite predictor coefficients of multivariate ARMA processes

We derive a closed-form expression for the finite predictor coefficients of multivariate ARMA (autoregressive moving-average) processes. The expression is given in terms of several explicit matrices that are of fixed sizes independent of the number of observations. The significance of the expression is that it provides us with a linear-time algorithm to compute the finite predictor coefficients. In the proof of the expression, a correspondence result between two relevant matrix-valued outer functions plays a key role. We apply the expression to determine the asymptotic behavior of a sum that appears in the autoregressive model fitting and the autoregressive sieve bootstrap. The results are new even for univariate ARMA processes.

An Experiment on Autoregressive and Threshold Autoregressive Models with Non-Gaussian Error with Application to Realized Volatility

This article explores the fitting of Autoregressive (AR) and Threshold AR (TAR) models with a non-Gaussian error structure. This is motivated by the problem of finding a possible probabilistic model for the realized volatility. A Gamma random error is proposed to cater for the non-negativity of the realized volatility. With many good properties, such as consistency even for non-Gaussian errors, the maximum likelihood estimate is applied. Furthermore, a non-gradient numerical Nelder–Mead method for optimization and a penalty method, introduced for the non-negative constraint imposed by the Gamma distribution, are used. In the simulation experiments, the proposed fitting method found the true model with a rather insignificant bias and mean square error (MSE), given the true AR or TAR model. The AR and TAR models with Gamma random error are then tested on empirical realized volatility data of 30 stocks, where one third of the cases are fitted quite well, suggesting that the model may have potential as a supplement for current Gaussian random error models with proper adaptation.

Mean warming not variability drives marine heatwave trends

Marine heatwaves have been shown to be increasing in frequency, duration and intensity over the past several decades. Are these changes related to rising mean temperatures, changes to temperature variability, or a combination of the two? Here we investigate this question using satellite observations of sea surface temperature (SST) covering 36 years (1982–2017). A statistical climate model is used to simulate SST time series, including realistic variability based on an autoregressive model fit to observations, with specified trends in mean and variance. These simulated SST time series are then used to test whether observed trends in marine heatwave properties can be explained by changes in either mean or variability in SST, or both. We find changes in mean SST to be the dominant driver of the increasing frequency of marine heatwave days over approximately 2/3 of the ocean; while it is the dominant driver of changes in marine heatwave intensity (temperature anomaly) over approximately 1/3 of the ocean. We also find that changes in mean SST explain changes in both MHW properties over a significantly larger proportion of the world’s ocean than changes in SST variance. The implication is that given the high confidence of continued mean warming throughout the twenty-first century due to anthropogenic climate change we can expect the historical trends in marine heatwave properties to continue over the coming decades.

Optimal repetition time reduction for single subject event-related functional magnetic resonance imaging.

PurposeShort TRs are increasingly used for fMRI as fast sequences such as simultaneous multislice excitation become available. These have been associated with apparent sensitivity improvements, although greater temporal autocorrelation at shorter TRs can inflate sensitivity measurements leading to uncertainty regarding the optimal approach.MethodsIn volunteers (n = 10), the optimal TR was assessed at the single subject level for event‐related designs (visual stimulation) with 4 frequencies of presentation at 4 TR values (412‐2550 ms). T‐values in the visual cortex localized in each individual were obtained and receiver operating characteristics (ROC) analysis was performed by counting voxels within and outside expected task active regions at different thresholds. This analysis was repeated using 4 different autoregressive (AR) models; SPM AR(1) and SPM AR(fast) which globally estimate autocorrelation, and fMRIstat AR(1) and AR(5) that use a local estimate.ResultsThe use of modest multiband factors of 2 or 3 with a reduction in TR to 1000 ± 200 ms had greater sensitivity and specificity as shown by higher T‐values in visual cortex and ROC analysis. At these TRs, the ROC analysis demonstrated that a local AR model fit improved performance while high order AR models were unnecessary.ConclusionsModest TR reductions (to 1000 ± 200 ms) optimally improved event‐related fMRI performance independent of design frequency. Autoregressive models with a local as opposed to global fit performed better, while low order autoregressive models were sufficient at the optimal TR.

Increase in random component of heart rate variability coinciding with developmental and degenerative stages of life

Objective: To adapt to a new environment or situation, biological systems explore the most convenient state while moving between attractors by the force of random fluctuation. From this concept, the random component in physiological signals is assumed to increase during developmental and degenerative stages of life. To examine this hypothesis, we measured the age-dependent changes in the random component of heart rate variability (HRV) in 24-h electrocardiography (ECG) big data and in patients with a developmental disorder (DD). Approach: We measured separately regulated and random components of HRV with autoregressive (AR) model fitting, by which the ratio of random component as the fractional variance of AR residual time series. From the ALLSTAR database of about 304 000 ambulatory 24-h ECGs, we randomly extracted the data of 1930 men and 1987 women uniformly for all ages from 0 to 100 years old (100 cases per 5-year strata for each sex). Data were also obtained from male pediatric patients with DD (age 10–15 years). Main results: While the variance of the regulated component of HRV increased from age 0 to 20, decreased with age until 40, and reached a plateau in both sexes, the ratio of the random component was high at birth, decreased with age until 35 in men and 30 in women, and increased again after 75 in men and 85 in women (P < 0.0001 for all). In patients with a DD, the ratio of the random component was significantly lower than that in age-and-sex matched subjects in the database. Significance: We found that the ratio of the random component of HRV is increased during developmental and degenerative stages of life and that it may be reduced in DD patients during their development.

Causal drivers of Barents Sea capelin (Mallotus villosus) population dynamics on different time scales

AbstractThe dynamics of marine populations are usually forced by biotic and abiotic factors occurring at different intensity levels and time scales. Deriving the time frame within which each factor has a causal influence is important for predicting population trajectories. This paper presents a statistical methodology for establishing (i) the strength of causal coupling between population dynamics and environmental (biotic and abiotic) factors, and (ii) the time scales over which causal covariates have significant influence on the population dynamics. The methodology is based on combining a multivariate autoregressive model fit to data (to determine causal direction) with a quantification of the RPC of covariates in frequency domain (to quantify the strength of connection). The methodology is applied to test the existence of causal coupling between the capelin biomass and a selected number of covariates identified in the literature.

Some properties of the autoregressive-aided block bootstrap

We investigate properties of a hybrid bootstrap procedure for general, strictly stationary sequences, called the autoregressive-aided block bootstrap which combines a parametric autoregressive bootstrap with a nonparametric moving block bootstrap. The autoregressive-aided block bootstrap consists of two main steps, namely an autoregressive model fit and an ensuing (moving) block resampling of residuals. The linear parametric model-fit prewhitenes the time series so that the dependence structure of the remaining residuals gets closer to that of a white noise sequence, while the moving block bootstrap applied to these residuals captures nonlinear features that are not taken into account by the linear autoregressive fit. We establish validity of the autoregressive-aided block bootstrap for the important class of statistics known as generalized means which includes many commonly used statistics in time series analysis as special cases. Numerical investigations show that the hybrid bootstrap procedure considered in this paper performs quite well, it behaves as good as or it outperforms in many cases the ordinary moving block bootstrap and it is robust against mis-specifications of the autoregressive order, a substantial advantage over the autoregressive bootstrap.

Multivariate Least Squares Forecasting Averaging by Vector Autoregressive Models

This paper proposes a multivariate least squares Mallows averaging approach to the issue of forecast combination by vector autoregressive (VAR) model fitting. Our approach extends the current literature on frequentist least squares model/forecast averaging methods, in particular Hansen (2008), to multivariate time series models. We provide a theoretical foundation of our approach by presenting the relation between the proposed multivariate Mallows averaging criterion and the in-sample mean squared error and out-of-sample mean squared forecast error. We also establish the asymptotic properties such as unbiasedness and optimality of our approach. In a simulation experiment, the proposed approach performs well in finite samples relative to other selection and averaging methods. For an empirical illustration, we apply our methodology to forecasting U.S. macroeconomic dynamic systems based on small-scale and medium-scale VARs fitted to the datasets that were previously studied by Sims (1980) and Stock and Watson (2009).

Associations between Depressive Symptoms and Social Support in Adults with Diabetes: Comparing Directionality Hypotheses with a Longitudinal Cohort.

Individuals with diabetes are at increased risk of elevated depressive symptoms, and social support has been identified as a key factor in the health of this population. Cross-sectional associations between depressive symptoms and social support have been demonstrated. Three classes of hypotheses differentially describe the direction of this association: (1) depressive symptoms influence social support, (2) social support influences depressive symptoms, and (3) reciprocal associations exist between depressive symptoms and social support. The aim of this study was to compare these hypotheses. Depressive symptoms and social support were measured via telephone survey in a large cohort study of individuals with diabetes (n = 1754) in Quebec, Canada. After baseline, data were collected annually for 4years. Path models depicting each hypothesis, as well as a stability model containing only autoregressive effects, were generated, and model fit was compared with Akaike's Information Criterion (AIC). The reciprocal model was selected as the best fitting model because it had the lowest AIC. This model demonstrated that depressive symptoms predicted subsequent social support at all time points and that social support predicted subsequent depressive symptoms at most time points. It appears that the association between depressive symptoms and social support in people with diabetes is best characterized as reciprocal. Results underscore the importance of directly comparing competing hypotheses and offer a more accurate depiction of the association between depressive symptoms and social support among people with diabetes.

A Bayesian analysis of an agricultural field trial with three spatial dimensions

Modern technology now has the ability to generate large datasets over space and time. Such data typically exhibit high autocorrelations over all dimensions. The field trial data motivating the methods of this paper were collected to examine the behaviour of traditional cropping and to determine a cropping system which could maximise water use for grain production while minimising leakage below the crop root zone. They consist of moisture measurements made at 15 depths across 3 rows and 18 columns, in the lattice framework of an agricultural field. Bayesian conditional autoregressive (CAR) models are used to account for local site correlations. Conditional autoregressive models have not been widely used in analyses of agricultural data. This paper serves to illustrate the usefulness of these models in this field, along with the ease of implementation in WinBUGS, a freely available software package. The innovation is the fitting of separate conditional autoregressive models for each depth layer, the ‘layered CAR model’, while simultaneously estimating depth profile functions for each site treatment. Modelling interest also lies in how best to model the treatment effect depth profiles, and in the choice of neighbourhood structure for the spatial autocorrelation model. The favoured model fitted the treatment effects as splines over depth, and treated depth, the basis for the regression model, as measured with error, while fitting CAR neighbourhood models by depth layer. It is hierarchical, with separate conditional autoregressive spatial variance components at each depth, and the fixed terms which involve an errors-in-measurement model treat depth errors as interval-censored measurement error. The Bayesian framework permits transparent specification and easy comparison of the various complex models compared.

Structural damage detection using autoregressive-model-incorporating multivariate exponentially weighted moving average control chart

A novel structural damage detection scheme using autoregressive-model-incorporating multivariate exponentially weighted moving average (MEWMA) control chart is presented in this paper. This scheme comprises procedures based on the undamaged or reference state of the structure being monitored and those based on its damaged or current state. In the procedures based on the reference state, sets of multivariate data are formulated by a series of autoregressive (AR) model fitting, and these data are then subjected to MEWMA control chart analysis to establish a benchmark damage indicator. The damage indicator obtained in the procedures based on the current state is compared with the benchmark for the purpose of structural damage detection. The autocorrelation in the multivariate data is addressed, and special procedures to allow for the uncertainty involved in process parameter estimation as well as those for control limit determination are proposed for structural damage detection application. A numerically simulated case study is used to verify the efficacy of the proposed scheme and to show its advantages. A parametric study is also included to study the effects of some parameters and to demonstrate the robustness of the scheme against parameter selection.

An algorithm for robust fitting of autoregressive models

An algorithm for robust fitting of AR models is given, based on a linear regression idea. The new method appears to outperform the Yule–Walker estimator in a setting of data contaminated with outliers.

Autoregressive coefficients based Hotelling’s T2 control chart for structural health monitoring

This paper presents an innovative structural health monitoring (SHM) scheme based on time series analysis and multivariate statistical process control (MSPC) techniques. The scheme consists of two major procedures, viz vibration response data representation and characteristics monitoring. First, a series of autoregressive (AR) models are fitted to the response time histories of a structure to be monitored. Representing the health condition of the structure, the coefficients of these AR models are extracted to form a set of multivariate data known as vibration response data characteristics. Hotelling’s T 2 control chart is then applied to monitor these characteristics obtained. As an MSPC tool, Hotelling’s T 2 control chart has the capacity of simultaneously monitoring the multivariate characteristics data without having to neglect the inherent relation between the components of the data. The efficacy of the proposed SHM scheme is demonstrated by numerically simulated acceleration time histories based on a progressively damaged reinforced concrete (RC) frame, either with or without addressing the autocorrelation in the characteristics data. The results are compared with those obtained by using univariate Shewhart X ¯ control chart to show the advantages of the proposed scheme in terms of the sensitivity of the defined damage indicator with respect to damage severity. A parametric study is also included to investigate the effects of the number of data points used for AR model fitting, the order of AR models, the number and location of sensors on the proposed scheme and to further illustrate its potential as a promising SHM approach.

Robust detection of gearbox deterioration using compromised autoregressive modeling and Kolmogorov–Smirnov test statistic. Part II: Experiment and application

Optimal maintenance decision analysis is heavily dependent on the accuracy of condition indicators. A condition indicator that is subject to such varying operating conditions as load is unable to provide precise condition information of the monitored object for making optimal operational maintenance decisions even if the maintenance program is established within a rigorous theoretical framework. For this reason, the performance of condition monitoring techniques applied to rotating machinery under varying load conditions has been a long-term concern and has attracted intensive research interest. Part I of this study proposed a novel technique based on adaptive autoregressive modeling and hypothesis tests. The method is able to automatically search for the optimal time-series model order and establish a compromised autoregressive model fitting based on the healthy gear motion residual signals under varying load conditions. The condition of the monitored gearbox is numerically represented by a modified Kolmogorov–Smirnov test statistic. Part II of this study is devoted to applications of the proposed technique to entire lifetime condition detection of three gearboxes with distinct physical specifications, distinct load conditions, and distinct failure modes. A comprehensive and thorough comparative study is conducted between the proposed technique and several counterparts. The detection technique is further enhanced by a proposed method to automatically identify and generate fault alerts with the aid of the Wilcoxon rank-sum test and thus requires no supervision from maintenance personnel. Experimental analysis demonstrated that the proposed technique applied to automatic identification and generation of fault alerts also features two highly desirable properties, i.e. few false alerts and early alert for incipient faults. Furthermore, it is found that the proposed technique is able to identify two types of abnormalities, i.e. strong ghost components abruptly appearing in and disturbing the nominal operating process and the breakage of gear teeth, whereas conventional condition indicators are usually not capable of detecting the presence of the former.

Instantaneous multivariate EEG coherence analysis by means of adaptive high-dimensional autoregressive models

This study presents an efficient algorithm for the fitting of multivariate autoregressive models (MVAR) with time-dependent parameters to multidimensional signals. Thereby, the dimension of the model may be chosen to equal the number of signal channels. The autoregressive (AR) parameter matrices are estimated by an extension of the recursive least squares (RLS) algorithm with forgetting factor. The estimation procedure includes a single trial as well as an ensemble mean approach. The latter approach allows the simultaneous fit of one mean MVAR model to a set of single trials, each of them representing the measurement of the same task. A particular advantage of this ensemble mean approach is that it requires only a low computation effort in comparison to well known procedures applied to single trials. Furthermore, the ensemble mean approach is linked with a high adaptation capability. The properties of the estimator are investigated using simulated time series. It can be demonstrated that the adaptation capability of the estimation (measured by its adaptation speed and variance) does not depend on the model dimension. The mean MVAR fit is applied to 19-dimensional EEG data, recorded during an elementary comparison procedure. The calculation of ordinary and multiple coherence is discussed. The sensitivity of the multiple instantaneous EEG coherence will be demonstrated.

Developmental models of infant and childhood temperament.

The developmental courses of specific temperamental constructs were explored by using structural equation model fitting. Maternal ratings were obtained from either 2 or 3 different temperament questionnaires for 180 children at 3, 6, 12, 18, 24, 36, and 48 months of age. Several formal structural models were fit in infancy (3-18 months), in the toddler-preschooler period (24-48 months), and across all measurement occasions. In infancy, the autoregressive simplex model fit well for all 4 composites considered: Positive Emotionality, Distress-Anger, Fear, and Activity Level. In contrast to the considerable change in temperament during infancy, temperament appears to be very stable from 24 to 48 months of age, and a common factor model fits well with these data. Across all measurement occasions, models that allowed for stability in temperament to be at least partially mediated through intermediate forms of the trait fit best.

Spectral analysis of heart rate, arterial pressure, and muscle sympathetic nerve activity in normal humans.

We investigated the frequency components of fluctuations in heart rate, arterial pressure, respiration, and muscle sympathetic nerve activity (MSNA) in 11 healthy women using an autoregressive model and examined the relation among variables using Akaike's relative power contribution analysis with multivariate autoregressive model fitting. Power spectral analysis of MSNA revealed two peaks, with low-frequency (LF) and high-frequency (HF) components. The LF component of MSNA was a major determinant of the LF component of arterial pressure and R-R interval variability (0.70 +/- 0.07 and 0.18 +/- 0.05, respectively). The effect of the LF component of MSNA on arterial pressure showed no change in response to propranolol but was diminished (0.35 +/- 0.08) by phentolamine (P < 0.02). The effect of the LF component of MSNA on R-R interval was not altered by pharmacological sympathetic nerve blockade. The HF component of MSNA did not influence other variables but was influenced by R-R interval, arterial pressure, and respiration. These findings indicate that the LF component of MSNA reflects autonomic oscillations, whereas the HF component is passive and influenced by other cardiovascular variables.