Autocorrelation In Data Research Articles

Efficient random field modeling of soil deposits properties

The autocorrelation function (ACF) of the soil profile in some sites in Shandong province, China is studied using cone penetration test (CPT) data. This is done in the context of a random field modeling of the soil deposits. It is found that the different types of soil profile have different stochastic parameters, and there is no obvious trend along the depth of the soil profile. Thus, the soil profile is examined within each layer. Numerical values for three existing analytical (ACF) models are derived by the least squares fitting approach for the different types of soil. Further, comparisons of the autocorrelation function between the tip resistance and sleeve friction were examined. Based on the autocorrelation data analysis, a new autocorrelation model, named linear-exponential-cosine (LNCS), is considered with differentiability at the origin of the spatial lag axis, and alternating sign along this axis. For all of the four ACF models, a related integral equation is numerically solved for determining the associated Karhunen-Loeve (K-L) representation. In this regard, it is noted that the new model is not only more physics-consistent, but also yields quite good computational efficiency. In the end, the random field of the soil profile is modeled using a two-dimensional K-L expansion with the new model, assuming separability in two dimensions.

Markov modeling of peptide folding in the presence of protein crowders.

We use Markov state models (MSMs) to analyze the dynamics of a β-hairpin-forming peptide in Monte Carlo (MC) simulations with interacting protein crowders, for two different types of crowder proteins [bovine pancreatic trypsin inhibitor (BPTI) and GB1]. In these systems, at the temperature used, the peptide can be folded or unfolded and bound or unbound to crowder molecules. Four or five major free-energy minima can be identified. To estimate the dominant MC relaxation times of the peptide, we build MSMs using a range of different time resolutions or lag times. We show that stable relaxation-time estimates can be obtained from the MSM eigenfunctions through fits to autocorrelation data. The eigenfunctions remain sufficiently accurate to permit stable relaxation-time estimation down to small lag times, at which point simple estimates based on the corresponding eigenvalues have large systematic uncertainties. The presence of the crowders has a stabilizing effect on the peptide, especially with BPTI crowders, which can be attributed to a reduced unfolding rate ku, while the folding rate kf is left largely unchanged.

Spatial autoregressive models for statistical inference from ecological data

AbstractEcological data often exhibit spatial pattern, which can be modeled as autocorrelation. Conditional autoregressive (CAR) and simultaneous autoregressive (SAR) models are network‐based models (also known as graphical models) specifically designed to model spatially autocorrelated data based on neighborhood relationships. We identify and discuss six different types of practical ecological inference using CAR and SAR models, including: (1) model selection, (2) spatial regression, (3) estimation of autocorrelation, (4) estimation of other connectivity parameters, (5) spatial prediction, and (6) spatial smoothing. We compare CAR and SAR models, showing their development and connection to partial correlations. Special cases, such as the intrinsic autoregressive model (IAR), are described. Conditional autoregressive and SAR models depend on weight matrices, whose practical development uses neighborhood definition and row‐standardization. Weight matrices can also include ecological covariates and connectivity structures, which we emphasize, but have been rarely used. Trends in harbor seals (Phoca vitulina) in southeastern Alaska from 463 polygons, some with missing data, are used to illustrate the six inference types. We develop a variety of weight matrices and CAR and SAR spatial regression models are fit using maximum likelihood and Bayesian methods. Profile likelihood graphs illustrate inference for covariance parameters. The same data set is used for both prediction and smoothing, and the relative merits of each are discussed. We show the nonstationary variances and correlations of a CAR model and demonstrate the effect of row‐standardization. We include several take‐home messages for CAR and SAR models, including (1) choosing between CAR and IAR models, (2) modeling ecological effects in the covariance matrix, (3) the appeal of spatial smoothing, and (4) how to handle isolated neighbors. We highlight several reasons why ecologists will want to make use of autoregressive models, both directly and in hierarchical models, and not only in explicit spatial settings, but also for more general connectivity models.

A semiparametric nonlinear mixed model approach to phase I profile monitoring

ABSTRACTWhen process data follow a particular curve in quality control, profile monitoring is suitable and appropriate for assessing process stability. Previous research in profile monitoring focusing on nonlinear parametric (P) modeling, involving both fixed and random-effects, was made under the assumption of an accurate nonlinear model specification. Lately, nonparametric (NP) methods have been used in the profile monitoring context in the absence of an obvious linear P model. This study introduces a novel technique in profile monitoring for any nonlinear and auto-correlated data. Referred to as the nonlinear mixed robust profile monitoring (NMRPM) method, it proposes a semiparametric (SP) approach that combines nonlinear P and NP profile fits for scenarios in which a nonlinear P model is adequate over part of the data but inadequate of the rest. These three methods (P, NP, and NMRPM) account for the auto-correlation within profiles and treats the collection of profiles as a random sample with a common population. During Phase I analysis, a version of Hotelling’s T2 statistic is proposed for each approach to identify abnormal profiles based on the estimated random effects and obtain the corresponding control limits. The performance of the NMRPM method is then evaluated using a real data set. Results reveal that the NMRPM method is robust to model misspecification and performs adequately against a correctly specified nonlinear P model. Control charts with the NMRPM method have excellent capability of detecting changes in Phase I data with control limits that are easily computable.

Multioutput least square SVR-based multivariate EWMA control chart: The performance evaluation and application

Autocorrelation leads to a bias estimator of standard control charts. It is important to develop control chart that allows autocorrelation and to evaluate its performance. The objective of this paper is to evaluate the performance of multioutput least square support vector regression (MLS-SVR)-based multivariate exponentially weighted moving average (MEWMA) control chart for monitoring multivariate autocorrelated data. For first order of vector autoregressive (VAR) and first order of vector moving average data, the proposed control chart tends to yield stable in-control average run length at about 200. The proposed control chart becomes more insensitive due to the increase of MEWMA smoothing parameter. In the real application, the proposed method is successfully applied to monitor water turbidity and chlorine residual data in the drinking water manufacturing process.

Development of a methodology to assess future trends in low flows at the watershed scale using solely climate data

Low flow conditions are governed by short-to-medium term weather conditions or long term climate conditions. This prompts the question: given climate scenarios, is it possible to assess future extreme low flow conditions from climate data indices (CDIs)? Or should we rely on the conventional approach of using outputs of climate models as inputs to a hydrological model? Several CDIs were computed using 42 climate scenarios over the years 1961–2100 for two watersheds located in Québec, Canada. The relationship between the CDIs and hydrological data indices (HDIs; 7- and 30-day low flows for two hydrological seasons) were examined through correlation analysis to identify the indices governing low flows. Results of the Mann-Kendall test, with a modification for autocorrelated data, clearly identified trends. A partial correlation analysis allowed attributing the observed trends in HDIs to trends in specific CDIs. Furthermore, results showed that, even during the spatial validation process, the methodological framework was able to assess trends in low flow series from: (i) trends in the effective drought index (EDI) computed from rainfall plus snowmelt minus PET amounts over ten to twelve months of the hydrological snow cover season or (ii) the cumulative difference between rainfall and potential evapotranspiration over five months of the snow free season. For 80% of the climate scenarios, trends in HDIs were successfully attributed to trends in CDIs. Overall, this paper introduces an efficient methodological framework to assess future trends in low flows given climate scenarios. The outcome may prove useful to municipalities concerned with source water management under changing climate conditions.

Optimal sampling methods for modelling the occupancy of Arctic grayling (Thymallus arcticus) in the Canadian Barrenlands

In occupancy models, imperfect detectability of animals is usually corrected for by using temporally repeated surveys to estimate probability of detection. Substituting spatial replicates for temporal replicates could be an advantageous sampling strategy in remote Arctic regions, but may lead to serious violations of model assumptions. Using a case study of site occupancy of adfluvial young-of-year Arctic grayling (Thymallus arcticus) in Barrenland tundra streams, we assessed the reliability and efficiency of alternative sampling strategies: (i) randomly distributed versus sequential adjacent spatial replicates; (ii) visual versus electrofishing surveys; and (iii) spatial versus temporal replicates. Sequential, adjacent spatial replicates produced spatially autocorrelated data. Autocorrelation was relieved using randomly distributed spatial replicates, but using these randomly distributed spatial replicates introduced significant error into estimates of the probability of occupancy in streams. Models designed for spatially autocorrelated data could minimize this bias. Visual and electrofishing surveys produced comparable probabilities of detection. Spatially replicated surveys performed better than temporal replicates. The easiest and relatively most cost-effective sampling methods performed as well as, or better than, the more established, expensive, and logistically difficult alternatives for occupancy estimation.

Effects of intrinsic sources of spatial autocorrelation on spatial regression modelling

Abstract Detecting and dealing with spatial autocorrelation (SA) are indispensable steps in analyses of geospatial data. Despite consensus that SA originates from both extrinsic factors and intrinsic interactions, previous studies on regression analysis of spatially autocorrelated data have rarely controlled for intrinsic sources in addition to extrinsic ones to assess ceteris paribus (i.e. causal) effects of interest, with the strict exogeneity assumption that errors containing unexplained variance are uncorrelated with explanatory variables for all observations. This assumption becomes invalid when intrinsic SA is not an external process modelled as errors and needs to be controlled for. Here, we aimed to assess the extent to which not controlling for intrinsic SA negatively affects model performance, specifically in terms of type I error rate and unbiasedness of coefficient estimates, and to identify models that are able to handle these problems. To this end, we applied two categories of regression models that do (intrinsic category) or do not (extrinsic category) explicitly control for intrinsic SA to artificial data generated with both SA sources. These models included the extended spatial Durbin model (ESDM) and its nested models. Four analytic scenarios simulated modelling conditions with increasing complexity of variables omitted during modelling. The two more complex scenarios involved additional violations of strict exogeneity. We found that intrinsic—just as extrinsic—SA can produce incorrect type I error rates, if not explicitly controlled for. Failing to control for intrinsic SA also generated bias in estimates of ceteris paribus effects. However, ESDM from the intrinsic category exhibited consistently good performance in dealing with intrinsic SA across all the scenarios, but suffered other violations of strict exogeneity. Overall, model specification should control for both extrinsic and intrinsic processes generating SA in spatial data to provide reliable type I error rates and unbiased estimates of ceteris paribus effects. Given the likely widespread occurrence in observational spatial data of unknown or unmeasurable processes, ESDM should be a generally preferred starting point to explore the optimal model specification for estimating ceteris paribus effects, with due caution to other violations of strict exogeneity.

Two-scale dispersal estimation for biological invasions via synthetic likelihood.

Biological invasions reshape environments and affect the ecological and economic welfare of states and communities. Such invasions advance on multiple spatial scales, complicating their control. When modeling stochastic dispersal processes, intractable likelihoods and autocorrelated data complicate parameter estimation. As with other approaches, the recent synthetic likelihood framework for stochastic models uses summary statistics to reduce this complexity; however, it additionally provides usable likelihoods, facilitating the use of existing likelihood-based machinery. Here, we extend this framework to parameterize multi-scale spatio-temporal dispersal models and compare existing and newly developed spatial summary statistics to characterize dispersal patterns. We provide general methods to evaluate potential summary statistics and present a fitting procedure that accurately estimates dispersal parameters on simulated data. Finally, we apply our methods to quantify the short and long range dispersal of Chagas disease vectors in urban Arequipa, Peru, and assess the feasibility of a purely reactive strategy to contain the invasion.

Use of control charts with regression analysis for autocorrelated data in the context of logistic financial budgeting

The aim of this paper is to explore whether the use of control charts with regression analysis is an effective way to evaluate financial budget requests (autocorrelated data) in the transport logistics sector. First, the variables are selected. Second, a regression analysis is performed to model the financial variables. Third, three types of traditional control charts are tested (individuals, CUSUM and EWMA), using simulation to monitor the regression scaled residuals. The results show that the individual control chart of 2.7-sigma offers an appropriate performance for the context of this study. This paper provides new evidence regarding a type of variable and context not reported in the literature. In addition, it proposes a control chart approach of scaled regression residuals, with two differentiators: (1) residuals offer better practical interpretation and (2) regressions do not incorporate the time variable, as traditionally occurs, but a missionary process variable (loading units) and a control one.

MALT-45: A 7 mm survey of the southern Galaxy – II. ATCA follow-up observations of 44 GHz class I methanol masers

We detail interferometric observations of 44GHz class I methanol masers detected by MALT-45 (a 7mm unbiased auto-correlated spectral-line Galactic-plane survey) using the Australia Telescope Compact Array. We detect 238 maser spots across 77 maser sites. Using high-resolution positions, we compare the class I CH$_3$OH masers to other star formation maser species, including CS (1-0), SiO $v=0$ and the H53$\alpha$ radio-recombination line. Comparison between the cross- and auto-correlated data has allowed us to also identify quasi-thermal emission in the 44GHz class I methanol maser line. We find that the majority of class I methanol masers have small spatial and velocity ranges ($<$0.5pc and $<$5 km s$^{-1}$), and closely trace the systemic velocities of associated clouds. Using 870$\mu$m dust continuum emission from the ATLASGAL survey, we determine clump masses associated with class I masers, and find they are generally associated with clumps between 1000 and 3000 $M_\odot$. For each class I methanol maser site, we use the presence of OH masers and radio recombination lines to identify relatively evolved regions of high-mass star formation; we find that maser sites without these associations have lower luminosities and preferentially appear toward dark infrared regions.

Quantifying transcription factor-DNA binding in single cells in vivo with photoactivatable fluorescence correlation spectroscopy.

Probing transcription factor (TF)-DNA interactions remains challenging in complex in vivo systems such as mammalian embryos, especially when TF copy numbers and fluorescence background are high. To address this difficulty, fluorescence correlation spectroscopy (FCS) can be combined with the use of photoactivatable fluorescent proteins to achieve selective photoactivation of a subset of tagged TF molecules. This approach, termed paFCS, enables FCS measurements within single cell nuclei inside live embryos, and obtains autocorrelation data of a quality previously only attainable in simpler in vitro cell culture systems. Here, we present a protocol demonstrating the applicability of paFCS in developing mouse embryos by outlining its implementation on a commercial laser-scanning microscope. We also provide procedures for optimizing the photoactivation and acquisition parameters and determining key parameters describing TF-DNA binding. The entire procedure can be performed within ∼2 d (excluding embryo culture time), although the acquisition of each paFCS data set takes only ∼10 min. This protocol can be used to noninvasively reveal cell-to-cell variation in TF dynamics, as well as critical, fate-predicting changes over the course of early embryonic development.

Generalised block bootstrap and its use in meteorology

Abstract. In an earlier paper, Rakonczai et al.(2014) emphasised the importance of investigating the effective sample size in case of autocorrelated data. The simulations were based on the block bootstrap methodology. However, the discreteness of the usual block size did not allow for exact calculations. In this paper we propose a new generalisation of the block bootstrap methodology, which allows for any positive real number as expected block size. We relate it to the existing optimisation procedures and apply it to a temperature data set. Our other focus is on statistical tests, where quite often the actual sample size plays an important role, even in the case of relatively large samples. This is especially the case for copulas. These are used for investigating the dependencies among data sets. As in quite a few real applications the time dependence cannot be neglected, we investigated the effect of this phenomenon on the used test statistic. The critical value can be computed by the proposed new block bootstrap simulation, where the block size is determined by fitting a VAR model to the observations. The results are illustrated for models of the used temperature data.

Monitoring of serially correlated processes using residual control charts

Control charts act as the most important tool for monitoring of process parameters. The assumption of independence that underpins the implementation of the charts is violated when process observations are correlated. The e ect of this issue can lead to the malfunctioning of the usual control charts by causing a large number of false alarms or slowing the detection ability of the chart in unstable situations. In this paper, weinvestigated the performance of the Mixed EWMA-CUSUM and Mixed CUSUM-EWMA charts for the ecient monitoring of autocorrelated data. The charts are applied to the residuals obtained from tting an autoregressive (AR) model to the autocorrelated observations. The performance of these charts is compared with the performances of the residual Shewhart, EWMA, CUSUM, combined Shewhart-CUSUM, and combinedShewhart-EWMA charts. Performance criteria such as Average Run Length (ARL) and Extra Quadratic Loss (EQL) are used for the evaluation and comparison of the charts. Illustrative examples are presented to demonstrate the application of the charts to serially correlated observations

Optimization design of the CUSUM and EWMA charts for autocorrelated processes

AbstractThe traditional control charts produce frequent false alarm signals in the presence of autocorrelation. The implementation of the modified chart scheme is a way of handling the problem of autocorrelation in control charts. In modified charts, the standard control limits of the traditional charts are adjusted to offset the influence because of the autocorrelation. The exponentially weighted moving average– and cumulative sum–modified charts are 2 widely used charts for monitoring autocorrelated data. These charts have design parameters in their formulation, and the choice of these parameters play significant roles in the detection of out‐of‐control situations. In reality, the magnitude of the mean shift is uncertain, and this leads to a difficulty in the choice of design parameters by the practitioner. The use of optimal parameters can enhance process performance in such situations. In this paper, we determine optimal design parameters for the charts using an exhaustive search procedure. In the optimization process, we determine the parameters that produce the smallest extra quadratic loss (EQL) value for each autocorrelation coefficient. This criterion measures the anticipated loss attributed to poor quality in the process. The loss in quality is lowered by minimizing the EQL and the combination of parameters in the chart that yields the smallest EQL has a better detection ability over the entire shift range. For the purpose of this work, we concentrate on autocorrelation that can be specifically modelled with autoregressive models. This article provides the practitioner with optimal parameters that can be used to enhance the overall effectiveness of the charts over an entire shift range.

Choose your neighborhood wisely: implications of subsampling and autocorrelation structure in simultaneous autoregression models for landscape ecology

Context Large datasets that exhibit residual spatial autocorrelation are common in landscape ecology, introducing issues with model inference. Computationally intensive statistical techniques such as simultaneous autoregression (SAR) are used to provide credible inference, yet landscape studies make choices about autocorrelation structure and data reduction techniques without adequate understanding of the consequences for model estimation and inference.

Economic statistical design of ARMA control chart through a Modified Fitness-based Self-Adaptive Differential Evolution

In this paper the economic statistical design of an Auto-Regressive Moving Average (ARMA) control chart for autocorrelated data has been investigated. In particular, the total cost of the chart subject to a constraint on the minimum in-control average run length has been taken into account. Due to the autocorrelation of the data, a simulation approach was adopted to assess both in-control and out-of-control average run lengths as the charting parameters change. In order to select the optimal parameters of the ARMA chart a Modified Fitness-based Self-Adaptive Differential Evolution algorithm, named MF-SADE, has been developed. To validate the effectiveness of MF-SADE in solving the proposed control chart design problem, an extensive comparison campaign involving four different metaheuristics from the relevant literature was arranged. Once the obtained numerical results confirmed the outperformance of SADE, it was used to carry out a sensitivity analysis including parameters of the cost model and those concerned with the underlying process, denoted as u and v. Among the numerous findings, the sensitivity analysis put in evidence the relationship between u, v and the variables of the economical statistical issue under investigation.

How Do We Analyze Auto-correlated Data?

How Do We Analyze Auto-correlated Data?

Assessing thermal comfort and energy efficiency in buildings by statistical quality control for autocorrelated data

In this paper, a case study of performing a reliable statistical procedure to evaluate the quality of HVAC systems in buildings using data retrieved from an ad hoc big data web energy platform is presented. The proposed methodology based on statistical quality control (SQC) is used to analyze the real state of thermal comfort and energy efficiency of the offices of the company FRIDAMA (Spain) in a reliable way. Non-conformities or alarms, and the actual assignable causes of these out of control states are detected. The capability to meet specification requirements is also analyzed. Tools and packages implemented in the open-source R software are employed to apply the different procedures. First, this study proposes to fit ARIMA time series models to CTQ variables. Then, the application of Shewhart and EWMA control charts to the time series residuals is proposed to control and monitor thermal comfort and energy consumption in buildings. Once thermal comfort and consumption variability are estimated, the implementation of capability indexes for autocorrelated variables is proposed to calculate the degree to which standards specifications are met. According with case study results, the proposed methodology has detected real anomalies in HVAC installation, helping to detect assignable causes and to make appropriate decisions. One of the goals is to perform and describe step by step this statistical procedure in order to be replicated by practitioners in a better way.

Drinking water quality control: control charts for turbidity and pH

Water treatment processes are required to be in statistical control and capable of meeting drinking water specifications. Control charts are used to monitor the stability of quality parameters by distinguishing the in-control and out-of-control states. The basic assumption in standard applications of control charts is that observed data from the process are independent and identically distributed. However, the independence assumption is often violated in chemical processes such as water treatment. Autocorrelation, a measure of dependency, is a correlation between members of a series arranged in time. The residuals obtained from an autoregressive integrated moving averages (ARIMA) time series model plotted on a standard control chart is used to overcome the misleading of standard control charts in the case of autocorrelation. In this study, a special cause control (SCC) chart, also called a chart of residuals from the fitted ARIMA model, has been used for turbidity and pH data from a drinking water treatment plant in Samsun, Turkey. ARIMA (3,1,0) for turbidity and ARIMA (1,1,1) for pH were determined as the best time series models to remove autocorrelation. The results showed that the SCC chart is more appropriate for autocorrelated data to evaluate the stability of the water treatment process, since it provides a higher probability of coverage than an individual control chart.