Sampling Error Variance Research Articles

Estimating and Calibrating Markov Chain Sample Error Variance

Markov chain Monte Carlo (MCMC) methods are a powerful and versatile tool with applications spanning a wide spectrum of fields, including Bayesian inference, computational biology, and physics. One of the key challenges in applying MCMC algorithms is to deal with estimation error. The main result in this article is a closed form, non-asymptotic solution for the sample error variance of a single MCMC  estimate. Importantly, this result assumes that the state-space is finite and discrete. We demonstrate with examples how this result can help estimate and calibrate MCMC estimation error variance in the more general case, when the state-space is continuous and/or unbounded.

Correcting for measurement error under meta-analysis of z-transformed correlations.

This study mainly concerns correction for measurement error using the meta-analysis of Fisher's z-transformed correlations. The disattenuation formula of Spearman (American Journal of Psychology, 15, 1904, 72) is used to correct for individual raw correlations in primary studies. The corrected raw correlations are then used to obtain the corrected z-transformed correlations. What remains little studied, however, is how to best correct for within-study sampling error variances of corrected z-transformed correlations. We focused on three within-study sampling error variance estimators corrected for measurement error that were proposed in earlier studies and is proposed in the current study: (1) the formula given by Hedges (Test validity, Lawrence Erlbaum, 1988) assuming a linear relationship between corrected and uncorrected z-transformed correlations (linear correction), (2) one derived by the first-order delta method based on the average of corrected z-transformed correlations (stabilized first-order correction), and (3) one derived by the second-order delta method based on the average of corrected z-transformed correlations (stabilized second-order correction). Via a simulation study, we compared performance of these estimators and the sampling error variance estimator uncorrected for measurement error in terms of estimation and inference accuracy of the mean correlation as well as the homogeneity test of effect sizes. In obtaining the corrected z-transformed correlations and within-study sampling error variances, coefficient alpha was used as a common reliability coefficient estimate. The results showed that in terms of the estimated mean correlation, sampling error variances with linear correction, the stabilized first-order and second-order corrections, and no correction performed similarly in general. Furthermore, in terms of the homogeneity test, given a relatively large average sample size and normal true scores, the stabilized first-order and second-order corrections had type I error rates that were generally controlled as well as or better than the other estimators. Overall, stabilized first-order and second-order corrections are recommended when true scores are normal, reliabilities are acceptable, the number of items per psychological scale is relatively large, and the average sample size is relatively large.

Homogenization of daily temperatures using covariates and statistical learning—The case of parallel measurements

AbstractA data driven method based on generalized additive modelling (GAM) has been developed for homogenizing daily minimum and maximum temperature (TN, TX) series using parallel measurements and covariates. The method is applied to two coastal and two inland stations in the Netherlands. Between 1950 and 1972, these stations were relocated from cities to airports, accompanied by parallel measurement of at least 5 years at the old and new sites. Separating these parallel measurements in training and test data, the method compares numerous models involving covariates like the wind vector, cloudiness, specific humidity and sea surface temperature, and selects a model for each station. The resulting models offer an improvement compared to models based on temperature and season only: seasonal dependence is largely replaced by dependence on physical quantities. However, quantitatively, the impact is not large in the cases studied. One of the reasons might be that some covariates have only been measured at specific times not coinciding with the occurrences of the temperature minima or maxima. Additional benefits of the method are robustness and estimation of the sampling error variance of the daily homogenized daily temperature values.

A New Weighting Method in Meta-Analysis: The Weighting with Reliability Coefficient

This study aimed to investigate the impact of various weighting methods for effect sizes on the outcomes of meta-analyses that examined the effects of the 5E teaching method on academic achievement in science education. Two effect size weighting methods were explored: one based on the inverse of the sampling error variance and the other utilizing the reliability of measures in primary studies. The study also assessed the influence of including gray literature on the meta-analysis results, considering factors such as high heterogeneity and publication bias. The research followed a basic research design and drew data from 112 studies, encompassing a total of 149 effect sizes. An exhaustive search of databases and archives, including Google Scholar, Dergipark, HEI Thesis Center, Proquest, Science Direct, ERIC, Taylor & Francis, EBSCOhost, Web of Science, and five journals was conducted to gather these studies. Analyses were performed by utilizing the CMA v2 software and employing the random effects model. The findings demonstrated divergent outcomes between the two weighting methods—weighting by reliability coefficient yielded higher overall effect sizes and standard errors compared to weighting by inverse variance. Ultimately, the inclusion of gray literature was found to not significantly impact any of the weighting methods employed.

Accounting for year effects and sampling error in temporal analyses of invertebrate population and biodiversity change: a comment on Seibold et al. 2019

Abstract An accumulating number of studies are reporting severe insect declines. These studies aim to quantify temporal changes in invertebrate populations and community composition and attribute them to anthropogenic drivers. Seibold et al. 2019 (Nature, 574, 671–674) analysed arthropod biomass, abundance and species richness from forest and grassland plots in a region of Germany and reported declines of up to 78% between 2008 and 2018. However, their analysis did not account for the confounding effects of temporal pseudoreplication. We show that simply by including a year random effect in the statistical models and thereby accounting for the common conditions experienced by proximal sites in the same years, four of the five reported declines become non‐significant out of six tests overall. To place recent estimates of insect trends in a broader context, we analysed invertebrate biomass, abundance and richness from 640 time series from 1167 sites around the world. We found that the average trends across the terrestrial and freshwater realms were not significantly distinguishable from no net change. Shorter time series that are likely most affected by sampling error variance – such as those in Seibold et al. 2019 (Nature, 574, 671–674) – yielded the most extreme decline and increase estimates. We suggest that the media uptake of negative trends from short time series may be serving to exaggerate the ‘insect Armageddon’ and could undermine public confidence in research. We advocate that future research uses appropriate model structures to build a more robust understanding of biodiversity change.

Inferring Phenotypic Trait Evolution on Large Trees With Many Incomplete Measurements

Comparative biologists are often interested in inferring covariation between multiple biological traits sampled across numerous related taxa. To properly study these relationships, we must control for the shared evolutionary history of the taxa to avoid spurious inference. An additional challenge arises as obtaining a full suite of measurements becomes increasingly difficult with increasing taxa. This generally necessitates data imputation or integration, and existing control techniques typically scale poorly as the number of taxa increases. We propose an inference technique that integrates out missing measurements analytically and scales linearly with the number of taxa by using a post-order traversal algorithm under a multivariate Brownian diffusion (MBD) model to characterize trait evolution. We further exploit this technique to extend the MBD model to account for sampling error or nonheritable residual variance. We test these methods to examine mammalian life history traits, prokaryotic genomic and phenotypic traits, and HIV infection traits. We find computational efficiency increases that top two orders-of-magnitude over current best practices. While we focus on the utility of this algorithm in phylogenetic comparative methods, our approach generalizes to solve long-standing challenges in computing the likelihood for matrix-normal and multivariate normal distributions with missing data at scale. Supplementary materials for this article are available online.

The accuracy of dominance analysis as a metric to assess relative importance: The joint impact of sampling error variance and measurement unreliability.

Dominance analysis (DA) has been established as a useful tool for practitioners and researchers to identify the relative importance of predictors in a linear regression. This article examines the joint impact of two common and pervasive artifacts-sampling error variance and measurement unreliability-on the accuracy of DA. We present Monte Carlo simulations that detail the decrease in the accuracy of DA in the presence of these artifacts, highlighting the practical extent of the inferential mistakes that can be made. Then, we detail and provide a user-friendly program in R (R Core Team, 2017) for estimating the effects of sampling error variance and unreliability on DA. Finally, by way of a detailed example, we provide specific recommendations for how researchers and practitioners should more appropriately interpret and report results of DA. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Trends of Sampling Error in Surface Air Temperature on the Tibetan Plateau during Recent Decades

Understanding errors in surface air temperature (SAT) data and related uncertainties is crucial for climate studies because of their impact on the accuracy of statistical inferences in scientific conclusions. In recent decades, considerable research has focused on the trends and evolution of SAT on the Tibetan Plateau (TP). However, assessment of the uncertainties in SAT change on the TP has not been done adequately, which is of considerable importance for climate research. Using station-observed SAT data from the TP, this study estimates long-term variations and trends of sampling error variances in gridded monthly SAT data over recent decades. Results revealed large sampling error variances in northern and western parts of the TP but small variances in eastern, southern, and central areas. The sampling error variances also exhibited strong monthly variations with maximum errors in winter and minimum values in summer. Furthermore, spatial distributions of the trends of seasonal and annual mean sampling error variances were found distributed unevenly with decreasing trends found mainly in central and southern parts of the TP and increasing trends in northeastern, southeastern, and northwestern areas. Additionally, differences were also found in the trends of seasonal and annual mean sampling error variances on various timescales.

Supplemental Material for The Accuracy of Dominance Analysis as a Metric to Assess Relative Importance: The Joint Impact of Sampling Error Variance and Measurement Unreliability

Supplemental Material for The Accuracy of Dominance Analysis as a Metric to Assess Relative Importance: The Joint Impact of Sampling Error Variance and Measurement Unreliability

The effects of pilates on mental health outcomes: A meta-analysis of controlled trials

ObjectiveThis meta-analysis estimated the population effect size for Pilates effects on mental health outcomes. Data sourcesArticles published prior to August 2017 were located with searches of Pubmed, Medline, Cinahl, SportDiscus, Science Direct, PsychINFO, Web of Science, and Cochrane Controlled Trial Register using combinations of: Pilates, Pilates method, mental health, anxiety, and depression. Study selectionEight English-language publications that included allocation to a Pilates intervention or non-active control and a measure of anxiety and/or depressive symptoms at baseline and after the Pilates intervention were selected. Data extractionParticipant and intervention characteristics, anxiety and depressive symptoms and other mental health outcomes, including feelings of energy and fatigue and quality of life, were extracted. Hedges’ d effect sizes were computed, study quality was assessed, and random effects models estimated sampling error and population variance. Data synthesisPilates resulted in significant, large, heterogeneous reductions in depressive (Δ = 1.27, 95%CI: 0.44, 2.09; z = 3.02, p ≤ 0.003; N = 6, n = 261) and anxiety symptoms (Δ = 1.29, 95%CI: 0.24, 2.33; z = 2.40, p ≤ 0.02; N = 5, n = 231) and feelings of fatigue (Δ = 0.93, 95%CI: 0.21, 1.66; z = 2.52, p ≤ 0.012; N = 3, n = 161), and increases in feelings of energy (Δ = 1.49, 95%CI: 0.67, 2.30; z = 3.57, p < 0.001; N = 2, n = 116). ConclusionsThough this review included a small number of controlled trials with small sample sizes and non-active control conditions of variable quality, the available evidence reviewed here supports that Pilates improves mental health outcomes. Rigorously designed randomized controlled trials, including those that compare Pilates to other empirically-supported therapies, are needed to better understand Pilates’ clinical effectiveness and plausible mechanisms of effects.

The Effects of Exercise Training on Anxiety in Fibromyalgia Patients: A Meta-analysis.

This study aimed to evaluate the effects of exercise training on anxiety symptoms in patients with FM and to examine whether variables of theoretical or practical importance moderate the estimated mean effect. Twenty-five effects were derived from 10 articles published before June 2016 located using Google Scholar, MEDLINE, PsycINFO, PubMed, and Web of Science. Trials involved 595 patients with FM (mean age = 47.6 yr, 97.5% female) and included both randomization to exercise training (n = 297) or a nonexercise control condition (n = 298) and an anxiety outcome measured at baseline and during and/or after exercise training. Hedges' d effect sizes were computed, data for moderator variables were extracted, and random effects models were used to estimate sampling error and population variance for all analyses. Meta-regression quantified the extent to which patient and trial characteristics moderated the mean effect. Exercise training significantly reduced anxiety symptoms by a mean effect Δ of 0.28 (95% confidence interval [CI] = 0.16-0.40). No significant heterogeneity was observed (Q24 = 30.79, P = 0.16, I = 25.29%). Program duration (β = 1.44, z = 2.50, P ≤ 0.01) was significantly related to the overall effect, with significantly larger anxiety improvements resulting from programs lasting greater than 26 wk (Δ = 0.35, 95% CI = 0.05-0.66) compared with those lasting less than 26 wk (Δ = 0.26, 95% CI = 0.13-0.39). Exercise training improves anxiety symptoms among FM patients. The findings also suggest that larger anxiety symptom reductions will be achieved by focusing on longer exercise programs while promoting long-term adherence. Future well-designed investigations are required to examine the potential moderating effect of pain-related improvements in FM patients.

Internet financial reporting determinants: a meta-analytic review

PurposeThis study aims to examine the association between firm size, profitability, leverage, auditor type and internet reporting and investigate the moderating effect of legal system, investor protection, masculinity, economic development, construction of disclosure index and measurement proxies of independent variables.Design/methodology/approachThis study conducts a meta-analytic review for 59 research papers to synthesise quantitatively the results of previous literature on the determinants of internet reporting. This study uses Hunter and Schmidt’s (2004) procedures to conduct the analysis. There are three main procedures to be followed: calculating the weighted effect size, calculating observed correlation variance and sampling error variance and, finally, testing for homogeneity and moderating effects.FindingsThe results indicate a significant positive association between firm size, profitability, leverage, auditor type and internet reporting. The results confirm the prediction of agency theory, signalling theory, political cost hypothesis and diffusion of innovation theory. Moreover, the results show that investor protection, masculinity, economic development, construction of disclosure index and measurement proxies for independent variables moderate the association between profitability, leverage and internet reporting.Research limitations/implicationsThis study suffers from some limitations. First, corporate governance variables such as board size, role duality and board independence were not included in the analysis due to the limited number of studies that discuss the association between corporate governance and internet reporting. Second, the study does not control for the endogeneity problem.Practical implicationsFuture research has to consider the moderating effect of investor protection, masculinity, economic development, construction of disclosure index and measurement proxies for independent variables on the association between corporate characteristics and internet reporting. Future research can extend this work by examining the association between corporate governance, ownership structure and internet reporting. The findings regarding the determinants of internet reporting should be on concern of regulatory authorities.Originality/valueThis study contributes to and extends previous meta-analysis literature by examining internet reporting determinants, as previous financial reporting meta-analysis studies give no attention to internet reporting.

Trends and uncertainties in surface air temperature over the Tibetan Plateau, 1951–2013

Trends and uncertainties of surface air temperature over the Tibetan Plateau (TP) are evaluated by using observations at 100 meteorological stations during the period 1951–2013. The sampling error variances of gridded monthly data are estimated for every month and every grid box of data. The gridded data and their sampling error variances are used to calculate TP averages, their trends, and associated uncertainties. It is shown that large sampling error variances dominate northern and western TP, while small variances appear over southern and eastern TP. Every month from January to December has a positive linear trend during the study period. February has the largest trend of 0.34 ± 0.18°C (10 yr)–1, and April the smallest at 0.15 ± 0.11°C (10 yr)–1. The uncertainties decrease steadily with time, implying that they are not large enough to alter the TP warming trend.

Accounting for nonsampling error in estimates of HIV epidemic trends from antenatal clinic sentinel surveillance.

ObjectivesThe aim of the study was to propose and demonstrate an approach to allow additional nonsampling uncertainty about HIV prevalence measured at antenatal clinic sentinel surveillance (ANC-SS) in model-based inferences about trends in HIV incidence and prevalence.DesignMathematical model fitted to surveillance data with Bayesian inference.MethodsWe introduce a variance inflation parameter that accounts for the uncertainty of nonsampling errors in ANC-SS prevalence. It is additive to the sampling error variance. Three approaches are tested for estimating using ANC-SS and household survey data from 40 subnational regions in nine countries in sub-Saharan, as defined in UNAIDS 2016 estimates. Methods were compared using in-sample fit and out-of-sample prediction of ANC-SS data, fit to household survey prevalence data, and the computational implications.ResultsIntroducing the additional variance parameter increased the error variance around ANC-SS prevalence observations by a median of 2.7 times (interquartile range 1.9–3.8). Using only sampling error in ANC-SS prevalence ( ), coverage of 95% prediction intervals was 69% in out-of-sample prediction tests. This increased to 90% after introducing the additional variance parameter . The revised probabilistic model improved model fit to household survey prevalence and increased epidemic uncertainty intervals most during the early epidemic period before 2005. Estimating did not increase the computational cost of model fitting.Conclusions: We recommend estimating nonsampling error in ANC-SS as an additional parameter in Bayesian inference using the Estimation and Projection Package model. This approach may prove useful for incorporating other data sources such as routine prevalence from Prevention of mother-to-child transmission testing into future epidemic estimates.

Dealing with data conflicts in statistical inference of population assessment models that integrate information from multiple diverse data sets

Contemporary fisheries stock assessments often use multiple diverse data sets to extract as much information as possible about biological and fishery processes. However, models are, by definition, simplifications of reality and, therefore, misspecified. Model misspecification can cause degradation of results when multiple data sets are analyzed simultaneously. The process, observation, and sampling components of the model must all be, at least, approximately correct to minimize bias. Unfortunately, even the basic processes that are usually considered well understood (e.g., growth and selectivity) are misspecified in most, if not all, stock assessments. These misspecified processes, in combination with use of composition data, result in biased estimates of absolute abundance and abundance trends, which are often evident as “data conflicts.” This is compounded by over-weighting of composition data in many assessments owing to misuse of data-weighting approaches. The ‘law of conflicting data’ states that since data are facts, conflicting data implies model misspecification, but must be interpreted in the context of random sampling error. Down-weighting (or dropping) conflicting data is not necessarily appropriate because it may not resolve the model misspecification. Model misspecification and process variation can be accounted for in the variance parameters of the likelihoods (sampling error), but it is unclear when, or even if, this is appropriate. The appropriate method to deal with data conflicts depends on whether it is caused by random sampling error, process variation, observation model misspecification, or misspecification of the system (dynamics) model. Diagnostic approaches are urgently needed to evaluate goodness of fit and to identify model misspecification. We recommend external estimation of the sampling error variance in likelihood functions, modelling process variation in integrated models, and internal estimation of the standard deviation of the process variation. The required statistical framework is computationally intensive, but practical approximations are available, computational algorithms are being improved, and computer power is increasing. We provide a framework for model development that identifies and corrects model misspecification and illustrate the framework, using simulated data.

Small Area Estimation of Poverty Incidence with Sampling Error Variances Through Generalized Variance Function

Small area estimation based on area level models, particularly the EBLUP method, typically assumes that sampling error variances of the direct survey small area estimates are known. In practice, the sampling error variances are unknown. This paper generates EBLUP estimates of poverty incidence when the sampling error variances are estimated using the generalized variance function (GVF) approach. The precision of the EBLUP estimates is determined using a modified version of the Prasad-Rao MSPE estimator. The modification is made by adding an extra term that would account the uncertainty associated with estimating the sampling error variances. The performance of the modified Prasad-Rao estimator relative to the commonly used Prasad-Rao estimator is evaluated through a simulation study. Results have shown that the modified Prasad-Rao MSPE estimator has relatively greater bias than the commonly used Prasad-Rao MSPE estimator, particularly for small samples. A slight gain in precision is observed when using the modified PR MSPE estimator, especially for large samples. Moreover, the findings imply that estimating sampling error variances using GVF models can be a very useful strategy in the application of EBLUP small area estimation, most particularly in poverty incidence estimation.

Estimation of sampling error uncertainties in observed surface air temperature change in China

This study examines the sampling error uncertainties in the monthly surface air temperature (SAT) change in China over recent decades, focusing on the uncertainties of gridded data, national averages, and linear trends. Results indicate that large sampling error variances appear at the station-sparse area of northern and western China with the maximum value exceeding 2.0 K2 while small sampling error variances are found at the station-dense area of southern and eastern China with most grid values being less than 0.05 K2. In general, the negative temperature existed in each month prior to the 1980s, and a warming in temperature began thereafter, which accelerated in the early and mid-1990s. The increasing trend in the SAT series was observed for each month of the year with the largest temperature increase and highest uncertainty of 0.51 ± 0.29 K (10 year)−1 occurring in February and the weakest trend and smallest uncertainty of 0.13 ± 0.07 K (10 year)−1 in August. The sampling error uncertainties in the national average annual mean SAT series are not sufficiently large to alter the conclusion of the persistent warming in China. In addition, the sampling error uncertainties in the SAT series show a clear variation compared with other uncertainty estimation methods, which is a plausible reason for the inconsistent variations between our estimate and other studies during this period.

On This Issue's Cover

On This Issue's Cover

Effects of Pharmacotherapy on Combat-Related PTSD, Anxiety, and Depression: A Systematic Review and Meta-Regression Analysis.

The efficacy of pharmacotherapy for PTSD, anxiety, and depression among combat veterans is not well-established.ObjectivesTo estimate the effect of pharmacotherapy on PTSD, anxiety, and depression among combat veterans; to determine whether the effects varied according to patient and intervention characteristics; and to examine differential effects of pharmacotherapy on outcomes.Materials and MethodsGoogle Scholar, PILOTS, PsycINFO, PubMed, and Web of Science databases were searched through November 2014. Searches resulted in eighteen double-blind, placebo controlled trials of 773 combat veterans diagnosed with PTSD and included only validated pre- and post-intervention PTSD and anxiety or depression measures. Authors extracted data on effect sizes, moderators, and study quality. Hedges’ d effect sizes were computed and random effects models estimated sampling error and population variance. The Johnson-Neyman procedure identified the critical points in significant interactions to define regions of significance.ResultsPharmacotherapy significantly reduced (Δ, 95%CI) PTSD (0.38, 0.23-0.52), anxiety (0.42, 0.30-0.54), and depressive symptoms (0.52, 0.35-0.70). The effects of SSRIs and tricyclic antidepressants on PTSD were greater than other medications independent of treatment duration. The effect of SSRIs and tricyclic antidepressants were greater than other medications up to 5.2 and 13.6 weeks for anxiety and depression, respectively. The magnitude of the effect of pharmacotherapy on concurrently-measured PTSD, anxiety, and depression did not significantly differ.ConclusionsPharmacotherapy reduced PTSD, anxiety, and depressive symptoms in combat veterans. The effects of SSRIs and tricyclic antidepressants were greater for PTSD and occurred quicker for anxiety and depression than other medications.

Abusive Supervision: A Meta-Analysis and Empirical Review

We conducted a meta-analysis and empirical review of abusive supervision research in order to derive meta-analytic population estimates for the relationships between perceptions of abusive supervision and numerous demographic, justice, individual difference, leadership, and outcome variables. The use of psychometric correction enabled us to provide weighted mean correlations and population correlation estimates that accounted for attenuation due to measurement error and sampling error variance. Also, we conducted sensitivity analyses that removed the effects of large samples from analyses. Then, we conducted subgroup analyses using samples drawn from the United States to provide population correlation estimates that corrected for attenuation due to measurement error, sampling error variance, and indirect range restriction. Finally, we examined measurement artifacts resulting from various adaptations of Tepper’s abusive supervision measure. The results reveal that although the associations between perceptions of abusive supervision and outcome variables appear to be universally negative, the magnitude of the relationships between perceptions of abusive supervision and antecedent and outcome variables varies according to the design features of studies. Contributions to theory and practice, strengths and limitations, and directions for future research are discussed.