Hypothesis testing under uniform-block covariance structures

Generalized linear model analyses of repeated measurements typically rely on simplifying mathematical models of the error covariance structure for testing the significance of differences in patterns of change across time. The robustness of the tests of significance depends, not only on the degree of agreement between the specified mathematical model and the actual population data structure, but also on the precision and robustness of the computational criteria for fitting the specified covariance structure to the data. Generalized estimating equation (GEE) solutions utilizing the robust empirical sandwich estimator for modeling of the error structure were compared with general linear mixed model (GLMM) solutions that utilized the commonly employed restricted maximum likelihood (REML) procedure. Under the conditions considered, the GEE and GLMM procedures were identical in assuming that the data are normally distributed and that the variance‐covariance structure of the data is the one specified by the user.The question addressed in this article concerns relative sensitivity of tests of significance for treatment effects to varying degrees of misspecification of the error covariance structure model when fitted by the alternative procedures. Simulated data that were subjected to monte carlo evaluation of actual Type I error and power of tests of the equal slopes hypothesis conformed to assumptions of ordinary linear model ANOVA for repeated measures except for autoregressive covariance structures and missing data due to dropouts. The actual within‐groups correlation structures of the simulated repeated measurements ranged from AR(1) to compound symmetry in graded steps, whereas the GEE and GLMM formulations restricted the respective error structure models to be either AR(1), compound symmetry (CS), or unstructured (UN). The GEE‐based tests utilizing empirical sandwich estimator criteria were documented to be relatively insensitive to misspecification of the covariance structure models, whereas GLMM tests which relied on restricted maximum likelihood (REML) were highly sensitive to relatively modest misspecification of the error correlation structure even though normality, variance homogeneity, and linearity were not an issue in the simulated data.Goodness‐of‐fit statistics were of little utility in identifying cases in which relatively minor misspecification of the GLMM error structure model resulted in inadequate alpha protection for tests of the equal slopes hypothesis. Both GEE and GLMM formulations that relied on unstructured (UN) error model specification produced nonconservative results regardless of the actual correlation structure of the repeated measurements. A random coefficients model produced robust tests with competitive power across all conditions examined. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Because measurement scales of observed variables in social and behavioral sciences are often arbitrary, and because the sample correlation matrix, unlike the sample covariance matrix, is scale independent, analyses of correlation structures based on sample correlation matrices are desirable, and often done (e.g., Velicer & Jackson, 1990; Goffin & Jackson, 1992). The problem is that most popular covariance structure software is built around the sampling theory designed to analyze covariance structures. This theory is based on the multivariate sampling distribution of a covariance matrix, which is not the same as that of a correlation matrix. A number of papers have addressed problems relevant to this issue in the contexts of exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). These address problems of parameter estimation, tests of hypotheses, and standard error estimation. Our main focus here will be on the problem of standard error estimation. This is critical to accurate decision-making regarding the interpretation of constructs and the necessity of parameters such as factor loadings as indicators of such constructs. Since typical practice in factor analysis is to interpret a factor by its highly loading variables in a standardized solution, it is also important to verify that such loadings are statistically significantly different from zero. For example, if one uses confirmatory factor analysis in a sequential system for personality scale construction (e.g., Jackson, 1970), it would be essential to be able to determine whether or not the loading of a particular item on the trait factor that it is hypothesized to reflect is significantly different from zero. Furthermore, confirmatory factor analyses of the higher-order structure of personality scales (e.g., Jackson, Paunonen, Fraboni, & Goffin, 1996) could not be meaningfully interpreted in the absence of accurate estimates of the standard errors of the parameters. The previous examples are only two of a multitude of possible examples where the estimation of standard errors assumes a prominent role in the evaluation of construct validity. In this paper, we provide some new methods to achieve this purpose.

Longitudinal data, comprising repeated measurements of the same individuals over time, arise frequently in cardiology and the biomedical sciences in general. For example, Frison and Pocock1 used repeated measurements of the liver enzyme creatine kinase in serum of cardiac patients to study changes in liver function over a 12-month study period. The main goal, indeed the raison d’etre , of a longitudinal study is characterization of changes in the response of interest over time. Ordinarily, changes in the response are also related to selected covariates. For example, Frison and Pocock1 compared changes in creatine kinase between patients randomized to active drug and placebo. The past 25 years have witnessed remarkable developments in statistical methods for the analysis of longitudinal data. Despite these important advances, researchers in the biomedical sciences have been somewhat slow to adopt these methods and often rely on statistical techniques that fail to adequately account for longitudinal study designs. The goal of the present report is to provide an overview of some recently developed methods for longitudinal analyses that are more appropriate, with a focus on 2 methods for continuous responses: the analysis of response profiles and linear mixed-effects models. The analysis of response profiles is better suited to settings with a relatively small number of repeated measurements, obtained on a common set of occasions, whereas linear mixed-effects models are suitable in more general settings. Before describing these methods, we review some of the defining features of longitudinal studies and highlight the main aspects of longitudinal data that complicate their analysis. ### Covariance Structure A common feature of repeated measurements on an individual is correlation; that is, knowledge of the value of the response on one occasion provides information about the likely value of the response on a future occasion. Another common feature of longitudinal data is heterogeneous …

The coordinate free (geometric) approach to univariate linear models has added both insight and understanding to the problems of Gauss Markov (GM) estimation and hypothesis testing. One of the initial papers emphasizing the geometric aspects of univariate linear models is Kruskal's (1961). The coordinate free approach is used in this paper to treat GM estimation in a multivariate analysis context. In contrast to the univariate situation, a central question for multivariate linear models is the existence of GM estimates. Of course, it is the more complicated covariance structure in the multivariate case that creates the concern over the existence of GM estimates. As the emphasis is on GM estimation, first and second moment assumptions (as opposed to distributional assumptions) play the key role. Classical results for the univariate linear model are outlined in Section 1. In addition, a recent theorem due to Kruskal (1968) concerning the equality of GM and Least Squares (LS) estimates is discussed. A minor modification of Kruskal's result gives a very useful, necessary and sufficient condition for the existence of GM estimators for arbitrary covariance structures and a fixed regression manifold. In Section 2, the outer product of two vectors and the Kronecker product of linear transformations is discussed and applied to describe the covariance structure of a random matrix. This application includes the case of a random sample from a multivariate population with covariance matrix $\Sigma > 0 ("\Sigma > 0"$ means that $\Sigma$ is positive definite). The question of GM estimation in the standard multivariate linear model is taken up in Section 3. This model is described as follows: a random matrix $Y: n \times p$, whose rows are uncorrelated and each row has a covariance matrix $\Sigma > 0$, is observed. The mean matrix of $Y, \mu$, is assumed to have the form $\mu = ZB$ where $Z: n \times q$ is known and of rank $q$, and $B: q \times p$ is a matrix of regression coefficients. For this model, GM estimators for $\mu$ and $B$ exist and are well known (see Anderson (1958) chapter 8). The main result in Section 3 establishes a converse to this classical result. Explicitly, let $Y$ have the covariance structure as above and assume $\Omega$ is a fixed regression manifold. It is shown that if a GM estimator for $\mu\in\Omega$ exists, then each element $\mu\in\Omega$ can be written as $\mu = ZB$ where $Z: n \times q$ is fixed and $B: q \times p$ ranges over all $q \times p$ real matrices. The results in Section 4 and Section 5 are similar to the main result of Section 3. A complete description of all regression manifolds for which GM estimators exist is given for two different kinds of covariance assumptions concerning $\Sigma$ ($\Sigma$ as above). In Section 4, it is assumed that $\Sigma$ has a block diagonal form with two blocks. Section 5 is concerned with the case when $\Sigma$ has the so-called intra-class correlation form.

This study tests the dimensionality hypothesis, according to which diverse epistemically unfounded beliefs share a single underlying liability in addition to domain-specific components. We focus on conspiracy, paranormal, pseudoscientific, religious, and pseudo-profound ("bullshit") beliefs and examine whether their covariance structure is best captured by a common factor plus residual domain-specific factors. Using a nationally representative Russian sample (N = 1268), we modeled the five belief domains with competing confirmatory factor analysis models. We compared one-factor, correlated factors, higher order, and bifactor specifications, with the bifactor model prespecified as the focal test of the dimensionality hypothesis. We then related the general and domain-specific factors to cognitive and motivational antecedents. A prespecified bifactor solution, in which a general factor explained about half of the common variance while specific factors captured residual domain structure, provided the best fit. Conventional domain scores may systematically confound the general liability with domain-specific variance, potentially distorting cross-domain comparisons and inferences about domain-specific antecedents. Some cognitive and motivational antecedents emerged as central correlates of the general factor, whereas conspiracy, religious, pseudoscientific, paranormal, and pseudo-profound beliefs each showed distinct additional profiles. The findings support a dimensional view of epistemically unfounded beliefs anchored in a common cognitive-motivational architecture. We discuss the general factor as potentially reflecting an evolved predisposition favoring Type I errors (the "smoke detector principle") and archaic magical thinking, plausibly implemented at the proximate level in low-dimensional psychological "conceptual spaces" that align heterogeneous beliefs along a shared axis of receptivity to epistemically unfounded claims. Explicitly modeling this structure is crucial for valid measurement, comparison, and targeted intervention across epistemically unfounded belief domains, as opposed to relying on undifferentiated composite scores.

Jackson and Bradley (1959, 1961a, 1961b) developed and studied a sequential (multivariate) $T^2$ test of hypotheses on a vector of means, and an analogous $\chi^2$ test for known covariance structure. The present paper presents the results of Monte Carlo sampling on the operating characteristics and average sample numbers (ASN) of these tests. Consideration is restricted to the behavior of these tests at specific null and alternate hypotheses $(H_0$ and $H_1)$ with nominal $\alpha$ and $\beta$ errors of .05. The empirical $\alpha$ and $\beta$ errors are, in general, less than .05 and appear to decrease as the number of variables increases. The empirical ASN are appreciably smaller than the corresponding fixed sample sizes, and approximate the ASN that Jackson and Bradley obtained using Bhate's conjecture. The estimation of the fixed sample sizes were based on the nominal $\alpha$ and $\beta$ errors of .05 while the sequential test ASN were, of course, associated with the resulting smaller error probabilities. Thus the true advantage of the sequential test is understated by the above sample size comparisons. This study investigates the behavior of the sequential test at $H_0$ and $H_1$ only. A similar study involving points between $H_0$ and $H_1$ would be of definite values for (1) ascertaining whether the advantages of the sequential procedure hold under situations other than $H_0$ and $H_1$, and (2) suggesting methods to overcome the conservatism of the test as it now stands.

SUMMARY There is concern that the usual analysis of crossover designs with more than two treatments is subject to bias due to correlations between the measurements on the same experimental units. It has been shown by Kunert in the special case of balanced Latin squares that this bias can be present but that it is limited. Extending the work of Kunert we show in the present paper that there is a constant X* which guarantees that the estimate for the variance of any treatment contrast from the usual model multiplied by X* has an expectation which is at least as big as the true variance. This result holds for any within-unit covariance structure and it is valid for a class of commonly applied designs, allowing for fewer periods than treatments. The constant X* depends on the number of units, periods and treatments but not on the data or the unknown covariance matrix. We also deal with the effect that our result can have on tests for hypotheses about treatment contrasts.

Factor analysis, path analysis, structural equation modeling, and related multivariate statistical methods are based on maximum likelihood or generalized least squares estimation developed for covariance structure models. Large-sample theory provides a chi-square goodness-of-fit test for comparing a model against a general alternative model based on correlated variables. This model comparison is insufficient for model evaluation: In large samples virtually any model tends to be rejected as inadequate, and in small samples various competing models, if evaluated, might be equally acceptable. A general null model based on modified independence among variables is proposed to provide an additional reference point for the statistical and scientific evaluation of covariance structure models. Use of the null model in the context of a procedure that sequentially evaluates the statistical necessity of various sets of parameters places statistical methods in covariance structure analysis into a more complete framework. The concepts of ideal models and pseudo chi-square tests are introduced, and their roles in hypothesis testing are developed. The importance of supplementing statistical evaluation with incremental fit indices associated with the comparison of hierarchical models is also emphasized. Normed and nonnormed fit indices are developed and illustrated.

When designing a study that uses structural equation modeling (SEM), an important task is to decide an appropriate sample size. Historically, this task is approached from the power analytic perspective, where the goal is to obtain sufficient power to reject a false null hypothesis. However, hypothesis testing only tells if a population effect is zero and fails to address the question about the population effect size. Moreover, significance tests in the SEM context often reject the null hypothesis too easily, and therefore the problem in practice is having too much power instead of not enough power. An alternative means to infer the population effect is forming confidence intervals (CIs). A CI is more informative than hypothesis testing because a CI provides a range of plausible values for the population effect size of interest. Given the close relationship between CI and sample size, the sample size for an SEM study can be planned with the goal to obtain sufficiently narrow CIs for the population model parameters of interest. Latent curve models (LCMs) is an application of SEM with mean structure to studying change over time. The sample size planning method for LCM from the CI perspective is based on maximum likelihood and expected information matrix. Given a sample, to form a CI for the model parameter of interest in LCM, it requires the sample covariance matrix S, sample mean vector , and sample size N. Therefore, the width (w) of the resulting CI can be considered a function of S, , and N. Inverting the CI formation process gives the sample size planning process. The inverted process requires a proxy for the population covariance matrix Σ, population mean vector μ, and the desired width ω as input, and it returns N as output. The specification of the input information for sample size planning needs to be performed based on a systematic literature review. In the context of covariance structure analysis, Lai and Kelley (2011) discussed several practical methods to facilitate specifying Σ and ω for the sample size planning procedure.

This paper reviews Joreskog's model for the analysis of covariance structures by first introducing the simpler case of confirmatory factor analysis. The mathematical results necessary for estimation and hypothesis testing are presented in a way which should be more accessible to sociologists than the original sources. The usefulness of Joreskog's techniques is indicated by reformulating a series of models which have been estimated by sociologists using techniques without statistical justification in the format of covariance structures. Identification is considered in this context. The argument is made that these methods can greatly extend our ability to construct structural equation models containing measurement error.

This paper discusses the use of the Sparse Matrix Transform (SMT) to model the covariance structure of high-dimensional data in the likelihood ratio test used for hypothesis testing. The SMT has been shown to produce more accurate estimates of covariance matrices when the number of training samples n is much less than the number of dimensions p of the data. Several experiments with face recognition and hyperspectral images show that SMT-based hypothesis testing can be superior to other methods in at least two general aspects: First, the SMT-based method is more robust to the size of the training set, remaining accurate even when only a few training samples are available; Second, the total computation required to apply the method is very low, making it attractive for use in low-power devices, or in applications requiring fast computation.

In this study, we investigate the impact of covariance within uncertainties on the inference of cosmological and astrophysical parameters, specifically focusing on galaxy stellar mass functions derived from the CAMELS simulation suite. Utilizing both Fisher analysis and implicit likelihood inference, we explore how different covariance structures, including simple toy models and physics-motivated uncertainties, affect posterior distributions and parameter variances. Our methodology utilizes forward modeling via emulators that are trained on CAMELS simulations to produce stellar mass functions based on input parameters, subsequently incorporating Gaussian noise as defined by covariance matrices. We examine both toy model covariance matrices and physically motivated covariance matrices derived from observational factors like the stellar initial mass function and photometric aperture size. Our results demonstrate that covariance terms significantly influence parameter inference, often leading to tighter constraints or revealing complex, multimodal posterior distributions. These findings underscore the necessity of accounting for covariance when interpreting astrophysical observations, especially in fields where accurate parameter estimation is critical for model validation and hypothesis testing.

Assuming a covariance structure with blocked compound symmetry, it was showed that unbiased estimators for the covariance matrices are optimal under normality. In this paper, we derive the asymptotic distribution of the correlation matrix using unbiased estimators and discuss its use in hypothesis testing. The accuracy of the result is investigated through numerical simulation and the method is applied to real data.

In some studies, there is interest in testing the variance structure, as in the context of multivariate or modelling techniques. Therefore, the importance of using hypothesis tests on covariance structures is emphasized. The purpose of this study was to perform a detailed performance study regarding the power and type I error rate of some existing identity and sphericity tests, considering the scenarios with different numbers of variables (2 to 64) and sample sizes (5 to 100). The proposal of Ledoit and Wolf (2002) is the most appropriate to test the identity structure. For the sphericity test, the version of John (1972), modified by Ledoit and Wolf (2002), followed by the proposal of Box (1949), were the ones with the best performance.

Abstract. The skill of the state-of-the-art climate field reconstruction technique BARCAST (Bayesian Algorithm for Reconstructing Climate Anomalies in Space and Time) to reconstruct temperature with pronounced long-range memory (LRM) characteristics is tested. A novel technique for generating fields of target data has been developed and is used to provide ensembles of LRM stochastic processes with a prescribed spatial covariance structure. Based on different parameter setups, hypothesis testing in the spectral domain is used to investigate if the field and spatial mean reconstructions are consistent with either the fractional Gaussian noise (fGn) process null hypothesis used for generating the target data, or the autoregressive model of order 1 (AR(1)) process null hypothesis which is the assumed temporal evolution model for the reconstruction technique. The study reveals that the resulting field and spatial mean reconstructions are consistent with the fGn process hypothesis for some of the tested parameter configurations, while others are in better agreement with the AR(1) model. There are local differences in reconstruction skill and reconstructed scaling characteristics between individual grid cells, and the agreement with the fGn model is generally better for the spatial mean reconstruction than at individual locations. Our results demonstrate that the use of target data with a different spatiotemporal covariance structure than the BARCAST model assumption can lead to a potentially biased climate field reconstruction (CFR) and associated confidence intervals.