Challenges in Multilevel Modelling: Cross-Group Measurement Noninvariance and Measurement Errors. A Monte Carlo Simulation Study

Multilevel Statistical Models

Multilevel models were originally developed to allow linear regression or ANOVA models to be applied to observations that are not mutually independent. This lack of independence commonly arises due to clustering of the units of observations into 'higher level units' such as patients in hospitals. In linear mixed models, the within-cluster correlations are modelled by including random effects in a linear model. In this paper, we discuss generalizations of linear mixed models suitable for responses subject to systematic and random measurement error and interval censoring. The first example uses data from two cross-sectional surveys of schoolchildren to investigate risk factors for early first experimentation with cigarettes. Here the recalled times of the children's first cigarette are likely to be subject to both systematic and random measurement errors as well as being interval censored. We describe multilevel models for interval censored survival times as special cases of generalized linear mixed models and discuss methods of estimating systematic recall bias. The second example is a longitudinal study of mental health problems of patients nested in clinics. Here the outcome is measured by multiple questionnaires allowing the measurement errors to be modelled within a linear latent growth curve model. The resulting model is a multilevel structural equation model. We briefly discuss such models both as extensions of linear mixed models and as extensions of structural equation models. Several different model structures are examined. An important goal of the paper is to place a number of methods that readers may have considered as being distinct within a single overall modelling framework.

This simulation study investigated use of the multilevel structural equation model (MLSEM) for handling measurement error in both mediator and outcome variables (M and Y) in an upper level multilevel mediation model. Mediation and outcome variable indicators were generated with measurement error. Parameter and standard error bias, confidence interval coverage, and power to detect the ab mediated effect using Empirical-M confidence interval estimates were assessed for the correct MLSEM versus a conventional multilevel model (MM) that used composite scores for M and Y. The following conditions were manipulated: level 1 and 2 sample sizes, intraclass correlation, degree of measurement error in M, and the true value of ab. The MLSEM more accurately recovered the ab effect's value, but serious convergence issues were encountered with MLSEM estimates based on fewer than 80 clusters. More power for detecting a nonzero ab was found for MM than for MLSEM estimates.

In recent years, multilevel models(MLM) have been frequently used for studying multilevel mediation in social sciences. However, there still exist sampling errors and measurement errors even after separating the between-group effects from the within-group ones of multilevel mediation. To solve this problem, a new method has been developed abroad by integrating MLM with structural equation models(SEM) under the framework of multilevel structural equation models(MSEM) to set latent variables and multiple indicators. That method has proved to rectify sampling errors and measurement errors effectively and obtain more accurate mediating effect values. And it is applicable to a larger variety of multilevel mediation analysis and able to provide fit indices of the models. After introducing the new method, we proposed a procedure for analyzing multilevel mediation by using MSEM, with an example illustrating how to execute it with the software MPLUS. Directions for future study on multilevel mediation and MSEM were discussed at the end of the paper.

Multilevel modeling (MLM) is frequently used to detect group differences, such as an intervention effect in a pre-test-post-test cluster-randomized design. Group differences on the post-test scores are detected by controlling for pre-test scores as a proxy variable for unobserved factors that predict future attributes. The pre-test and post-test scores that are most often used in MLM are summed item responses (or total scores). In prior research, there have been concerns regarding measurement error in the use of total scores in using MLM. To correct for measurement error in the covariate and outcome, a theoretical justification for the use of multilevel structural equation modeling (MSEM) has been established. However, MSEM for binary responses has not been widely applied to detect intervention effects (group differences) in intervention studies. In this article, the use of MSEM for intervention studies is demonstrated and the performance of MSEM is evaluated via a simulation study. Furthermore, the consequences of using MLM instead of MSEM are shown in detecting group differences. Results of the simulation study showed that MSEM performed adequately as the number of clusters, cluster size, and intraclass correlation increased and outperformed MLM for the detection of group differences.

Measured surface-atmosphere fluxes of energy (sensible heat, H, and latent heat, LE) and CO 2 ( FCO 2) represent the “true” flux plus or minus potential random and systematic measurement errors. Here, we use data from seven sites in the AmeriFlux network, including five forested sites (two of which include “tall tower” instrumentation), one grassland site, and one agricultural site, to conduct a cross-site analysis of random flux error. Quantification of this uncertainty is a prerequisite to model-data synthesis (data assimilation) and for defining confidence intervals on annual sums of net ecosystem exchange or making statistically valid comparisons between measurements and model predictions. We differenced paired observations (separated by exactly 24 h, under similar environmental conditions) to infer the characteristics of the random error in measured fluxes. Random flux error more closely follows a double-exponential (Laplace), rather than a normal (Gaussian), distribution, and increase as a linear function of the magnitude of the flux for all three scalar fluxes. Across sites, variation in the random error follows consistent and robust patterns in relation to environmental variables. For example, seasonal differences in the random error for H are small, in contrast to both LE and FCO 2, for which the random errors are roughly three-fold larger at the peak of the growing season compared to the dormant season. Random errors also generally scale with R n ( H and LE) and PPFD ( FCO 2). For FCO 2 (but not H or LE), the random error decreases with increasing wind speed. Data from two sites suggest that FCO 2 random error may be slightly smaller when a closed-path, rather than open-path, gas analyzer is used.

This contribution focuses on a model that is gaining currency in cross-national research, namely multilevel structural equation modelling (MSEM). Similarly to standard multilevel modelling (MLM), this model distinguishes between various levels of analysis (e. g., individuals nested within countries) and, in doing so, takes the hierarchical structure of cross-national data into account. However, MSEM incorporates a latent-variable approach into the multilevel framework, making it possible to assess the measurement quality of latent constructs. As such, MSEM is a synthesis of structural equation modeling (SEM) and MLM that combines the best of both worlds. The MSEM approach makes it possible to model multilevel mediations and group-level outcomes, and therefore provides a more complete representation of Coleman’s bathtub model. This contribution presents the statistical and conceptual background of MSEM in a formal but accessible manner. The paper discusses applications of MSEM that are particularly useful for cross-national comparative research (CNCR), namely two-level confirmatory factor analysis (CFA), multilevel mediation models, and models for group-level outcomes. A practical step-by-step strategy on how MSEM can be used for applied research is provided and illustrated by means of a didactical example.

One of the most important tasks of shipping for ensuring the efficiency and safety of navigation is accurately determining the ship position during the voyage. Research on the accuracy of finding a ship coordinates is always at the top of the list of urgent navigational issues, even nowadays with the widespread implementation of state-ofthe-art information technologies and the progress of navigation systems and techniques. If to follow measurement standards, there are two close terms applied in marine navigation in this context. They are “accuracy” and “precision” of measurements, which in principle might confuse the understanding of the task onboard the ship. In marine navigation, the traditional understanding of ship current position accuracy is to assess the impact of a combination of random errors in measured navigation parameters on random errors in computed coordinates. Two-dimensional confidence intervals centered on the fixed ship position serve as a graphical representation of the assessed accuracy, which is algebraically, might be described by the covariance matrix of coordinate errors. As a result of the high level of uncertainty caused by the effect of random errors in real-time measurements, it is impossible to assess the accuracy of ship position during the voyage. The magnitude and mathematical sign of these errors are unknown, and the navigation parameter indications themselves do not provide any information on their accuracy. For this reason, a priori prediction of the precision of navigation parameters associated with certain sets of measurements made in the past under specific standard conditions are used to assess the accuracy of the real-time ship position. This priori data inspires to substitute the concept of accuracy of the real-time ship position with the concept of precision of the navigation system, technique, or device operation, which is essentially the theoretical inconsistency. The purpose of this study is to partially solve this inconsistency. The study outcome is a proposed indicator that, by using information gathered from redundant measurements of navigation parameters obtained in real time without the use of priori data, can indirectly assess the accuracy of the real-time fixed coordinates of the ship position. The indicator concept is based on assessing the area of the real-time figure of errors. This area is limited by its outer contour of position lines and has a high degree of measurement redundancy to guarantee that, the probability of locating the true point within this figure is 100 %. In addition to having high precision features, modern navigation technology also makes it possible to handle a larger volume of measurement data utilizing contemporary technologies, such as Big Data platforms, which do not restrict the number of measurements. Consequently, higher number of navigation measurements can significantly raise the probability of finding the true position in the ensuing complicated figure of errors. The shape of the figure allows for a spatial analysis of the proximity of potential navigational hazards and the ship location, by using, for example, the least squares method. The area of such a figure is a characteristic of the uncertainty (an analog of precision) of coordinate errors, and its minimum area provides the best accuracy. The ability to determine the real-time figure area with sheer certainty of existing the true point in it stimulates the development of the next more technologically advanced and encouraging level of alternative and autonomous methods for determining the vessel position. These methods are based primarily on the possibility of increasing the volume of processed measurement information. It directly relates, for example, to the development of azimuthal methods of nautical astronomy, which make it possible to perform an unlimited number of autonomous navigation measurements in the absence of a visible horizon, which undoubtedly becomes valuable when ships are sailing in high latitude regions, especially during a long period of polar night.

The differential association of socioeconomic vulnerabilities and neglect-related child protection involvement across geographies: Multilevel structural equation modeling

BackgroundIt is often thought that random measurement error has a minor effect upon the results of an epidemiological survey. Theoretically, errors of measurement should always increase the spread of a distribution. Defining an illness by having a measurement outside an established healthy range will lead to an inflated prevalence of that condition if there are measurement errors.Methods and resultsA Monte Carlo simulation was conducted of anthropometric assessment of children with malnutrition. Random errors of increasing magnitude were imposed upon the populations and showed that there was an increase in the standard deviation with each of the errors that became exponentially greater with the magnitude of the error. The potential magnitude of the resulting error of reported prevalence of malnutrition were compared with published international data and found to be of sufficient magnitude to make a number of surveys and the numerous reports and analyses that used these data unreliable.ConclusionsThe effect of random error in public health surveys and the data upon which diagnostic cut-off points are derived to define “health” has been underestimated. Even quite modest random errors can more than double the reported prevalence of conditions such as malnutrition. Increasing sample size does not address this problem, and may even result in less accurate estimates. More attention needs to be paid to the selection, calibration and maintenance of instruments, measurer selection, training & supervision, routine estimation of the likely magnitude of errors using standardization tests, use of statistical likelihood of error to exclude data from analysis and full reporting of these procedures in order to judge the reliability of survey reports.

Multilevel modeling (MLM) is widely used for the multilevel data structure in social science. Two MLM limitations involve partitioning variance for a within-level predictor. First, MLM does not automatically partial out the between-level variance for a within-level predictor. Second, MLM assumes a within-level predictor to be measurement-error free. Thus, the error variance cannot be separated out. The study evaluated the impact of improperly partitioning two-level variances and measurement error variances for a within-level predictor in MLM. Our findings suggested (a) whether the research interest lies in a within-level or a between-level predictor, group-mean centering or latent-mean centering must be used to partition two-level variances; (b) for the measurement error variance, the within-level factor loadings should be as high as possible (≥0.80 in our simulation settings). If these two requirements are not met, multilevel structural equation modeling (MSEM) should always be adopted for the analysis.

Structural equation modeling (SEM) is commonly used to explore relations between latent variables, such as beliefs and attitudes. However, comparing structural relations across a large number of groups, such as countries or classrooms, can be challenging. Existing SEM approaches may fall short, especially when measurement non-invariance is present. In this paper, we propose Mixture Multilevel SEM (MixML-SEM), a novel approach to comparing relationships between latent variables across many groups. MixML-SEM gathers groups with the same structural relations in a cluster, while accounting for measurement non-invariance in a parsimonious way by means of random effects. Specifically, MixML-SEM captures measurement non-invariance using multilevel confirmatory factor analysis and, then, it estimates the structural relations and mixture clustering of the groups by means of the structural-after-measurement approach. In this way, MixML-SEM ensures that the clustering is focused on structural relations and unaffected by differences in measurement. In contrast, Multilevel SEM (ML-SEM) estimates measurement and structural models simultaneously, and both with random effects. In comparison to ML-SEM, MixML-SEM provides better estimates of the structural relations, especially when (some of) the groups are large. This is because combining information from multiple groups within a cluster leads to more accurate estimates of the structural relations, whereas, in case of ML-SEM, these estimates are affected by shrinkage bias. We demonstrate the advantages of MixML-SEM through simulations and an empirical example on how social pressure to be happy relates to life satisfaction across 40 countries.

Estimating true standard deviations.

Longitudinal analysis versus cross-sectional analysis in assessing the factors influencing shoppers’ impulse purchase behavior – Do the store ambience and salesperson interactions really matter?

International large-scale assessments in education (ILSAs) follow complex sampling designs and ultimately create hierarchical data structures with students nested in classrooms, classrooms in schools, schools in regions, etc. To describe adequately key issues in education, such as socioeconomic gaps in academic achievement or the relations among school characteristics and student achievement using ILSA data, researchers need to consider the hierarchical data structure in statistical models. Multilevel modeling is one approach to account for such hierarchies and consider variables at different levels of analysis. This chapter provides an overview of the prominent multilevel modeling approaches to analyzing ILSA data, illustrates and discusses their strengths and weaknesses, and highlights the key methodological decisions researchers have to take in this context. The first part reviews the current practices of multilevel modeling in secondary analyses of ILSA data. This rapid systematic review is followed by a second part in which we present, illustrate, and discuss multilevel modeling approaches, including multilevel regression, multilevel structural equation models, and multilevel mixture models. Next to model estimation and fit evaluation, we review key issues associated with the multilevel modeling of ILSA data and focus on handling plausible values, multigroup and incidental multilevel data structures, and weighting. Our chapter provides worked examples showcasing the potential of multilevel modeling for ILSA data analysis.