Cross-classified Data Research Articles

Decomposing the True Score Variance in Rated Responses to Divergent Thinking-Tasks for Assessing Creativity: A Multitrait-Multimethod Analysis.

It is shown how the Correlated Traits Correlated Methods Minus One (CTC(M - 1)) Multitrait-Multimethod model for cross-classified data can be modified and applied to divergent thinking (DT)-task responses scored for miscellaneous aspects of creative quality by several raters. In contrast to previous Confirmatory Factor Analysis approaches to analyzing DT-tasks, this model explicitly takes the cross-classified data structure resulting from the employment of raters into account and decomposes the true score variance into target-specific, DT-task object-specific, rater-specific, and rater-target interaction-specific components. This enables the computation of meaningful measurement error-free relative variance-parameters such as trait-consistency, object-method specificity, rater specificity, rater-target interaction specificity, and model-implied intra-class correlations. In the empirical application with alternate uses tasks as DT-measures, the model is estimated using Bayesian statistics. The results are compared to the results yielded with a simplified version of the model, once estimated with Bayesian statistics and once estimated with the maximum likelihood method. The results show high trait-correlations and low consistency across DT-measures which indicates more heterogeneity across the DT-measurement instruments than across different creativity aspects. Substantive deliberations and further modifications, extensions, useful applications, and limitations of the model are discussed.

Profile-based latent class distance association analyses for sparse tables:application to the attitude of European citizens towards sustainable tourism

AbstractSocial and behavioural sciences often deal with the analysis of associations for cross-classified data. This paper focuses on the study of the patterns observed on European citizens regarding their attitude towards sustainable tourism, specifically their willingness to change travel and tourism habits to be more sustainable. The data collected the intention to comply with nine sustainable actions; answers to these questions generated individual profiles; moreover, European country belonging is reported. Therefore, unlike a variable-oriented approach, here we are interested in a person-oriented approach through profiles. Some traditional methods are limited in their performance when using profiles, for example, by sparseness of the contingency table. We removed many of these limitations by using a latent class distance association model, clustering the row profiles into classes and representing these together with the categories of the response variable in a low-dimensional space. We showed, furthermore, that an easy interpretation of associations between clusters’ centres and categories of a response variable can be incorporated in this framework in an intuitive way using unfolding. Results of the analyses outlined that citizens mostly committed to an environmentally friendly behavior live in Sweden and Romania; citizens less willing to change their habits towards a more sustainable behavior live in Belgium, Cyprus, France, Lithuania and the Netherlands. Citizens preparedness to change habits however depends also on their socio-demographic characteristics such as gender, age, occupation, type of community where living, household size, and the frequency of travelling before the Covid-19 pandemic.

A correlated traits correlated (methods - 1) multitrait-multimethod model for augmented round-robin data.

We didactically derive a correlated traits correlated (methods - 1) [CTC(M - 1)] multitrait-multimethod (MTMM) model for dyadic round-robin data augmented by self-reports. The model is an extension of the CTC(M - 1) model for cross-classified data and can handle dependencies between raters and targets by including reciprocity covariance parameters that are inherent in augmented round-robin designs. It can be specified as a traditional structural equation model. We present the variance decomposition as well as consistency and reliability coefficients. Moreover, we explain how to evaluate fit of a CTC(M - 1) model for augmented round-robin data. In a simulation study, we explore the properties of the full information maximum likelihood estimation of the model. Model (mis)fit can be quite accurately detected with the test of not close fit and dynamic root mean square errors of approximation. Even with few small round-robin groups, relative parameter estimation bias and coverage rates are satisfactory, but several larger round-robin groups are needed to minimize relative parameter estimation inaccuracy. Further, neglecting the reciprocity covariance-structure of the augmented round-robin data does not severely bias the remaining parameter estimates. All analyses (including data, R scripts, and results) and the simulation study are provided in the Supporting Information. Implications and limitations are discussed.

A two-step log-linear procedure for graphical representation and inference of associations in cross-classified data for disease diagnosis.

Biometrical sciences and disease diagnosis in particular, are often concerned with the analysis of associations for cross-classified data, for which distance association models give us a graphical interpretation for non-sparse matrices with a low number of categories. In this framework, usually binary exploratory and response variables are present, with analysis based on individual profiles being of great interest. For saturated models, we show the usual linear relationship for log-linear models is preserved in full dimension for the distance association parameterization. This enables a two-step procedure to facilitate the analysis and the interpretation of associations in terms of unfolding after the overall and main effects are removed. The proposed procedure can deal with cross-classified data for profiles by binary variables, and it is easy to implement using traditional statistical software. For disease diagnosis, the problems of a degenerate solution in the unfolding representation, and that of determining significant differences between the profile locations are addressed. A hypothesis test of independence based on odds ratio is considered. Furthermore, a procedure is proposed to determine the causes of the significance of the test, avoiding the problem of error propagation. The equivalence between a test for equality of odds ratio pairs and the test for equality of location for two profiles in the unfolding representation in the disease diagnosis is shown. The results have been applied to a real example on the diagnosis of coronary disease, relating the odds ratios with performance parameters of the diagnostic test.

Cross-Classified Item Response Theory Modeling With an Application to Student Evaluation of Teaching

The cross-classified data structure is ubiquitous in education, psychology, and health outcome sciences. In these areas, assessment instruments that are made up of multiple items are frequently used to measure latent constructs. The presence of both the cross-classified structure and multivariate categorical outcomes leads to the so-called item-level data with cross-classified structure. An example of such data structure is the routinely collected student evaluation of teaching (SET) data. Motivated by the lack of research on multilevel IRT modeling with crossed random effects and the need of an approach that can properly handle SET data, this study proposed a cross-classified IRT model, which takes into account both the cross-classified data structure and properties of multiple items in an assessment instrument. A new variant of the Metropolis–Hastings Robbins–Monro (MH-RM) algorithm was introduced to address the computational complexities in estimating the proposed model. A preliminary simulation study was conducted to evaluate the performance of the algorithm for fitting the proposed model to data. The results indicated that model parameters were well recovered. The proposed model was also applied to SET data collected at a large public university to answer empirical research questions. Limitations and future research directions were discussed.

Individual Mobility across Clusters: The Impact of Ignoring Cross-Classified Data Structures in Discrete-Time Survival Analysis

A multilevel-discrete time survival model may be appropriate for purely hierarchical data, but when data are non-purely hierarchical due to individual mobility across clusters, a cross-classified discrete time survival model may be necessary. The purpose of this research was to investigate the performance of a cross-classified discrete-time survival model and assess the impact of ignoring a cross-classified data structure on the model parameters of a conventional discrete-time survival model and a multilevel discrete-time survival model. A Monte Carlo simulation was used to examine the performance of three discrete-time survival models when individuals are mobile across clusters. Simulation factors included the value of the between-clusters variance, number of clusters, within-cluster sample size, Weibull scale parameter, and mobility rate. The results suggest that substantial relative parameter bias, unacceptable coverage of the 95% confidence intervals, and severely biased standard errors are possible for all model parameters when a discrete-time survival model is used that ignores the cross-classified data structure. The findings presented in this study are useful for methodologists and practitioners in educational research, public health, and other social sciences where discrete-time survival analysis is a common methodological technique for analyzing event-history data.

Comparing random effects models, ordinary least squares, or fixed effects with cluster robust standard errors for cross-classified data.

Cross-classified random effects modeling (CCREM) is a common approach for analyzing cross-classified data in psychology, education research, and other fields. However, when the focus of a study is on the regression coefficients at Level 1 rather than on the random effects, ordinary least squares regression with cluster robust variance estimators (OLS-CRVE) or fixed effects regression with CRVE (FE-CRVE) could be appropriate approaches. These alternative methods are potentially advantageous because they rely on weaker assumptions than those required by CCREM. We conducted a Monte Carlo Simulation study to compare the performance of CCREM, OLS-CRVE, and FE-CRVE in models, including conditions where homoscedasticity assumptions and exogeneity assumptions held and conditions where they were violated, as well as conditions with unmodeled random slopes. We found that CCREM out-performed the alternative approaches when its assumptions are all met. However, when homoscedasticity assumptions are violated, OLS-CRVE and FE-CRVE provided similar or better performance than CCREM. When the exogeneity assumption is violated, only FE-CRVE provided adequate performance. Further, OLS-CRVE and FE-CRVE provided more accurate inferences than CCREM in the presence of unmodeled random slopes. Thus, we recommend two-way FE-CRVE as a good alternative to CCREM, particularly if the homoscedasticity or exogeneity assumptions of the CCREM might be in doubt. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Supplemental Material for Comparing Random Effects Models, Ordinary Least Squares, or Fixed Effects With Cluster Robust Standard Errors for Cross-Classified Data

Supplemental Material for Comparing Random Effects Models, Ordinary Least Squares, or Fixed Effects With Cluster Robust Standard Errors for Cross-Classified Data

Handling Missing Data in Cross-Classified Multilevel Analyses: An Evaluation of Different Multiple Imputation Approaches

Multiple imputation (MI) is a popular method for handling missing data. In education research, it can be challenging to use MI because the data often have a clustered structure that need to be accommodated during MI. Although much research has considered applications of MI in hierarchical data, little is known about its use in cross-classified data, in which observations are clustered in multiple higher-level units simultaneously (e.g., schools and neighborhoods, transitions from primary to secondary schools). In this article, we consider several approaches to MI for cross-classified data (CC-MI), including a novel fully conditional specification approach, a joint modeling approach, and other approaches that are based on single- and two-level MI. In this context, we clarify the conditions that CC-MI methods need to fulfill to provide a suitable treatment of missing data, and we compare the approaches both from a theoretical perspective and in a simulation study. Finally, we illustrate the use of CC-MI in real data and discuss the implications of our findings for research practice.

Propensity Score Matching with Cross-Classified Data Structures: A Comparison of Methods

In the current study, we compare propensity score (PS) matching methods for data with a cross-classified structure, where each individual is clustered within more than one group, but the groups are not hierarchically organized. Through a Monte Carlo simulation study, we compared sequential cluster matching (SCM), preferential within cluster matching (PWCM), greedy matching (GM), and optimal full matching (OFM), using propensity scores from four different models. The results indicated that the four matching methods performed well when PSs were estimated with logistic regression containing both level-1 and level-2 covariates. When the level-2 covariates were omitted in the logistic regression PS model, matching methods resulted in biased treatment effect estimates. However, omission of level-2 covariates did not result in biased estimates when the PS model was a logistic cross-classified random effects model (CCREM). SCM and PWCM outperformed GM and OFM with a logistic CCREM that included level-1 and level-2 covariates.

Modern applications of cross-classified random effects models in social and behavioral research: Illustration with R package PLmixed.

Cross-classified random effects models (CCREMs) have been developed for appropriately analyzing data with a cross-classified structure. Despite its flexibility and the prevalence of cross-classified data in social and behavioral research, CCREMs have been under-utilized in applied research. In this article, we present CCREMs as a general and flexible modeling framework, and present a wide range of existing models designed for different purposes as special instances of CCREMs. We also introduce several less well-known applications of CCREMs. The flexibility of CCREMs allows these models to be easily extended to address substantive questions. We use the free R package PLmixed to illustrate the estimation of these models, and show how the general language of the CCREM framework can be translated into specific modeling contexts.

Multiple imputation approaches for handling incomplete three-level data with time-varying cluster-memberships.

Three‐level data arising from repeated measures on individuals clustered within higher‐level units are common in medical research. A complexity arises when individuals change clusters over time, resulting in a cross‐classified data structure. Missing values in these studies are commonly handled via multiple imputation (MI). If the three‐level, cross‐classified structure is modeled in the analysis, it also needs to be accommodated in the imputation model to ensure valid results. While incomplete three‐level data can be handled using various approaches within MI, the performance of these in the cross‐classified data setting remains unclear. We conducted simulations under a range of scenarios to compare these approaches in the context of an acute‐effects cross‐classified random effects substantive model, which models the time‐varying cluster membership via simple additive random effects. The simulation study was based on a case study in a longitudinal cohort of students clustered within schools. We evaluated methods that ignore the time‐varying cluster memberships by taking the first or most common cluster for each individual; pragmatic extensions of single‐ and two‐level MI approaches within the joint modeling (JM) and the fully conditional specification (FCS) frameworks, using dummy indicators (DI) and/or imputing repeated measures in wide format to account for the cross‐classified structure; and a three‐level FCS MI approach developed specifically for cross‐classified data. Results indicated that the FCS implementations performed well in terms of bias and precision while JM approaches performed poorly. Under both frameworks approaches using the DI extension should be used with caution in the presence of sparse data.

Distance-based logistic model for cross-classified categorical data.

Logistic regression models are a powerful research tool for the analysis of cross-classified data in which a categorical response variable is involved. In a logistic model, the effect of a covariate refers to odds, and the simple relationship between the coefficients and the odds ratio often makes these the parameters of interest due to their easy interpretation. In this article we present a distance-based logistic model that allows a simple graphical interpretation of the association coefficients using the odds ratio in a contingency table. Two configurations are estimated, one for the rows and one for the columns, as the categories of a polytomous predictor and a nominal response variable respectively, such that the local odds ratio and the distances between the predictor and response categories are inversely related. The associations in terms of the odds ratios, or the ratios of the odds to their geometric means, are interpreted through distances for the most common coding schemes of the predictor variable, and the relationship between the distances related to different codings is investigated in its full dimension. The performance of the estimation procedure is analysed with a Monte Carlo experiment. The interpretation of the model and its performance, as well as its comparison with a two-step procedure involving first a logistic regression and then unfolding, is illustrated using real data sets.

Cross-classified multilevel models improved standard error estimates of covariates in clinical outcomes – a simulation study

ObjectiveTo compare estimates of effect and variability resulting from standard linear regression analysis and hierarchical multilevel analysis with cross-classified multilevel analysis under various scenarios. Study design and settingWe performed a simulation study based on a data structure from an observational study in clinical mental health care. We used a Markov chain Monte Carlo approach to simulate 18 scenarios, varying sample sizes, cluster sizes, effect sizes and between group variances. For each scenario, we performed standard linear regression, multilevel regression with random intercept on patient level, multilevel regression with random intercept on nursing team level and cross-classified multilevel analysis. ResultsApplying cross-classified multilevel analyses had negligible influence on the effect estimates. However, ignoring cross-classification led to underestimation of the standard errors of the covariates at the two cross-classified levels and to invalidly narrow confidence intervals. This may lead to incorrect statistical inference. Varying sample size, cluster size, effect size and variance had no meaningful influence on these findings. ConclusionIn case of cross-classified data structures, the use of a cross-classified multilevel model helps estimating valid precision of effects, and thereby, support correct inferences.

The Impact of Ignoring a Crossed Factor in Cross-Classified Multilevel Modeling.

The present study investigated estimate biases in cross-classified random effect modeling (CCREM) and hierarchical linear modeling (HLM) when ignoring a crossed factor in CCREM considering the impact of the feeder and the magnitude of coefficients. There were six simulation factors: the magnitude of coefficient, the correlation between the level 2 residuals, the number of groups, the average number of individuals sampled from each group, the intra-unit correlation coefficient, and the number of feeders. The targeted interests of the coefficients were four fixed effects and two random effects. The results showed that ignoring a crossed factor in cross-classified data causes a parameter bias for the random effects of level 2 predictors and a standard error bias for the fixed effects of intercepts, level 1 predictors, and level 2 predictors. Bayesian information criteria generally outperformed Akaike information criteria in detecting the correct model.

Comparing cross-classified mixed effects and Bayesian structural equations modeling for stimulus sampling designs: A simulation study

Researchers examining a wide range of psychological phenomena, including interpersonal perception, attitude formation, and stereotype activation, apply stimulus sampling designs (SSD). The standard SSD study requires participant raters to provide evaluations of a series of target stimuli (e.g., photographs, media clips, vignettes), and the constituent responses are simultaneously nested within participants and stimuli, yielding a cross-classified data structure. Prior methodological work has illustrated the application of both cross-classified mixed effects, and cross-classified structural equation modeling to accommodate the corresponding dependency structure. Despite their widespread application, little is known about how sample size for both participants and stimuli is associated with inferential power and coverage in SSDs. Even less is known about the feasibility of random slopes, or whether the frequentist (maximum likelihood) and Bayesian (MCMC) estimators differ in accuracy or efficiency under the design conditions typically observed in SSD studies. We conducted a Monte Carlo simulation study to better understand parameter bias, statistical power, and confidence or credible interval coverage, as a function of the number of participant raters and target stimuli, effect size, as well as the presence of random slopes, and modeling framework. Recommendations for future research are provided.

Random Forests Approach for Causal Inference with Clustered Observational Data

There is a growing interest in using machine learning (ML) methods for causal inference due to their (nearly) automatic and flexible ability to model key quantities such as the propensity score or the outcome model. Unfortunately, most ML methods for causal inference have been studied under single-level settings where all individuals are independent of each other and there is little work in using these methods with clustered or nested data, a common setting in education studies. This paper investigates using one particular ML method based on random forests known as Causal Forests to estimate treatment effects in multilevel observational data. We conduct simulation studies under different types of multilevel data, including two-level, three-level, and cross-classified data. Our simulation study shows that when the ML method is supplemented with estimated propensity scores from multilevel models that account for clustered/hierarchical structure, the modified ML method outperforms preexisting methods in a wide variety of settings. We conclude by estimating the effect of private math lessons in the Trends in International Mathematics and Science Study data, a large-scale educational assessment where students are nested within schools.

Self-reported memory problems in adult-onset cancer survivors in the United States with or without: Type 2 diabetes mellitus, hypertension and hyperlipidemia.

e24062 Background: Cancer and chronic diseases (CD) are independent predictors of cognitive dysfunction, especially memory problems that can impact patients’ psychosocial functioning and quality of life. We examined the prevalence of memory problems in cancer patients with or without CD, including Type 2 Diabetes mellitus (DM), hypertension (HTN) and hyperlipidemia (HLD). Methods: We examined data from people who completed the US National Health and Nutrition Examination Survey from 2009 to 2016 to determine the prevalence of memory problems in cancer survivors vs. non-cancer survivors with and without a history of CD. Participants answered the ‘yes’ or ‘no’ question: “Are you limited in any way because of difficulty remembering or because you experience periods of confusion?” History of Cancer, DM, HTN and HLD were self-reported. We used Rao-Scott Modified Chi-Square Test (X2test) for cross-classified categorical data from complex sample survey. Age, sex, race-ethnicity, education, poverty, and general health were controlled using logistic regression. Results: The sample included 21813 adults, ³ 20 years old, and ≈52% female. There were 40% Whites, 22% Blacks, 25% Mexican American/Other Hispanics, and 13% other race/multi-racial. About 46% graduated high School/had a GED (21.2%) or completed less than high school (24.4%). Of those with cancer ( N= 2025), memory problems were associated with DM (17.72%; X2test, p= 0.003) and (HTN, 13.42%; X2test, p= 0.007), but not HLD (11.66%; X2test, p= 0.786). Of those without cancer, memory problems were associated with DM, HTN and HLD (all p< 0.0001). Prevalence of CD was much greater in cancer survivors with memory problems ( p< .05). Conclusions: Strategies to mitigate memory problems in cancer survivors should consider the implications of DM, HTN and HLD, particularly given the prevalence of these chronic morbidities in cancer populations.

A Latent Block Distance-Association Model for Profile by Profile Cross-Classified Categorical Data

Distance association models constitute a useful tool for the analysis and graphical representation of cross-classified data in which distances between points inversely describe the association between two categorical variables. When the number of cells is large and the data counts result in sparse tables, the combination of clustering and representation reduces the number of parameters to be estimated and facilitates interpretation. In this article, a latent block distance-association model is proposed to apply block clustering to the outcomes of two categorical variables while the cluster centers are represented in a low dimensional space in terms of a distance-association model. This model is particularly useful for contingency tables in which both the rows and the columns are characterized as profiles of sets of response variables. The parameters are estimated under a Poisson sampling scheme using a generalized EM algorithm. The performance of the model is tested in a Monte Carlo experiment, and an empirical data set is analyzed to illustrate the model.

Assessing Differences between Nested and Cross-Classified Hierarchical Models

In multilevel data, cross-classified data structures are common. For example, this occurs when individuals move to different regions in longitudinal data or students go to different secondary schools than their primary school peers. In both cases, the data structure is no longer fully nested. Estimating cross-classified multilevel models is computationally intensive, so researchers have used several shortcuts to decrease run time. We consider how these shortcuts affect parameter estimates. In particular, we compare parameter estimates from fully nested and cross-classified models using a series of Monte Carlo simulations. When the outcome is continuous, we identify systematic differences in estimated standard errors and some differences in the estimated variance components. When the outcome is binary, we also find differences in the estimated coefficients. Accordingly, we caution researchers to avoid fully nested model specifications when cross-classification exists but suggest some limited conditions under which parameter estimates are unlikely to be different.