Multilevel Factor Analysis Research Articles

General and specific thinking skills and schooling: Preparing the mind to new learning

Enhancing thinking skills is an important goal of formal education. It is often embedded in national curricula, which, however, are seldom based on theoretical understanding of the structure of the skills or how they should be taught. Accordingly, there is only limited information available about schools’ success in this important task. The present study has two goals: firstly, to find support for the theoretical assumption of the nested structure of thinking skills with a core factor of formal thinking and specialised structures for verbal and quantitative reasoning; and secondly, to test the differentiated development of these skills in school. This was done by studying class-level variation of sixth graders’ thinking skills in a multilevel factor analysis when initial between-class differences at grade three had been taken into account. The data (N≈1543) were drawn from a learning to learn panel study in one of the major cities of Finland. The results showed that the core factor for formal thinking could be identified at both the individual and the class level, and that at the individual level there were statistically significant residual factors for verbal and quantitative reasoning. Initial between-class differences explained only a third of the variance of class-level formal thinking. This was interpreted to indicate the effect of schooling.

A daily measure of positive and negative alcohol expectancies and evaluations: documenting a two-factor structure and within- and between-person variability.

Despite the documented importance of alcohol outcome expectancies in predicting alcohol use and related consequences, little research has explored within-person variability in expectancies. This article details the construction and psychometric analysis of a measure of alcohol expectancies specifically designed for daily assessment. We developed a 15-item instrument to measure the likelihood of experiencing various outcomes from drinking, as well as the subjective evaluation of these outcomes. College students (N = 352; mean age = 19.7 years, SD = 1.26; 53.4% female) participated in a yearlong study wherein they completed three computerized interviews daily via mobile phones for 2 weeks in each academic quarter. Multilevel exploratory factor analysis was used to examine dimensionality at between-person and within-person levels, and generalizability coefficients were calculated to establish reliability. Intraclass correlation coefficients were generally between .30 and .40, demonstrating both between-person and within-person variability. Exploratory factor analysis resulted in a two-factor solution of positive and negative effects of alcohol, and two items with equivocal loadings were dropped from the final scale. The two subscales showed excellent reliabilities at within- and between- person levels, and the measure demonstrated good convergent and discriminant validity with a commonly used expectancy measure. Drinkers hold many expectations about the effects of alcohol, and measures are needed that are designed to capture both stable and context-dependent aspects of these beliefs. Results demonstrated significant day-to-day variation in the strength and valuation of alcohol expectancies, and the scale demonstrated good psychometric properties that establish its appropriateness for use in daily process studies of alcohol use.

Research on Social Organization Employee's Job Satisfaction in China Based on Multi-level Factor Analysis Method

Human resource is the core resources of social organization, and social organization employee's job satisfaction is also the key research topic in charity research. On the basis of the national survey, this paper draws factor analysis method to evaluate employee's job satisfaction of 438 social organizations in China, and takes four principal components as measurement indicators of the criterion level through analytic hierarchy process, the ten domains of social organization as measurement indicators of the program level. The finding shows: (1) Social organization employee's job satisfaction scores low in China. (2) Employee recognition makes up the most important weight of social organization employee's job satisfaction. (3) The difference among the fields of social organization employee's job satisfaction is huge. Social organization is a special organization with Chinese characteristic, which is similar with NGO, NPO and CSO. Social organizations are compose of foundation, non-governmental non-commercial enterprise and association, which are usually divided into ten domains, including education, health, culture, technology, sports, labor, civil affairs, social medium service, legal service and others. At present, a dilemma of social organization is the growing labour shortage, exacerbated by an increasing demand for human resource, fewer employees starting and more employees leaving the profession for the other sectors. According to the Ministry of Civil Affairs of the People's Republic of China, there were 440 thousand social organizations and 6.18 million employees, which accounts for only 0.5% of the population in 2010; the number of social organizations increased to 500 thousand in 2012, while the total employees decreased to 6.13 million, which accounted for only 0.48% of the population. The phenomenon draws increasing research attention to the motives, reward preferences and job satisfaction of staffs in social organization. A better understanding of these issues and of the facets affecting employee's intention to remain employed should benefit their retention. Unlike for-profit organization and public sector, however, research on social organization employee's job satisfaction remains scant. Job satisfaction is one of the strongest predictors of intent to stay and retention, and there is also some evidence that the quality of service is related to providers' job satisfaction (1) . A better understanding of social organization employee's job satisfaction can therefore benefit the sector significantly (2) .

Development of an ecological momentary assessment scale for appetite.

BackgroundAn understanding of eating behaviors is an important element of health education and treatment in clinical populations. To understand the biopsychosocial profile of eating behaviors in an ecologically valid way, ecological momentary assessment (EMA) is appropriate because its use is able to overcome the recall bias in patient-reported outcomes (PROs). As appetite is a key PRO associated with eating behaviors, this study was done to develop an EMA scale to evaluate the within-individual variation of momentary appetite and uses this scale to discuss the relationships between appetite and various psychological factors.MethodsTwenty healthy participants (age 23.6 ± 4.2 years old) wore a watch-type computer for a week. Several times a day, including just before and after meals, they recorded their momentary psychological stress, mood states, and ten items related to appetite. In addition, they recorded everything they ate and drank into a personal digital assistant (PDA)-based food diary. Multilevel factor analysis was used to investigate the factor structure of the scale, and the reliability and validity of the scale were also explored.ResultsMultilevel factor analyses found two factors at the within-individual level (hunger/fullness and cravings) and one factor at the between-individual level. Medians for the individually calculated Cronbach’s alphas were 0.89 for hunger/fullness, 0.71 for cravings, and 0.86 for total appetite (the sum of all items). Hunger/fullness, cravings, and total appetite all decreased significantly after meals compared with those before meals, and hunger/fullness, cravings, and total appetite before meals were positively associated with energy intake. There were significant negative associations between both hunger/fullness and total appetite and anxiety and depression as well as between cravings, and depression, anxiety and stress.ConclusionsThe within-individual reliability of the EMA scale to assess momentary appetite was confirmed in most subjects and it was also validated as a useful tool to understand eating behaviors in daily settings. Further refinement of the scale is necessary and further investigations need to be conducted, particularly on clinical populations.

Measuring psychosocial environments using individual responses: an application of multilevel factor analysis to examining students in schools.

Interest in understanding how psychosocial environments shape youth outcomes has grown considerably. School environments are of particular interest to prevention scientists as many prevention interventions are school-based. Therefore, effective conceptualization and operationalization of the school environment is critical. This paper presents an illustration of an emerging analytic method called multilevel factor analysis (MLFA) that provides an alternative strategy to conceptualize, measure, and model environments. MLFA decomposes the total sample variance-covariance matrix for variables measured at the individual level into within-cluster (e.g., student level) and between-cluster (e.g., school level) matrices and simultaneously models potentially distinct latent factor structures at each level. Using data from 79,362 students from 126 schools in the National Longitudinal Study of Adolescent to Adult Health (formerly known as the National Longitudinal Study of Adolescent Health), we use MLFA to show how 20 items capturing student self-reported behaviors and emotions provide information about both students (within level) and their school environment (between level). We identified four latent factors at the within level: (1) school adjustment, (2) externalizing problems, (3) internalizing problems, and (4) self-esteem. Three factors were identified at the between level: (1) collective school adjustment, (2) psychosocial environment, and (3) collective self-esteem. The finding of different and substantively distinct latent factor structures at each level emphasizes the need for prevention theory and practice to separately consider and measure constructs at each level of analysis. The MLFA method can be applied to other nested relationships, such as youth in neighborhoods, and extended to a multilevel structural equation model to better understand associations between environments and individual outcomes and therefore how to best implement preventive interventions.

Multilevel Factor Analysis by Model Segregation

Measures of classroom environments have become central to policy efforts that assess school and teacher quality. This has sparked a wide interest in using multilevel factor analysis to test measurement hypotheses about classroom-level variables. One approach partitions the total covariance matrix and tests models separately on the between-classroom and within-classroom levels. This article shows that when using this approach, robust test statistics, including rescaled and residual-based test statistics provide better inferences about the classroom-level measurement structure than the widely used likelihood ratio test statistic even when the number of classrooms is large, and there is no excess kurtosis in the observed variables. This article then presents an empirical example and a simulation study to demonstrate how item intraclass correlations and within-group sample sizes influence test statistic performance. The results have implications for the study of classroom environments.

Cross-Level Measurement Invariance in School and Classroom Environment Surveys

Measures of classroom and school environments are central to policy efforts that assess school and teacher quality. These measures are often formed by aggregating individual survey responses to form group-level measures, and assume an invariant measurement model holds across the individual and group levels. This article explores the tenability of this assumption by applying multilevel factor analysis to two well-known surveys: the Working Conditions Survey, which assesses school environments, and the Tripod Classroom Environment Survey. The examples illustrate the consequences of using common factor analytic methods that assume cross-level invariance, and demonstrate how distorted perceptions of factorial structure can influence inferences about the relationship between working conditions and teacher mobility.

Preschoolers' oral language abilities: A multilevel examination of dimensionality

This study explored the dimensionality of children's oral language using a fully crossed design, where modality (expressive versus receptive) was crossed with linguistic domain (grammar, semantics and phonology). The present study was also the first of its kind to employ multilevel factor analysis to control confounding classroom effects. Assessments of oral language were completed by 529 children who attended 85 preschool classrooms. The sample was heterogeneous with regard to language ability, ethnicity, sex and SES. Classroom membership was more highly associated with preschoolers' semantics and grammar than with preschoolers' phonological systems. A general language ability was found to drive preschoolers' performances on tests of semantics and grammar, regardless of modality. Furthermore, articulation and speech perception were found separate but correlated abilities. Implications for assessment and diagnosis of oral language impairments are discussed.

Investigating neighbourhood effects on health: Using community-survey data for developing neighbourhood-related constructs

Structural and social neighbourhood constructs have been developed for studying a neighbourhood's influence on a variety of health outcomes; community surveys are being increasingly used for capturing such information. This paper has proposed a six-fold approach which integrates existing methodologies (i.e. multilevel factor analysis, ecometrics, multilevel spatial multiple membership models and multilevel latent class analysis) for estimating reliable and valid measurement of neighbourhood conditions. The proposed approach used seven demographic and socio-economic variables reported in a community survey by 20,413 individuals residing in 244 neighbourhoods in Medellin, Colombia, to measure structural neighbourhood conditions. The set of variables reliably measured one neighbourhood construct: the deprivation index; this showed significant variation between neighbourhoods as well as significant spatial clustering across the city. The approach presented here should enable public health researchers to better estimate neighbourhood indicators and may result in more accurate assessment of the relationship between neighbourhood characteristics and individual-level health outcomes.

Translating multilevel theory into multilevel research: challenges and opportunities for understanding the social determinants of psychiatric disorders

The observation that features of the social environment, including family, school, and neighborhood characteristics, are associated with individual-level outcomes has spurred the development of dozens of multilevel or ecological theoretical frameworks in epidemiology, public health, psychology, and sociology, among other disciplines. Despite the widespread use of such theories in etiological, intervention, and policy studies, challenges remain in bridging multilevel theory and empirical research. This paper set out to synthesize these challenges and provide specific examples of methodological and analytical strategies researchers are using to gain a more nuanced understanding of the social determinants of psychiatric disorders, with a focus on children's mental health. To accomplish this goal, we begin by describing multilevel theories, defining their core elements, and discussing what these theories suggest is needed in empirical work. In the second part, we outline the main challenges researchers face in translating multilevel theory into research. These challenges are presented for each stage of the research process. In the third section, we describe two methods being used as alternatives to traditional multilevel modeling techniques to better bridge multilevel theory and multilevel research. These are (1) multilevel factor analysis and multilevel structural equation modeling; and (2) dynamic systems approaches. Through its review of multilevel theory, assessment of existing strategies, and examination of emerging methodologies, this paper offers a framework to evaluate and guide empirical studies on the social determinants of child psychiatric disorders as well as health across the life course.

Multilevel ordinal factor analysis of the positive and negative syndrome scale (PANSS).

Clinical assessments of the presence and severity of psychopathology are often collected by health care professionals in mental health services or clinical researchers trained to use semi-structured interviews. Clustering by interviewer or rater needs to be considered when performing psychometric analyses such as factor analysis or item response modelling as non-independence of observations arises in these situations. We apply more suitable multilevel methods to analyse ordinally scored Positive and Negative Syndrome Scale (PANSS) items. Our aim is to highlight the differences in results that occur when the data are analysed using a hierarchically sensitive approach rather than using a traditional (aggregated) analysis. Our sample (n = 507) consisted of patients diagnosed with schizophrenia who participated in a multi-centre randomized control clinical trial, the DIALOG study. Analyses reported and compared include an exploratory factor analysis as well as several recently published multifactor models re-estimated within a confirmatory analysis framework. Our results show that the fit of the model and the parsimony of the exploratory factor analysis (EFA) models indicated by the number of factors necessary to explain the inter-correlation among PANSS items improved significantly when data clustering is taken into account through multilevel analysis. Our modeling results support the pentagonal PANSS model first proposed by White et al. (1997). Copyright © 2014 John Wiley & Sons, Ltd.

Multilevel Factor Analyses of Family Data From the Hawai’i Family Study of Cognition

In this study, we reanalyzed the classic Hawai’i Family Study of Cognition (HFSC) data using contemporary multilevel modeling techniques. We used the HFSC baseline data ( N = 6,579) and reexamined the factorial structure of 16 cognitive variables using confirmatory (restricted) measurement models in an explicit sequence. These models were initially fitted using multilevel confirmatory factor analysis techniques and the invariant six-factor models with two higher order factors fit fairly well (εa< 0.08) to the total, between- and within-family data. More crucially, a model requiring metric factorial invariance proved to be a reasonable fit to the between and within matrices, and allowed the ratio of the between-family variation to total family variation (eta-squared) to be calculated separately for each common factor (η2: Gc = .27, Gf = .22, Gm = .15, Gs = .04, Gv = .30, and SP = .16). Higher order factors were fitted using multilevel structural equation modeling techniques and these suggested a reasonable two-factor solution with unequal family impacts. These results suggest that (a) A “G only model” does not fit the data very well, and there are many sources of individual differences in cognitive abilities; (b) the sources of the individual differences in cognition can be measured the same way between and within families; and (c) even after the unique test components are removed, cognitive differences are larger within families than between families. We consider other general multivariate family models, and we raise questions about family influences.

Perfectionistic self-presentation predicts social anxiety using daily diary methods

Perfectionistic self-presentation is thought to confer risk for social anxiety. Although this relationship is thought to occur dynamically from moment-to-moment, no research has yet tested this relationship using experience sampling methods. The present study stringently tested whether perfectionistic self-presentation predicted social anxiety beyond several important covariates using a 21-day experience sampling design. A sample of 165 undergraduates (75.6% women) completed a series of questionnaires each day for 21days using palm pilots. Generalizability theory and multilevel factor analyses suggested daily measures of perfectionistic self-presentation, social anxiety, perfectionism cognitions, and depressed mood evidence within-subjects and between-subjects variability, can be measured reliably, and represent distinct factors, allowing hypothesis testing. Multilevel regressions showed perfectionistic self-presentation predicted social anxiety at the between-subjects and within-subjects levels, even when controlling for socially prescribed perfectionism, perfectionism cognitions, and depressed mood. Overall, perfectionistic self-presentation emerged as a robust predictor of daily social anxiety, clearly extending prior cross-sectional research on this topic. By understanding how perfectionism operates from day-to-day, we can better understand the processes that give rise to social anxiety, and ultimately how to devise more effective ways to help people suffering from social anxiety.

Characteristics of teaching and students’ emotions in the classroom: Investigating differences across domains

This study investigated the relations between eight characteristics of teaching and students’ academic emotions (enjoyment, pride, anxiety, anger, helplessness and boredom) across four academic domains (mathematics, physics, German, and English). 121 students (50% female; 8th and 11th graders) were asked about their perceptions of teaching characteristics and their academic emotions using the experience sampling method (real-time approach) for a period of 10 school days, with intraindividual analyses conducted using a multilevel approach. Multilevel exploratory factor analysis revealed that the eight teaching characteristics (understandability, illustration, enthusiasm, fostering attention, lack of clarity, difficulty, pace, level of expectation) represented two factors, labeled supportive presentation style (e.g., comprising understandability) and excessive lesson demands (e.g., comprising difficulty). In line with our hypothesis, we found clear relations on the intraindividual level between the two factors of teaching characteristics and students’ academic emotions in the classroom (e.g., supportive presentation style positively related to students’ enjoyment and negatively related to their boredom). Further, and supporting the universality assumption of teaching characteristics/academic emotions relations, the strength of relations between the two factors of teaching characteristics and academic emotions was very similar across the four academic domains. Implications for future research and educational practice are discussed.

Multilevel Factor Analytic Models for Assessing The Relationship Between Nurse-Reported Adverse Events and Patient Safety

SummaryWe explore health outcomes and nurse staffing data that are multivariate multilevel structured. These data can be used to relate latent constructs such as patient safety to hospital, nursing unit, nurse and patient characteristics by using factor analytic models. It is important that the multilevel nature of the data is taken into account; otherwise it can lead to invalid inferences. We explore the relationship between patient safety and nurse-reported adverse events from the Belgian chapter of the Europe Nurse Forecasting Survey. The data were split into a learning and a validation data set. Since no a priori factor structure has been proposed in the literature, we establish the factor structure by using a frequentist exploratory factor analysis on the learning data set and validate the factor structure proposed by using a Bayesian confirmatory factor analysis on the validation data set. Multivariate analysis-of-variance discrepancy measures were used to assess the need for multilevel factor analysis. We establish that there was substantial between-nursing-unit, but not between-hospital, variability to warrant the use of multilevel factor analyses. The final model was a two-level (nurse level and nursing unit level) factor analytic model with two factors at both levels. The Bayesian approach offers more flexibility in fitting the multilevel confirmatory factor analysis. To avoid double usage of the data the validation and learning data sets were used to fit and assess the goodness of fit of the multilevel confirmatory factor analysis respectively.

Psychometric properties of the Swiss version of the Nursing Home Survey on Patient Safety Culture

The Nursing Home Survey on Patient Safety Culture (NHSPSC) was specifically developed for nursing homes to assess a facility’s safety climate and it consists of 12 dimensions. After its pilot testing, however, no further psychometric analyses were performed on the instrument. For this study of safety climate in Swiss nursing home units, the NHSPSC was linguistically adapted to the Swiss context and to address the unit as well as facility level, with the aim of testing aspects of the validity and reliability of the Swiss version before its use in Swiss nursing home units. Psychometric analyses were performed on data from 367 nursing personnel from nine nursing homes in the German-speaking part of Switzerland (response rate=66%), and content validity (CVI) examined. The statistical influence of unit membership on respondents’ answers, and on their agreement concerning their units’ safety climate, was tested using intraclass correlation coefficients (ICCs) and the rWG(J) interrater agreement index. A multilevel exploratory factor analysis (MEFA) with oblimin rotation was applied to examine the questionnaire’s dimensionality. Cronbach’s alpha and Raykov’s rho were calculated to assess factor reliability. The relationship of safety climate dimensions with clinical outcomes was explored. Expert feedback confirmed the relevance of the instrument’s items (CVI=0.93). Personnel showed strong agreement in their perceptions in three dimensions of the questionnaire. ICCs supported a multilevel analysis. MEFA produced nine factors at the within-level (in comparison to 12 in the original version) and two factors at the between-level with satisfactory fit statistics. Raykov’s Rho for the single level factors ranged between 0.67 and 0.86. Some safety climate dimensions show moderate, but non-significant correlations with the use of bedrails, physical restraint use, and fall-related injuries. The Swiss version of the NHSPSC needs further refinement and testing before its use can be recommended in Swiss nursing homes: its dimensionality needs further clarification, particularly to distinguish items addressing the unit-level safety climate from those at the facility level.

Phenomenology of future-oriented mind-wandering episodes.

Recent research suggests that prospective and non-prospective forms of mind-wandering possess distinct properties, yet little is known about what exactly differentiates between future-oriented and non-future-oriented mind-wandering episodes. In the present study, we used multilevel exploratory factor analyses (MEFA) to examine the factorial structure of various phenomenological dimensions of mind-wandering, and we then investigated whether future-oriented mind-wandering episodes differ from other classes of mind-wandering along the identified factors. We found that the phenomenological dimensions of mind-wandering are structured in four factors: representational format (inner speech vs. visual imagery), personal relevance, realism/concreteness, and structuration. Prospective mind-wandering differed from non-prospective mind-wandering along each of these factors. Specifically, future-oriented mind-wandering episodes involved inner speech to a greater extent, were more personally relevant, more realistic/concrete, and more often part of structured sequences of thoughts. These results show that future-oriented mind-wandering possesses a unique phenomenological signature and provide new insights into how this particular form of mind-wandering may adaptively contribute to autobiographical planning.

Psychometrische eigenschappen van meetinstrumenten in dagboekonderzoek

The psychometric qualities of measurement instruments in diary studies: an illustration on the basis of the work engagement scale The psychometric qualities of measurement instruments in diary studies: an illustration on the basis of the work engagement scale Despite the emerging popularity of diary studies, until now little attention has been paid to the psychometric properties of measures used in diary studies. When the time frame of questionnaires in diary studies is adapted, it is important to examine whether the factor structure remains the same. The aim of this article is to introduce a method to examine the factor structure of daily variables. This method will be illustrated by performing a multilevel confirmative factor analysis on the questionnaire used to measure daily work engagement. The results showed that daily work engagement can be measured by adapting the time frame of the questionnaire to measure enduring work engagement, although it is eligible to replace one of the nine items.

Environmental Profile of a Community’s Health (EPOCH): An Ecometric Assessment of Measures of the Community Environment Based on Individual Perception

BackgroundPublic health research has turned towards examining upstream, community-level determinants of cardiovascular disease risk factors. Objective measures of the environment, such as those derived from direct observation, and perception-based measures by residents have both been associated with health behaviours. However, current methods are generally limited to objective measures, often derived from administrative data, and few instruments have been evaluated for use in rural areas or in low-income countries. We evaluate the reliability of a quantitative tool designed to capture perceptions of community tobacco, nutrition, and social environments obtained from interviews with residents in communities in 5 countries.Methodology/ Principal FindingsThirteen measures of the community environment were developed from responses to questionnaire items from 2,360 individuals residing in 84 urban and rural communities in 5 countries (China, India, Brazil, Colombia, and Canada) in the Environmental Profile of a Community’s Health (EPOCH) study. Reliability and other properties of the community-level measures were assessed using multilevel models. High reliability (>0.80) was demonstrated for all community-level measures at the mean number of survey respondents per community (n = 28 respondents). Questionnaire items included in each scale were found to represent a common latent factor at the community level in multilevel factor analysis models.Conclusions/ SignificanceReliable measures which represent aspects of communities potentially related to cardiovascular disease (CVD)/risk factors can be obtained using feasible sample sizes. The EPOCH instrument is suitable for use in different settings to explore upstream determinants of CVD/risk factors.

All too human? Chimpanzee and orang-utan personalities are not anthropomorphic projections

Ratings of chimpanzee, Pan troglodytes, and orang-utan, Pongo pygmaeus and Pongo abelii, personality reveal dimensions resembling those found in humans. Critics have argued that this similarity derives from anthropomorphic projection or other rater-based effects. We developed two forms of data reduction analyses to determine whether these dimensions can best be explained by the inherent tendencies of the animals (e.g. orang-utans that are curious are playful) or anthropomorphic projections of raters (e.g. believing that orang-utans that are curious should be playful). We found that personality dimensions derived after differences between rater means and rater*item interactions had been removed from ratings replicated the previously discovered dimensions. Conversely, we found a different set of dimensions when analysing items from which differences between animal means and animal*item interactions had been removed. Finally, we used multilevel factor analysis to examine whether the published structure was replicated when we extracted factors based on the within-level animal differences in item scores effects while allowing between-rater differences to covary freely. Again, the personality dimensions were similar