Psychometric Properties and Measurement Invariance of the Multidimensional School Engagement Scale

ABSTRACTWhile past research has shown the Student Engagement Instrument (SEI) to be an adequate measure of school engagement, this self-report questionnaire currently exists as a number of different versions, each comprising various subsets of the original 35 items. The first objective of this study was to use confirmatory factor analysis to assess the fit of different factorial structures with data acquired from 1,229 Portuguese adolescents in the 7th to 12th grades. Our second objective was then to test measurement invariance across gender and school year and to examine the psychometric properties of the factorial structure of the SEI with the best fit to our data. Our analyses revealed that the 15-item Brief-SEI structure fit our data best, and continued to fit the data well with the addition of two-higher order factors representing cognitive and psychological engagement. This higher-order model showed strict measurement invariance across gender and strong measurement invariance across school year. Internal consistency of the subscales was good and the subscales were correlated (albeit weakly) with academic performance. This study therefore provides a synthesis of current research on the SEI, and offers a validated instrument that is consistent with Appleton’s theoretical conceptualization of student engagement.

The Student Engagement Instrument (SEI) is a self-report measure of cognitive and affective engagement with school. Prior SEI validation studies have focused primarily on construct validity through analyses of internal consistency, factor analysis, and measurement invariance. Results are presented here from a two-pronged study of the criterion validity of SEI scores. Using a middle school sample ( N = 35,900), concurrent validity was assessed through analyses of group differences in SEI scores across student subgroups expected to differ in cognitive and affective engagement levels: behaviorally disengaged versus non-disengaged, high-risk versus low-risk disability status, and high versus low academic achievement. Next, through multiple logistic regression analyses, the 4-year predictive validity of SEI scores for on-time graduation and dropout was assessed in a cohort of first-time ninth graders ( N = 11,588). Nearly all SEI factors demonstrated directionally consistent associations with each criterion, including considerable long-term predictive associations with both dropout and on-time graduation.

Updating cross-cultural research of the past decade on the relationship between life aspirations and wellbeing, we compared Romanian (N=69) and US (N=64) undergraduates on the contribution of the importance and likelihood of attaining intrinsic and extrinsic aspirations to psychological maladjustment and life satisfaction, and on the qualitative meaning they assign to financial success. Similarly to prior studies, we found that extrinsic and intrinsic aspirations tended to be either negatively or positively correlated with life satisfaction, respectively; however, wealth predicted life satisfaction for Romanian students. Unlike previous research, we found generally negative relationships between intrinsic aspirations and psychological maladjustment. Although there were no differences between Romanian and US undergraduates on extrinsic and intrinsic orientation, on the overall importance of attaining aspirations, or on specific extrinsic and intrinsic aspirations, Romanian students expressed weaker expectations of fulfilling intrinsic aspirations than did US students. Finally, the groups produced similar rankings of aspirations and assigned similar meaning to financial success. The results favored a social cognitive rather than a self-determination model of psychological wellbeing in that expectations for attaining aspirations were more often predictive of life satisfaction than were their content. We interpret these findings and their convergence and departure from earlier research in terms of political economic, demographic, and cultural factors. We encourage future cross-cultural investigations of the social construction of aspirations, subsidiation of seemingly contradictory aspirations to each other, and cognitive and ecological mediation of the complex relationship of aspirations to psychological functioning.

Modern theories of authoritarianism have stressed the importance of threat to the expression of authoritarian attitudes and intolerance. Arguably, authoritarian tendencies may have increased during COVID-19 pandemic, a major threat to life and security. One issue arising when comparing mean scores is that of measurement invariance. Meaningful comparisons are only possible, if latent constructs are similar between groups and/or across time. This prerequisite is rarely ever tested in research on authoritarianism. In this study, we aim to analyze the short scale for authoritarianism KSA-3 by investigating its measurement invariance on two levels (three first-order and one second-order factors) and latent mean changes using two German representative samples (N = 4,905). Specifically, we look at differences before and during the pandemic (2017 vs. 2020). While measurement invariance holds across both levels in all conditions, we find a decrease in latent means in 2020, contrary to expectations and established theories. Moreover, latent means differ with regard to gender, education, and east–west Germany. We conclude that analyses of latent means and measurement invariance instead of mean comparisons with composites should become the standard. Future studies should focus on threat as a moderator between authoritarianism and intolerance, and on possible interactions with context variables.

This study utilized international, large-scale assessment data to compare science motivational beliefs of adolescents within and between countries and genders. The study focused on the beliefs about science of eighth graders, including their self-concept in science, the intrinsic value they ascribed to science, and their beliefs about the utility of the subject. The study data were derived from the Trends in International Mathematics and Science Study in 2019 (TIMSS) that was conducted in Taiwan, Australia, and the United States. To ensure the validity of mean cross-group comparisons, the measurement invariance (MI) of the constructs was first assessed. The multiple-group confirmatory factor analysis and latent factor mean comparisons were applied to the data. The results indicated that the MI of science motivational beliefs across the three countries attained only metric invariance, rendering a latent mean comparison implausible. However, the cross-gender MI within each country attained scalar invariance, supporting the comparison of means across genders. The science motivational beliefs of females were significantly lower than those of males, with the exception of beliefs of US students about their utility value. The findings of this study raise concerns about the validity of current international comparisons of science motivational beliefs of the students while supporting the use of TIMSS data to identify gender differences in science motivation within each country. The implications of MI across countries and genders are discussed, and the importance of establishing MI is highlighted. The findings affirm that gender disparities in science motivational beliefs can be compared using constructs with sound psychometric properties.

Research suggests the need to assess both positive and negative forms of student engagement. The purpose of this study was to pilot disaffection items with the Student Engagement Instrument (SEI) with a sample of middle school students from a rural area in the Southeastern U.S. This study explored the factor structure of the piloted items alongside the SEI, measurement invariance, and associations between student engagement and disaffection with educational outcomes such as mathematics and reading test scores, discipline referrals, and absences. Results hold implications for our theoretical understanding of engagement, suggesting that engagement and disaffection dimensions are theoretically and psychometrically distinct.

Generalized and generative thinking in US and Chinese students’ mathematical problem solving and problem posing

Student Engagement: Associations with Teachers and Peers as Motivators

School engagement and mental health are frequently linked, but within-person associations over time are largely unstudied. Emerging statistical techniques can better gauge how longitudinal changes in school engagement or mental health influence individual-level outcomes. Two cohorts of students (recruited in 2017 or 2018) in 5 Los Angeles high schools completed baseline surveys at the high school transition and 3 annual follow-up surveys through 11th grade. Random-intercept cross-lagged panel models explored the strength and directionality of associations between school engagement (Student Engagement Instrument) and mental health (Mental Health Inventory). Among 431 participants, we observed between-person and within-person correlations between school engagement and mental health. Autoregressive effects of school engagement and mental health on future levels of these variables were identified, with more consistent effects in late high school. A single cross-lagged effect from mental health to school engagement across the high school transition was identified. Decreased school engagement may signal corresponding cross-sectional changes in mental health, presenting opportunities for monitoring and intervention. Changes in school engagement or mental health may influence later within-person changes in these constructs, but associations are likely weaker and less consistent than previously assumed based on techniques that conflate between-person and within-person effects. This study assesses bidirectional and longitudinal relationships between school engagement and mental health among rising high school students. Both school engagement and mental health impacted future levels of these variables with stronger effects observed in the later high school years. Relationships between school engagement and mental health were less frequently observed than in prior studies, with an effect of mental health on future school engagement seen only across the high school transition. Interventions that promote both school engagement and mental health may be particularly beneficial, and the high school transition may be a promising time to leverage these.

Preface S. L. Christenson, A. L. Reschly, and C. Wylie.- SECTION I: WHAT IS STUDENT ENGAGEMENT?.- Jingle, Jangle, and Conceptual Haziness: Evolution and Future Directions of the Engagement Construct A. L. Reschly and S. L. Christenson.- Developmental Dynamics of Student Engagement, Coping, and Everyday Resilience E. A. Skinner and J. R. Pitzer.- Engagement across Developmental Periods D. Mahatmya, B. J. Lohman, J. L. Matjasko, and A. Feldman Farb.- Ethnicity and Student Engagement G. E. Bingham and L. Okagaki.- Student Engagement: What is it? Why does it matter? J. D. Finn and K. Zimmer.- Section I Commentary: So What is Student Engagement Anyway: Commentary on Section I J. Eccles and M. Wang.- SECTION II: ENGAGEMENT AS LINKED TO MOTIVATIONAL VARIABLES.- A Self-Determination Theory Perspective on Student Engagement J. Reeve.- Achievement Goal Theory, Conceptualization of ability/Intelligence, and Classroom Climate E. M. Anderman and H. Patrick.- School Identification K. E. Voelkl.- Self-Efficacy as an Engaged Learner D. H. Schunk and C. A. Mullen.- A Cyclical Self-Regulatory Account of Student Engagement: Theoretical Foundations and Applications T. J. Cleary and B. J. Zimmerman.- Academic Emotions and Student Engagement R. Pekrun and L. Linnenbrink-Garcia.- Students' Interest and Engagement in Classroom Activities M. Ainley.- Section II Commentary: Motivation and Engagement: Conceptual, Operational, and Empirical Clarity A. Martin.- SECTION III: ENGAGEMENT AND CONTEXTUAL INFLUENCES.- Parental Influences on Achievement Motivation and Student Engagement J. Bempechat and D. J. Shernoff.- Families as Facilitators of Student Engagement: Toward a Home-School Partnership Model J. N. Raftery, W. S. Grolnick, and E. S. Flamm.- Teacher-Student Relationships and Engagement: Conceptualizing, Measuring, and Improving the Capacity of Classroom Interactions R. C. Pianta, B. K. Hamre, and J. P. Allen.- The Role of Peer Relationships in Student Academic and Extracurricular Engagement J. Juvonen, G. Espinoza, and C. Knifsend.- Understanding Student Engagement with a Contextual Model S. Lam, B. P. H. Wong, H. Yang, Y. Liu.- Allowing Choice and Nurturing an Inner Compass: Educational Practices Supporting Students' Need for Autonomy A. Assor.- The Engaging Nature of Teaching for Competency Development R. Hipkins.- Assessment as a Context for Student Engagement S. Nichols and H. Dawson.- Section III Commentary: Socio-Cultural Contexts, Social Competence, and Engagement at School K. Wentzel.- SECTION IV: ENGAGEMENT AND STUDENT OUTCOMES.- The Relationship between Engagement and High School Dropout R. W. Rumberger and S. Rotermund.- High School Reform and Student Engagement M. H. Davis and J. M. McPartland.- The Power of Mindsets: Nurturing Engagement, Motivation, and Resilience in Students R. Brooks, S. Brooks, and S. Goldstein.- The Relations of Adolescent Student Engagement with Troubling and High-Risk Behaviors A. Griffiths, E. Lilles, M. Furlong, and J. Sidwha.- Trajectories and Patterns of Student Engagement: Evidence from a Longitudinal Study C. Wylie and E. Hodgen.- Instructional Contexts for Engagement and Achievement in Reading J. T. Guthrie, A. Wigfield, and W. You.- A Self-regulated Learning Perspective on Student Engagement C. A. Wolters and D. J. Taylor.- Classroom Strategies to Enhance Academic Engaged Time M. Gettinger and M. J. Walter.- Deep Engagement as a Complex System: Identity, Learning Power and Authentic Enquiry R. Deakin Crick.- Section IV Commentary: Outcomes of Engagement and Engagement as an Outcome: Some consensus, divergences and unanswered questions M. Janosz.- SECTION V: MEASUREMENT ISSUES, INSTRUMENTS, AND APPROACHES.- Measuring Student Engagement: The Development of a Scale for Formative Use C. W. Darr.- Systems Consultation: Developing the Assessment-to-Intervention Link with the Student Engagement Instrument J. J. Appleton.- Finding the Humanity in the Data: Understanding, Measuring & Strengthening Student Engagement E. Yazzie-Mintz and K. McCormick.- The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-Report Instruments J. A. Fredricks and W. McColskey.- Issues and Methods in the Measurement of Student Engagement: Advancing the Construct through Statistical Modeling J. Betts.- Section V Commentary: Possible New Directions in the Measurement of Student Engagement K. Samuelsen.- Epilogue S. L. Christenson, A. L. Reschly and C. Wylie.

Assessing Student Engagement in School: Development and Validation of a Four-dimensional Scale

The underachievement of African American, Latino, and American Indian students in the United States has been partially attributed to poor engagement in school (e.g., Connell, Spencer & Aber, 1994; Steele, 1997). In this chapter, we consider the role of ethnicity in student engagement. A number of factors have been posited to influence minority students’ engagement in school. Okagaki (2001) conceptualized these factors into three broad domains: the roles of the student, the family, and the school. We begin with a discussion of factors within the student, such as students’ ethnic identity beliefs, experiences with discrimination, and bicultural efficacy, and the relations of these factors to students’ engagement in school. In the second section, we examine the role that parents’ beliefs, expectations, and behaviors play in ethnic minority students’ engagement in school, paying particular attention to beliefs and values that can be attributed to parents’ cultural models of education (Gallimore & Goldenberg, 2001; Lareau, 1996). Third, we consider how factors associated with teachers, peers, and friends relate to ethnic minority students’ engagement in school. In particular, we focus on students’ access to same ethnic teachers and peers, the quality of relationships with teachers and friends, and pedagogical practices that may facilitate ethnic minority students’ engagement in school. Finally, we identify the need for stronger empirical research around the identification and amelioration of the discontinuities between home and school cultures.

Assessing measurement invariance of paranoia scales across racial and ethnic groups in the U.S.

The Student Engagement Instrument (SEI) is a relatively new inventory designed to measure cognitive andaffective engagement in school for middle and high school students. We explored the reliability and validity ofthe SEI for 122 college students. Results provided evidence for adequate to good reliability andvalidity--indicating a good fit between the data and a 4-factor structure based on Teacher-Student Relationships,Peer Support at School, Future Aspirations and Goals, and Family Support for Learning. Two factorsrepresenting affective engagement (Peer Support at School and Teacher-Student Relationships) emerged asimportant predictors of career perceptions in our college student sample. Peer Support at School also predictedcollege GPA. Facilitating continuity in the operationalization and measurement of student engagement acrosssecondary and post-secondary settings, findings also highlight the potential importance of student engagement tocareer development.

Background Although extensive research was done on gender and grade-level attitudes towards science differences, few studies have examined the measurement invariance of these scales before making cross-group comparisons. This gap raises concerns regarding the validity of the conclusions drawn from such comparisons. Purpose This study addressed this issue by investigating the measurement and structural invariance of an attitude towards science scale across gender and grade level. Sample and methods Cross-sectional data were collected from 584 Turkish middle school students (55% female, 45% male). Multi-group confirmatory factor analysis was utilized for the examination of measurement invariance. Results Results of the multi-group confirmatory factor analysis supported partial measurement invariance, characterized by the invariance of all factor loadings, invariance of all but two intercepts and invariance of all but six-item uniquenesses for gender. On the other hand, there was a reasonable level of measurement invariance and partial structural invariance for 6th and 8th grades. Conclusions Based on these results, future use of the attitudes towards science scale is recommended with paying particular attention to several items for gender and the construct of self-concept in science for grade level. The psychometric implications of the findings, as well as limitations of the study, are discussed.