Multilevel multitrait-multimethod latent analysis of structurally different and interchangeable raters of school climate.

Abstract

Informant-based systems of assessment are common platforms for measuring a variety of educational and psychological constructs where the use of multiple informants is considered best practice. In many instances, structurally different informant types (e.g., students and teachers) are solicited on the basis of their unique roles with the target of measurement. The use of multiple informants provides an opportunity to evaluate the degree to which the obtained ratings are influenced by the trait of focus and extraneous sources that can be attributed to the rater. Data from a multilevel multitrait-multimethod design in which students (N = 35,565) and teachers (N = 9,112), from 340 middle schools, responded to items measuring 3 dimensions of school climate were evaluated through a multilevel correlated trait-correlated method latent variable model. Results indicated that ratings of school climate obtained by students and teachers demonstrated high levels of convergent validity, and that school-level ratings obtained by students and teachers were equitable in the assessment of teasing and bullying. Student ratings of support and structure yielded somewhat stronger evidence of convergent validity than ratings obtained by teachers as revealed by their respective trait factor loadings. This was explained in part by the higher levels of common method effects that were observed for teachers.