QUESTIONNAIRE VALIDATION BY CONFIRMATORY FACTOR ANALYSIS: AN IMPROVEMENT OVER MULTITRAIT‐MULTIMETHOD MATRICES*

Abstract

ABSTRACTThe Campbell and Fiske criteria for assessing the construct validity of multitrait‐multimethod (MTMM) matrices has had a long history of use. While various statistical techniques, including ANOVA, have attempted to provide rigor to the MTMM matrix design, numerous problems still remain unsolved. Part of the problem in using an MTMM matrix is the assumption of measurement independence. This study attempts to illustrate the misleading inferences that often occur from MTMM analysis when method variance overlap is not accurately assessed. Three questionnaires were designed that were not method independent. Traditional procedures for assessing MTMM matrices suggested the three scaling formats used were not burdened with unusual method variance. A reanalysis of the MTMM matrix employing a Confirmatory Factor Analysis technique illustrated that method variance was a problem. Finally, the need for studies that concentrate on the nature of method variance, its causes and effects, is discussed.