COMPARISON OF TWO PROCEDURES FOR ANALYZING MULTITRAIT MULTIMETHOD MATRICES

Abstract

In the years following the publication of Campbell and Fiske's (1959) classic paper on the use of the multitrait-multimethod matrix for evaluating construct validity, several procedures have been suggested for the analysis of such matrices. Schmitt, Coyle and Saari (1977) have reviewed and critiqued eight methods of analysis. One of the procedures consisted of applying the four criteria originally suggested by Campbell and Fiske. The four criteria are: (1) the correlations among measurements of the same trait by different methods (convergent validities) should be statistically significant; (2) the convergent validities should be higher than correlations between measurements of different traits by different methods; (3) the convergent validities should be higher than correlations between measurements of different traits by the same method; and (4) the pattern of correlation between measurements of different traits by the same method should be similar from method to method. This same pattern should be evident, in the correlations between measurements of different traits by two methods, for all pairs of methods. Of the other seven procedures, two are of interest to this paper: Jackson's (1975) revised multimethod factor analysis and exploratory factor analysis. Schmitt et al. illustrated the eight procedures by applying each to the same correlation matrix. These authors noted that there was substantial evidence of convergent and discriminant validity using the Campbell-Fiske criteria. Their analysis of the matrix, using Jackson's procedure, basically supports the conclusion with regard to discriminant validity; however, the exploratory factor analysis may not. As noted by Schmitt et al., the factor analysis resulted in several trait factors exhibiting correlations of about .80. Correlations of this magnitude suggest that the traits are not very discriminable. Lomax (1978) analyzed five multitrait-multimethod matrices using exploratory factor analysis and Jackson's multimethod factor analysis. The five matrices were chosen to meet the four Campbell-Fiske criteria to differing degrees. The references containing the matrices and the degree to which these were judged to meet the criteria are (a) Mitchell (1970), none of the criteria; (b) Levin (1973), the first criterion; (c) Roshal, Frieze and Wood (1971), the first and fourth criteria; (d) Stewart (1973), all criteria; and (e) Mosher (1968), all criteria. Evaluations using multimethod factor analysis and exploratory factor analysis were in agreement for the Mitchell and Stewart matrices, but not for the other three matrices. Disagreement was greatest for the Levin and Roshal et al. matrices. The nature of the disagreement was that the multimethod factor analysis procedure suggests better discriminant validity than the exploratory factor analysis. The purpose of this paper is to report the analyses of the Levin and Roshal et al. matrices. The salient points with regard to the analysis of the Mosher matrix will be summarized briefly.