Multitrait-multimethod Matrix Research Articles

Masculinity, femininity, and self-esteem: A multitrait-multimethod analysis

A multitrait-multimethod matrix analysis was conducted to determine (1) the degree to which self-report trait and behavior measures of sex role orientation converge, (2) the degree to which measures of masculinity and self-esteem can be discriminated, and (3) the relative degree of relationship of trait and behavior measures of sex role orientation to self-esteem. The results indicated that trait and behavior measures of masculinity and femininity showed little convergence, that trait masculinity did not meet the criteria for discrimination from self-esteem, and that trait masculinity was more strongly correlated with self-esteem than was behavior masculinity. These results were stronger for women than for men.

The specificity of quality-of-life measures among the seriously ill.

The need to address the impacts of serious disease and the effectiveness of interventions has led to the development of numerous measures of the quality of life. The research reported here explores the possibility that widely used measures do not truly distinguish among separate dimensions of the quality of life in a seriously ill population, but reflect a generalized tendency among such individuals to respond negatively to interviewers' questions about their well-being. This research examines three widely used measures--the Sickness Impact Profile (SIP), the Profile of Mood States (POMS), and the McGill Pain Questionnaire (MPQ)--as applied to 536 individuals with recent cancer diagnoses. In a multitrait-multimethod matrix, correlation coefficients among measures believed to reflect the same phenomena are consistently higher than correlation coefficients among measures supposed to reflect different phenomena. In a factor analysis performed on all 536 subjects, the unrotated factor matrix indicates that no single factor explains a preponderance of the variance in individual measures. Orthogonal rotation indicates that subscales from the SIP, POMS, and MPQ generally load on factors defined by the scales of which they are part. These findings were replicated on subsamples of subjects with particularly severe disease. The study suggests that the SIP, POMS, and MPQ measure the specific dimensions their names imply, even among individuals with illnesses posing immediate threats to survival.

Validity and Structural Reliability of Multidimensional Scaling

The author examines the convergent and discriminant validity of multidimensional scaling (MDS) using the multitrait-multimethod matrix approach. An assessment also is made of the impact on MDS solutions due to embedding of stimuli in similar stimuli and due to change in stimulus domain introduced via different kinds of stimuli. The results indicate reasonable validity and that MDS solutions are fairly robust to the impact of embedding but less robust to changes in stimulus domain.

Validating the S-O-R paradigm for consumer involvement with a convenience good

The Houston and Rothschild S-O-R consumer involvement paradigm is tested for validity with measures of involvement for a shampoo purchase decision. A multitrait-multimethod matrix approach confirms the convergent and discriminant validity of the measure employed. A causal modeling analysis suggests that enduring involvement may work through situational involvement to influence consumer responses. Marketing implications are discussed.

Student-teacher concordance on dimensions of student social competence: A multitrait-multimethod analysis

Teacher and self-ratings of social competence were obtained for 669 (341 males, 328 females) grades 3–6 students using the Social Behavior Assessment—Revised (SBA-R) and a self-report scale consisting of the identical items reworded in first person. Factor analysis and multitrait-multimethod matrix analysis were used to validate construct interpretations. The previous four-factor structure of the SBA-R was not replicated. However, the three-factor structure that did emerge was better defined. Although student-teacher concordance was moderate for the Self-Control/Social Convention and Academic Responsibility factors, it was minimal for the Social Participation factor. Evidence of discriminant validity was weak. As with other studies using rating scales, method bias was in evidence. Sex differences were found in terms of variance explained by each factor and convergent validity. Discussion focuses on problems associated with construct validation in general and validation of school-related social competence in particular.

The Activity/Achievement Indicator: A Possible Alternative to the Behavioral Consistency Method of Training and Experience Evaluation

This paper describes the development of a new type of selection procedure, the activity/achievement indicator (A/AI), and compares it with the behavioral consistency method of training and experience evaluation. Total scores from these two methods were found to correlate at r = .58. Examination of the six dimension scores of each method by means of the multitrait-multimethod matrix technique revealed convergent validity for only two of the dimensions, and discriminant validity for none. Implications of the findings are discussed.

Construct Validity of Medical Clinical Competence Measures: A Multitrait-Multimethod Matrix Study Using Confirmatory Factor Analysis

This study investigates the construct validity of three methods used to evaluate clinical competence in medicine: standardized test, supervisor performance ratings, and peer performance ratings. Three attributes of clinical competence are investigated: cognitive abilities, interpersonal skills, and professional qualities. Measures representing each attribute-method combination include: National Board of Medical Examiners (NBME) examination (standardized test of cognitive abilities); two scales derived from the California Psychological Inventory (CPI) (standardized test of interpersonal skills and professional qualities); and the three scales derived from the Resident Evaluation Form (REF) (peer and supervisor ratings of all three attributes). Scores for each attribute-method combination were obtained from a convenience sample of 166 resident physicians in three primary care specialties. These scores were cast into a multitrait-multimethod matrix design and analyzed using confirmatory factor analysis. Results suggest a lack of construct validity for the CPI and REF scales, moderate convergent validity for the NBME, and substantial method variance in the REF-derived ratings. Findings are discussed in terms of the implications for a theory of medical clinical competence, further research and development in clinical competence measurement, and current measurement practice in medical education.

Évaluation Des Prédispositions À L'Action Sociale

Citizen participation in social change may depend, in part, on certain beliefs about the world and about the power of the citizen to change social conditions. To evaluate the belief in personal power and the belief that social conditions are often unjust, two scales were created. The multi-trait multi-method matrix of Campbell and Fiske was used to establish convergent and divergent validity of the English version of the scales. The scales predicted results in relevant experiments and distinguished among community groups that were, or were not, involved in social action. This article presents French versions of the two scales and reliability data from two studies. In the first study, data were gathered from a sample of 100 Acadians in Nova Scotia. The results indicated that the scales have both convergent and divergent validity. In the second study, we administered both the English and the French versions of the scales to a bilingual sample. The correlations between the English and French versions were sufficiently high for the versions to be considered equivalent.

Generalizing over conditions by combining the multitrait-multimethod matrix and the representative design of experiments.

Abstract : Analytical methods should be substituted for the current largely intuitive methods for generalizing results over conditions. Toward that end we present a methodology that combines Campbell and Fiske's (1959) multitrait multimethod matrix and Brunswik's (1956) representative design of experiments. A description of a study of expert judgement and a critique of current practices illustrate the methodology. Keywords: Multitrait multimethod; Representative design; Expert judgement; Generalization; Methodology.

Generalizing over conditions by combining the multitrait-multimethod matrix and the representative design of experiments.

Generalizing over conditions by combining the multitrait-multimethod matrix and the representative design of experiments.

Convergent and Discriminant Validity of Crites's Career Maturity Inventory Attitude Scale, Counseling Form B-1

The multitrait-multimethod matrix procedure was employed to examine the convergent and discriminant validity of the five subscales of the Career Maturity Inventory (CMI) Attitude Scale, Counseling Form B-1. The subjects were 97 ninth graders from a suburban high school. The Counseling Form subscales were intercorrelated with five short-answer test items developed for this study and designed to assess the five variables making up the Career Choice Attitudes factor in Crites's theoretical model of Career Maturity. Evidence was found to support the convergent and discriminant validity of the Decisiveness, Involvement, and Independence subscales. Since convergence was not achieved between the Orientation and Compromise subscales of the Counseling Form and the short-answer test items intended to assess these variables, the discriminant validity of these two subscales could not be examined.

The Measurement of Three Lifestyle Values: Resourcefulness, Responsibility and Relationships to others

A theory proposing three distinct, effective types of family systems [1] guided the definition and operationalization of three lifestyle value constructs. A simple, reliable and valid measure of three aspects of living – resourcefulness, responsibility and relationships to others – was developed. In a first study, 216 graduate students in six fields rated themselves on a list of thirty-six adjectives selected to comprise the three scales. Factor analysis supported the constructs. Scores were independent of gender and bore expected relationships to program affiliation. In a second study, eighty-three adults completed a forced-choice form of the instrument, rating both themselves and their spouses. They also completed the Strong-Campbell Interest Inventory, the Myers-Briggs Type Indicator, and a semi-structured individual interview. Scores from the four measures were entered into a multitrait-multimethod matrix which demonstrated convergent and discriminant validity for the constructs. The usefulness of studying the priorities that people assign to the three aspects of living is explored.

The Construct Validity of Teaching Behavior Evaluation Methods: A Multitrait-Multimethod Analysis

The construct validity of five different methods of assessing five separate teaching behavior traits was examined in the framework of a S x 5 multitrait-multimethod matrix. The results indicated that videotape rating methods produced convergent and discriminant validity coefficients which were greater than or equal to those derived from the more traditional peer and student rating methods.

Validity of self-reports of alcohol and other drug use: A multitrait-multimethod assessment.

Four methods (self-reported rating, self-reported intake, peer-reported rating, peer-reported intake) and multiple analytical criteria were employed to assess the construct validity of reports of three types of substance use (alcohol, marijuana, and cigarettes) in a study of 194 college students who were members of friendship pairs. The resulting multitrait-multimethod matrix was analyzed in the traditional fashion as well as with confirmatory factor analysis. Convergent and discriminant validity for the three substance use traits were quite adequate, although the validity of one measure (the peer-reported intake measure of alcohol use) may be considered inadequate according to some criteria. This study demonstrates the usefulness of multimethod assessment and confirmatory factor analysis for research on construct validity and for derivation of more accurate measures of drug use.

A Preliminary Study to Assess Validity of the Personal Style Inventory

The Personal Style Inventory was designed to assess Jungian personality types. The present study determined its reliability and validity. The Myers-Briggs Type Indicator was used as a criterion and the multitrait-multimethod matrix method was used to assess reliability and validity. Reliability coefficients between the opposite sides of each scale were —1.00 and validity coefficients between corresponding scales of the Personal Style Inventory and the Myers-Briggs Type Indicator ranged between .52 and .70. The lowest validity coefficients were with Extraversion-Introversion scales. Significant differences between validity coefficients were also found between participants' congruent and incongruent scores on the Extraversion-Introversion and Judgment-Perception scales. The results suggest feasibility of using the Personal Style Inventory to assess Jung's personality types.

Validity of self-reports of alcohol and other drug use: a multitrait-multimethod assessment.

Four methods (self-reported rating, self-reported intake, peer-reported rating, peer-reported intake) and multiple analytical criteria were employed to assess the construct validity of reports of three types of substance use (alcohol, marijuana, and cigarettes) in a study of 194 college students who were members of friendship pairs. The resulting multitrait-multimethod matrix was analyzed in the traditional fashion as well as with confirmatory factor analysis. Convergent and discriminant validity for the three substance use traits were quite adequate, although the validity of one measure (the peer-reported intake measure of alcohol use) may be considered inadequate according to some criteria. This study demonstrates the usefulness of multimethod assessment and confirmatory factor analysis for research on construct validity and for derivation of more accurate measures of drug use.

Prediction of clinical performance of medical students: an integrative approach to evaluation.

Prediction of clinical performance of medical students in the clerkship phase was made possible in the earliest phase of medical education by a comprehensive and integrative evaluation instrument entitled 'Integrative Examination' (INTEX). A proportional sample of the content learned throughout the year is integrated for measuring knowledge, attitudes and skills on various cognitive levels within the clinical context of patient problems. INTEX was found to be a reliable instrument. Also it scores highly correlated with all the constituents of later clinical assessment, creating a valid multitrait-multimethod matrix. The educational implication of an integrative approach to evaluation is discussed.

Convergent and discriminant validity of selected masculinity and femininity scales

Using Campbell and Fiske's (1959) multitrait-multimethod matrix technique and Jackson's (1975) multimethod factor analysis, convergent and discriminant validity were investigated for masculinity and femininity scales from the Adjective Check List, Bem Sex Role Inventory, and PRF ANDRO Scale as compared with measures of dominance and nurturance. Results indicated very good convergent and discriminant validity for the masculinity and femininity scales from the Adjective Check List and Bem Inventory, but not for the PRF ANDRO Scale. Implications of these results for research on sex roles and psychological androgyny are discussed.

The Factorial Invariance of Student Evaluations of College Teaching

University instructors who taught the same course at least four times over a 4-year period were evaluated by different groups of students in each of these four classes (N = 316 instructors, 1,264 classes, 31,322 students). Factor analyses of the class average ratings clearly replicated the nine evaluation dimensions found in earlier studies. A multitrait-multimethod (MTMM) matrix was formed in which the nine dimensions were the multiple traits, and the four different offerings of the same course were the multiple methods. Across all nine dimensions the average convergent validity was r = .68. MTMM analyses with both the Campbell-Fiske MTMM criteria and confirmatory factor analysis (using LISREL) demonstrated substantial convergent and discriminant validity, but little method/halo effect. The confirmatory factor analysis also demonstrated the generalizability of the multivariate structure of the ratings across the four sets of courses. Taken together, these findings provide stronger evidence for the multidimensionality of student ratings than does any previously published research and describe a powerful analytic tool for use in student evaluation research.

The validity of experimental pain measures

Forty subjects served in a study investigating the characteristics of experimental pain measures. Subjects indicated when their pain threshold and tolerance levels had been reached with each of three Stressors: cold, pressure, and electrical shock. Using the multitrait-multimethod matrix procedure, the measures of threshold and tolerance were found to show both generality and discriminant validity across Stressors. Threshold judgements, which emphasize discrimination of nociceptive quality, and tolerance decisions, which indicate an unwillingness to receive more intense stimuli, are not equivalent measures of responsiveness. Both should be obtained in studies involving experimental pain. Stressors, while related, are also not equivalent. Minimum method variance was associated with the discomfort produced by electrical pulse trains.