Multitrait-multimethod Study Research Articles

Gratitude, indebtedness, and God: gratitude and indebtedness to God are more strongly correlated to each other than gratitude and indebtedness to a person or an object

ABSTRACT Four studies investigated the structural relationship between gratitude and indebtedness to God. In a longitudinal study (Study 1), gratitude and indebtedness to God (rs =.70 and .57) remained stable over 42–50 days, with a stronger correlation between gratitude and indebtedness to God than between dispositional gratitude and indebtedness. The gratitude-indebtedness to God correlation was stronger among less religious than more religious participants. A multitrait-multimethod study (Study 2) replicated Study 1 with self-reports and informant-reports. Study 3 replicated Studies 1 and 2 and found that gratitude to God was more distinct from indebtedness to God among those who see God as benevolent, mystical, and limitless. Study 4 compared gratitude and indebtedness to God, mothers, and mobile phones, and found that the gratitude-indebtedness correlation for God was much stronger than for mothers or phones. Overall, gratitude and indebtedness to God appear far more closely intertwined than gratitude and indebtedness to other entities.

Measuring the variability of personality traits with interval responses: Psychometric properties of the dual-range slider response format

Measuring the variability in persons’ behaviors and experiences using ecological momentary assessment is time-consuming and costly. We investigate whether interval responses provided through a dual-range slider (DRS) response format can be used as a simple and efficient alternative: Respondents indicate variability in their behavior in a retrospective rating by choosing a lower and an upper bound on a continuous, bounded scale. We investigate the psychometric properties of this response format as a prerequisite for further validation. First, we assess the test–retest reliability of factor-score estimates for the width of DRS intervals. Second, we test whether factor-score estimates of the visual analog scale (VAS) and the location of DRS intervals show convergent validity. Third, we investigate whether factor-score estimates for the DRS are uncorrelated between different personality scales. We present a longitudinal multitrait-multimethod study using two personality scales (Extraversion, Conscientiousness) and two response formats (VAS, DRS) at two measurement occasions (6–8 weeks apart) for which we estimate factor-score correlations in a joint item response theory model. The test–retest reliability of the width of DRS intervals was high (ρ^≥.73\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hat{\\rho } \\ge .73$$\\end{document}). Also, convergent validity between location scores of VAS and DRS was high (ρ^≥.88\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hat{\\rho } \\ge .88$$\\end{document}). Conversely, discriminant validity of the width of DRS intervals between Extraversion and Conscientiousness was poor (ρ^≥.94\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hat{\\rho } \\ge .94$$\\end{document}). In conclusion, the DRS seems to be a reliable response format that could be used to measure the central tendency of a trait equivalently to the VAS. However, it might not be well suited for measuring intra-individual variability in personality traits.

Psychometric properties of the German version of the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) in clinical routine settings: study design and protocol of a multitrait–multimethod study

IntroductionThe aim of this study is to investigate the diagnostic accuracy, psychometric properties and clinical utility of the German version of the Clinician-Administered Post-Traumatic Stress Disorder (PTSD) Scale for Diagnostic...

Languages for different health information readers: multitrait-multimethod content analysis of Cochrane systematic reviews textual summary formats

BackgroundAlthough subjective expressions and linguistic fluency have been shown as important factors in processing and interpreting textual facts, analyses of these traits in textual health information for different audiences are lacking. We analyzed the readability and linguistic psychological and emotional characteristics of different textual summary formats of Cochrane systematic reviews.MethodsWe performed a multitrait-multimethod cross-sectional study of Press releases available at Cochrane web site (n = 162) and corresponding Scientific abstracts (n = 158), Cochrane Clinical Answers (n = 35) and Plain language summaries in English (n = 156), French (n = 101), German (n = 41) and Croatian (n = 156). We used SMOG index to assess text readability of all text formats, and natural language processing tools (IBM Watson Tone Analyzer, Stanford NLP Sentiment Analysis and Linguistic Inquiry and Word Count) to examine the affective states and subjective information in texts of Scientific abstracts, Plain language summaries and Press releases.ResultsAll text formats had low readability, with SMOG index ranging from a median of 15.6 (95% confidence interval (CI) 15.3–15.9) for Scientific abstracts to 14.7 (95% CI 14.4–15.0) for Plain language summaries. In all text formats, “Sadness” was the most dominantly perceived emotional tone and the style of writing was perceived as “Analytical” and “Tentative”. At the psychological level, all text formats exhibited the predominant “Openness” tone, and Press releases scored higher on the scales of “Conscientiousness”, “Agreeableness” and “Emotional range”. Press releases had significantly higher scores than Scientific abstracts and Plain language summaries on the dimensions of “Clout”, and “Emotional tone”.ConclusionsAlthough the readability of Plain language summaries was higher than that of text formats targeting more expert audiences, the required literacy was much higher than the recommended US 6th grade level. The language of Press releases was generally more engaging than that of Scientific abstracts and Plain language summaries, which are written by the authors of systematic reviews. Preparation of textual summaries about health evidence for different audiences should take into account readers’ subjective experiences to encourage cognitive processing and reaction to the provided information.

The nature of creativity: The roles of genetic factors, personality traits, cognitive abilities, and environmental sources.

This multitrait multimethod twin study examined the structure and sources of individual differences in creativity. According to different theoretical and metrological perspectives, as well as suggestions based on previous research, we expected 2 aspects of individual differences, which can be described as perceived creativity and creative test performance. We hypothesized that perceived creativity, reflecting typical creative thinking and behavior, should be linked to specific personality traits, whereas test creativity, reflecting maximum task-related creative performance, should show specific associations with cognitive abilities. Moreover, we tested whether genetic variance in intelligence and personality traits account for the genetic component of creativity. Multiple-rater and multimethod data (self- and peer reports, observer ratings, and test scores) from 2 German twin studies-the Bielefeld Longitudinal Study of Adult Twins and the German Observational Study of Adult Twins-were analyzed. Confirmatory factor analyses yielded the expected 2 correlated aspects of creativity. Perceived creativity showed links to openness to experience and extraversion, whereas tested figural creativity was associated with intelligence and also with openness. Multivariate behavioral genetic analyses indicated that the heritability of tested figural creativity could be accounted for by the genetic component of intelligence and openness, whereas a substantial genetic component in perceived creativity could not be explained. A primary source of individual differences in creativity was due to environmental influences, even after controlling for random error and method variance. The findings are discussed in terms of the multifaceted nature and construct validity of creativity as an individual characteristic. (PsycINFO Database Record

The development and validation of the Self-Report Measure of Cognitive Abilities: A multitrait–multimethod study

The expansion in our understanding of the structure of differential cognitive abilities afforded by the Cattell–Horn–Carroll (CHC) model has brought with it the need to provide practitioners with efficient and effective methods for screening which abilities most critically require assessment. A Self-Report Measure of Cognitive Abilities could assist practitioners with this process. This article outlines the development and initial validation of the Self-Report Measure of Cognitive Abilities (SRMCA), a multi-item measure designed to indicate cognitive functioning in the CHC ability areas of Fluid reasoning (Gf), Comprehension-knowledge (Gc), and Visual processing (Gv). Validity was initially investigated and supported using exploratory factor analysis, and then cross-validated on a second sample using structural equation modelling (SEM). Subsequently, SEM based multitrait–multimethod analysis of the second sample confirmed convergent validity for the Gc and Gv subscales, but not the Gf subscale. The extent of method variance influence on the SRMCA was found to be non-existent, a markedly different result to that found for the single-item self-estimates of cognitive abilities. Results thus indicate that the use of multiple and specific items allows for self-ratings of distinct cognitive ability areas to be independent of one another. Suggestions for future research aimed at extending the current study are provided.

A multitrait–multimethod study of assessment instruments for complex problem solving

Recently published studies on Complex Problem Solving (CPS) suggest that assessments of CPS using multiple complex systems are only moderately related to tests of classical cognitive abilities. Further, CPS assessments show incremental validity beyond tests of other cognitive abilities when predicting relevant outcomes. However, these empirical accounts have relied on single CPS assessment instruments. We do not know whether these findings will generalize to the construct level across different CPS assessment instruments. To answer this question, we tested a sample of N=339 German university students who completed three CPS assessment instruments based on multiple complex systems (MicroDYN, the Genetics Lab, and MicroFIN) and the matrices subtest of the Intelligence Structure Test as measure of reasoning. Students further reported their school grades. Analyses including latent multitrait–multimethod models provided support for the conceptualization of CPS as a complex cognitive ability. Results indicated that different CPS assessment instruments showed sufficient convergent validity (with a consistency mostly between .50 and .60). In addition, we found evidence for the divergent validity of CPS from reasoning (reasoning predicted two CPS facets, knowledge and control, βKNOW=.49 and βCON=.53, respectively). In the prediction of academic achievement, CPS explained variance in natural science grades after we controlled for reasoning (βCPS=.22), whereas social science grades were not predicted. Our findings suggest that the validity of CPS generalizes across different measurement instruments.

A multigroup multitrait-multimethod study in two countries supports the validity of a two-factor higher order model of personality

This study examined the factor structure of the Big Five Inventory and tested the hypothesis that the five personality dimensions could be summarized by two higher order factors, namely, plasticity and stability, using multigroup multitrait-multimethod confirmatory factor analyses. We tested the higher order model in two young adult samples drawn from Germany and Turkey. Adequate inter-rater agreement between self- and informant reports was obtained. Among the models tested, a two-factor model was the most parsimonious model in which the first factor included Agreeableness, Conscientiousness, and low Neuroticism; and the second factor included Extraversion and Openness to Experience. Invariance of this model was supported by multiple-group analyses, suggesting a lack of variability across samples.

Targeted Peer Victimization and the Construction of Positive and Negative Self-Cognitions: Connections to Depressive Symptoms in Children

The goal was to examine the relation of covert/relational and overt/physical targeted peer victimization (TPV) to each other, to positive and negative self-cognitions, and to symptoms of depression. In a sample of elementary and middle school children, TPV was assessed by self-report, peer-nomination, and parent report in a multitrait–multimethod study. Positive and negative self-cognitions and depressive symptoms were assessed by self-report. Confirmatory factor analytic results support the convergent and discriminant validity of these two types of TPV. Both kinds of TPV were significantly related to positive and negative self-cognitions as well as self-reported depressive symptoms; however, structural equation modeling revealed that the effects of covert/relational TPV accounted for the effects of overt/physical TPV. In exploratory analyses, positive and negative self-cognitions explained the relation between TPV and depressive symptoms.

A developmental model of maternal and child contributions to disruptive conduct: the first six years

The parent-child relationship is considered important for children's future conscience, and conscience is seen as protecting them from disruptive behavior problems, but specific mechanisms of this developmental process are rarely studied. This multi-trait multi-method study examined, in a longitudinal design, paths linking early maternal responsiveness to the child with the child's future conscience and disruptive behavior in 102 mother-child dyads. We tested a conceptual model where maternal responsiveness to the child, observed at 7 and 15 months, engenders a responsive stance in the child, observed at 25 and 38 months; that stance, in turn, becomes enduring and generalized, promoting multiple aspects of the child's conscience, observed at 52 months. In turn, conscience serves as a protective factor from disruptive behavior problems, rated by mothers and fathers at 67 months. The postulated paths were examined using sequential regressions and mediation effects were tested using bootstrapping analyses. Child responsive stance at 25-38 months fully mediated the link between maternal responsiveness in infancy and conscience at 52 months, and conscience fully mediated the link between child responsive stance and future disruptive behavior at 67 months. Examination of developmental links among early maternal behavior, the child's responsive stance toward the mother, conscience, and disruptive behavior is a promising step toward elucidating mechanisms of children's adaptive and maladaptive trajectories.

Assessing personality in early adolescence through self-report and other-ratings a multitrait-multimethod analysis of the BFQ-C

The main purpose of this paper is to provide an assessment of the Big Five Questionnaire – Children construct validity by means of a multitrait-multimethod study among junior high school preadolescents from 13 to 14 years old. Self-report, teacher and parent ratings were gathered. Confirmatory Factor Analyses were performed to study convergent and discriminant validity. The so-called Correlated Uniquenesses model resulted in an adequate fit to the empirical data. Convergent validity was supported for all factors, especially between preadolescents and their parents. Discriminant validity is achieved only in a “weak” sense since the high correlation between Intellect/Openness and Conscientiousness. Furthermore results indicate that method effect is not negligible for all raters and all traits.

Reviving the search for social intelligence – A multitrait-multimethod study of its structure and construct validity

The present study conceptualized social intelligence as a performance construct and thus is based on Thorndike’s idea of social intelligence as a mental ability distinct from abstract and mechanical intelligence (1920). So far, there has been only limited success in identifying a unitary social intelligence construct distinct from academic intelligence. The objects of our study were twofold. First, we intended to demonstrate the multidimensionality of social intelligence. We postulated three cognitive ability domains (i.e., social understanding, memory, and knowledge). These domains were operationalized in a multitrait-multimethod design applying verbal, pictorial, and video-based performance measures. Secondly, we intended to demonstrate that social intelligence can be differentiated from academic intelligence. One hundred eighteen high school and first year psychology students (80 of them females, mean age 19.7 years) were tested. Confirmatory factor analysis supported the postulated factor-structure within social intelligence. Correlational and regression analysis yielded generally low validity coefficients between social and academic intelligence except for the social memory tests. Still, residual analysis showed unique common variance within the social memory domain. Consequently, the study provided evidence for the structure of the social intelligence performance model according to the postulated design and demonstrated the discriminability of SI from academic intelligence.

How valid are measures of beliefs about the causes of illness? The example of myocardial infarction

In Multi-Trait Multi-Method (MTMM) studies of causal attributions for laboratory events, there is little evidence of convergent and discriminant validity for attribution measures. We report the first MTMM study to investigate the validity of two methods of eliciting causal beliefs for an illness, specifically, myocardial infarction. Adult respondents (N = 107) listed causes of MI, then completed questionnaire rating scales for causal beliefs for MI. Measures were compared using both Campbell and Fiske's approach to MTMM analyses, and a Confirmatory Factor Analysis approach. Neither single item measures causal beliefs, nor scales of causal beliefs derived using exploratory factor analysis provided much evidence of convergent and discriminant validity. Confirmatory factor analysis showed that a model containing only causal belief factors provided a moderately good fit to the data. Adding a questionnaire method factor significantly improved the fit of the model, as well as substantially changing the pattern of factor loadings: loadings of questionnaire items on causal belief factors were markedly reduced. These results highlight major problems with the measurement of causal beliefs, and in particular question the validity of factor analysis of questionnaire measures of causal beliefs. They also suggest that at least some of the MI causal belief factors reported in the literature are artefacts of common questionnaire method variance.

The structure of emotional expressivity: Each emotion counts.

Several models of the dimensionality of emotional expressivity were examined in a multitrait-multimethod study. Targets and peer raters completed measures of the target's emotional expressivity (Berkeley Expressivity Questionnaire. BEQ; J. J. Gross & O. P. John, 1995; and a measure of emotion-specific expression) and the Big 5 personality dimensions. The results of structural equation modeling and analysis of variance revealed that an emotion-specific model was superior to models of valence-specific or unidimensional expressivity. The distinct emotions differed in their relations with the dimensions of the 5-factor model. These results were corroborated by self- and other reports. Finally, the degree of convergence between self- and other ratings differed between emotions, demonstrating the multidimensional character of emotional expressivity.

Effects of Validity and Bias on Gender Differences in the Appraisal of Children's Competence: Results of MTMM Analyses in a Longitudinal Investigation

Third graders (N = 598) and 6th graders (N = 630) participated in a 3-year, 6-wave, 2-cohort study of perceived competence. As part of a multitrait-multimethod longitudinal study, teachers, peers, and students completed measures of perceived academic competence, social acceptance, athletic competence, physical appearance, and behavioral conduct twice per year. The measures of competence manifested significant levels of convergent and discriminant validity for most domains at all grade levels and for both genders. Validity differences emerged between sources of information, between age groups, and between domains of competence. Evidence of bias also emerged, such that self-report measures systematically underestimated girls' competence in all domains and teacher ratings overestimated boys' competence in athletics. Corrected for bias, latent mean differences between boys and girls were still significant. Girls were perceived as more competent than boys in most domains, particularly behavioral conduct. Boys were perceived as more competent than girls in athletic competence. Developmental changes in perceived competence between 3rd and 8th grades were unremarkable.

Social and academic intelligences: a multitrait–multimethod study of their crystallized and fluid characteristics

Abstract The purposes of the present study were: (1) to replicate previous research on social intelligence that shows social intelligence to be multidimensional in nature and distinguishable from academic intelligence, (2) to extend prior research by assessing whether the crystallized/fluid distinction commonly discussed in the academic intelligence literature was applicable to the domain of social intelligence, and (3) to explore whether a hierarchical model of social and academic intelligences was consistent with the data. One hundred and sixty-nine university students completed verbal, pictorial and self-report measures of four constructs: social knowledge (hypothesized to reflect crystallized social intelligence), social inference (hypothesized to reflect fluid social intelligence), crystallized academic and fluid academic intelligences. In addition, other-report measures were collected for these constructs in this multitrait–multimethod study. Confirmatory factor analyses replicated previous research, documenting that the four trait constructs showed convergent and discriminant validities. Similar analyses also extended prior research by showing that the crystallized/fluid distinction might be applicable in the social intelligence domain (although the evidence was stronger for crystallized social intelligence than fluid social intelligence) and that a hierarchical model fits the data well.

Assessing Convergent and Discriminant Validity of the Influence Behavior Questionnaire

The purpose was to study convergent and discriminant validity of the 1997 Influence Behavior Questionnaire (short version) used to assess lateral attempts at influence. 479 German employees participated in a multitrait-multimethod study with nine trait measures (Rational Persuasion, Inspirational Appeal, Consultation, Ingratiation, Exchange, Personal Appeal, Coalition, Legitimating, Pressure) and two methods (self- and peer-ratings). Data, analyzed with incomplete structural equation models, supported scales' convergent but not discriminant validity of the influence scales.

Discrimination of two aspects of cognitive-social intelligence from academic intelligence.

The present study proposed a theoretical division of cognitive-social intelligence into declarative and procedural social knowledge (crystallized social knowledge) and flexible knowledge application aspects (social-cognitive flexibility) distinct from academic problem solving. Pictorial, verbal, self-, and teacher report measures of social-cognitive flexibility, crystallized social knowledge, and academic problem solving were collected for 169 high school seniors (102 girls and 67 boys) in a multitrait-multimethod study. Confirmatory factor analyses supported the discriminability of social-cognitive flexibility from academic problem solving and crystallized social knowledge. Crystallized social knowledge was not discriminable from academic problem solving. Significant correlations between these factors and teacher reports of social behavior confirmed that flexible application of social knowledge is an important aspect of social competence.

Modeling causal relations between academic and social competence and depression: a multitrait-multimethod longitudinal study of children.

The authors obtained self-reports, peer nominations, teacher ratings, and parent reports of depression and social and academic competence on 490 3rd graders and 455 6th graders near the beginning and end of the school year. Confirmatory factor analysis and structural equation modeling revealed that (a) measures showed significant convergent and discriminant validity; (b) within-wave correlations between constructs were large and significant, although the depression-social competence correlation was larger than the depression-academic competence correlation; (c) the cross-wave stability of all constructs was remarkably high; and (d) social competence at Wave 1 predicted depression at Wave 2 for 6th graders after controlling for depression at Wave 1. Depression did not predict change in either academic or social competence over time. Implications for competence-based and failure-based models of child depression are discussed.

Validity of three clinical performance assessments of internal medicine clerks.

To analyze the construct validity of three methods to assess the clinical performances of internal medicine clerks. A multitrait-multimethod (MTMM) study was conducted at the Case Western Reserve University School of Medicine to determine the convergent and divergent validity of a clinical evaluation form (CEF) completed by faculty and residents, an objective structured clinical examination (OSCE), and the medicine subject test of the National Board of Medical Examiners. Three traits were involved in the analysis: clinical skills, knowledge, and personal characteristics. A correlation matrix was computed for 410 third-year students who completed the clerkship between August 1988 and July 1991. There was a significant (p < .01) convergence of the four correlations that assessed the same traits by using different methods. However, the four convergent correlations were of moderate magnitude (ranging from .29 to .47). Divergent validity was assessed by comparing the magnitudes of the convergence correlations with the magnitudes of correlations among unrelated assessments (i.e., different traits by different methods). Seven of nine possible coefficients were smaller than the convergent coefficients, suggesting evidence of divergent validity. A significant CEF method effect was identified. There was convergent validity and some evidence of divergent validity with a significant method effect. The findings were similar for correlations corrected for attenuation. Four conclusions were reached: (1) the reliability of the OSCE must be improved, (2) the CEF ratings must be redesigned to further discriminate among the specific traits assessed, (3) additional methods to assess personal characteristics must be instituted, and (4) several assessment methods should be used to evaluate individual student performances.