Multitrait Multimethod Research Articles

Measuring the psychological burden of women with pelvic floor complaints: The psychometric characteristics of a new instrument.

To be able to optimize pelvic healthcare, it would be helpful to specifically assess women's psychological burden with pelvic floor complaints. In the absence of such an instrument, a new instrument was developed to measure this burden in women who seek help. In previous research, a comprehensive overview was yielded of women's restrictions and distress with pelvic floor complaints, and a conceptual model was developed of seven types of distress that were reflected by 33 statements. The present study was performed to investigate the psychometric properties of the new instrument, termed the Pelvic Floor Complaint-related Psychological Burden Inventory (PFC-PBI). In an online survey data was collected from women with and without pelvic floor complaints on the 33 statements. The internal consistency of the types of distress was tested using item-total correlation analysis, Principal Component and Confirmatory Factor Analyses were performed, and the convergent and divergent validity of the types of distress was examined against existing questionnaires using the Multi-Trait Multi-Method methodology. Based on the factor analyses, a 10-item instrument was tested. Outcomes show excellent internal consistency of this instrument, comprising a single component. The PFC-PBI demonstrated satisfactory convergent and divergent validity. This new measure appears to be a promising tool to inventory the psychological burden of women suffering from pelvic floor complaints. Research into its further development, implementation, and clinical use appears warranted.

Personalideer: A comprehensive review of personality studies in cervids

Understanding animal personality, that is, consistent behavioural variation among individuals, is essential for elucidating the evolutionary origins of human personality. Despite extensive research on personality in diverse taxa, its significance in deer (i.e. cervids, family Cervidae), a taxon with diverse socioecological strategies, remains largely unexplored. This review summarizes the deer personality literature and highlights the knowledge gap in the studied species, personality assessment methods and traits, the general goal of the study, and implications for human personality research. We identified 32 eligible studies on seven species of deer, representing only a small portion of existing deer taxonomic diversity. More than half of the articles focused on three common deer species, roe deer ( Capreolus capreolus), fallow der ( Dama dama), and elk ( Cervus canadensis). Most studies evaluated reactions to capture, handling, and approaching humans, interpreting these behaviours as ‘Boldness’ or a ‘Coping style’. Also, most articles investigated the personality links to ecological variables, followed by physiological measures, and management applications. We advise researchers to move beyond measuring single traits and adopt a multitrait–multimethod approach, as is common in human personality research. Finally, deer and other animal models might be viable alternatives for studying adaptive value and genetic and environmental underpinnings of personality when research on humans is complicated.

Translation and pilot validation of the Moroccan version of Stroke Impact Scale (SIS) 3.0

ABSTRACT Introduction The Stroke Impact Scale 3.0 appears to be a promising measure of health-related quality of life for stroke patients. However, the lack of a cross-cultural adaptation in Arabic dialect may limit its use in the Moroccan context. The objective of this study was to carry out a transcultural adaptation and pilote validation of the Stroke Impact Scale 3.0 in Moroccan Arabic dialect, commonly called « Darija ». Patients and methods A cross-sectional design was used for this study. The cross-cultural adaptation of the SIS 3.0 scale into Moroccan Darija was carried out following the guidelines proposed by Beaton et al (2000). Ceiling and floor effects were calculated for all scales. The internal consistency of multi-item scales was assessed using Cronbach’s α coefficient. Convergent and divergent validity were evaluated using the Multi-Trait Multi-Method Correlation Matrix method. Results A total of 102 patients, with a mean age of 49 ± 12.9 years, participated in the study. The results revealed that the dimensions of the translated scale had no floor effect and no ceiling effect. The internal consistency of SIS 3.0 (Cronbach’s alpha) was calculated, and all dimensions had good reliability, above the threshold of 0.70 (between 0.705 and 0.945). The results showed that the dimensions and their own items had correlation coefficients greater than 0.4, with good convergent and divergent validity. Conclusion The darija version appears to be culturally acceptable with good psychometric properties and can be used to measure the quality of life of stroke survivors in Morocco.

An In-depth Examination of Validity Assessment: Exploring Diverse Methodologies and Dimensions of Validity in Social Research Studies

Validity refers to the extent to which a test accurately measures what it claims to measure. Validity is a matter of degree. The study is done to know the various methods of validity. Validity is a fundamental aspect of research quality, which is why it is important to assess the various methods of validity. Descriptive research was used for the study. It was examined that face validity refers to an indicator that seems to measure its construct “on its face”. Content validity is a form of judgmental validity, that involves determining if a test adequately represents the content domain. Criterion-related validity methods include concurrent and predictive approaches. Construct validity assesses how well theoretical ideas are translated into measurable variables. Face validity involves examining whether a measurement appears suitable based on its operationalization on the face of it. Lawshe's content validity ratio quantifies expert agreement on the essentiality of individual items in a test or scale. In contrast, the content validity index considers overall expert agreement on the representativeness of items collectively within a test or scale. It considers the agreement among experts regarding the content validity of the entire set of items. Cohen’s Kappa is used to assess the level of agreement between two or more raters or observers when coding or categorizing data. The kappa coefficient is particularly useful when chance agreement among raters is possible. Item Objective Congruence evaluates the efficacy of items in measuring one or more objectives, accomplished by an unbiased expert panel provides evidence of content validity. The most significant assessment at this stage is determining if items and objectives are congruent. Concurrent and predictive validity are assessed using Pearson's correlation and Spearman rank correlation coefficients respectively. The Multi-trait Multimethod Matrix (MTMM), evaluates the construct validity of measures. Validity is essential for ensuring credible and meaningful research findings by enhancing the trustworthiness of the measuring instruments.

The Operations Triad Model and Youth Mental Health Assessments: Catalyzing a Paradigm Shift in Measurement Validation

ABSTRACT Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies across informants’ reports (i.e., informant discrepancies). Researchers should arguably treat these informant discrepancies as domain-relevant information: data relevant to understanding youth mental health domains (e.g., anxiety, depression, aggression). Yet, historically, in youth mental health research as in many other research areas, one set of paradigms has guided interpretations of informant discrepancies: Converging Operations and the Multi-Trait Multi-Method Matrix (MTMM). These paradigms (a) emphasize shared or common variance observed in multivariate data, and (b) inspire research practices that treat unique variance (i.e., informant discrepancies) as measurement confounds, namely random error and/or rater biases. Several years ago, the Operations Triad Model emerged to address a conceptual problem that Converging Operations does not address: Some informant discrepancies might reflect measurement confounds, whereas others reflect domain-relevant information. However, addressing this problem requires more than a conceptual paradigm shift beyond Converging Operations. This problem necessitates a paradigm shift in measurement validation. We advance a paradigm (Classifying Observations Necessitates Theory, Epistemology, and Testing [CONTEXT]) that addresses problems with using the MTMM in youth mental health research. CONTEXT optimizes measurement validity by guiding researchers to leverage (a) informants that produce domain-relevant informant discrepancies, (b) analytic procedures that retain domain-relevant informant discrepancies, and (c) study designs that facilitate detecting domain-relevant informant discrepancies.

A framework for digital talent assessment: Guidelines and applications

Rapid rises in technology and digitisation in Talent Assessment practices have not been met with an equal advance in research, methodology, and rigor as applies guidelines for best practice use. This risks an inevitable quality deficit in the application of these technologies and leave human resource (HR) Practitioners unempowered to optimise the benefits of a democratised assessment environment. Multi-trait multi-method (MTMM) assessment protocols provide for a framework for best use application of emerging digital talent assessment tools. In this synthesis review study, we outline guidelines for use of digital talent management and development tools and propose a range of practical strategies. We conclude by providing a practical framework for occupationally specific competence assessment suggestions for future research.

Convergent and Discriminant Validity of the English Prosodic Phonology Processing Test (EPPPT): A Multi-Trait Multimethod Approach

Most of the prosody perception tests do not capture how listeners perceive and interpret stress, tone, and intonation in the process of listening comprehension. The current study developed the English Prosodic Phonology Processing Test (EPPPT). A sample of 240 Luo speaking high school students were tested using Confirmatory Factory Analysis (CFA) in a multi-trait multimethod matrix. Four traits were measured including word prosody, sentence prosody, juncture, and discourse prosody. Three methods were used including the picture selection task, stress assignment task, and chunking task. CFA confirmed the current taxonomy of the diverse traits of English prosodic phonology: word stress, sentence stress, open and closed junctures, and discourse. The methods yielded statistically significant differences among the discriminant validity of these traits. The model fit was better when the different prosodic traits were specified (convergent validity), while the methods of testing yielded distinguishable, unique types of information about prosodic phonology processing. Using a battery of five tests, the results of EPPPT showed that the traits are quite independent of each other and the method effect is not significant.
 Keywords: reliability, validity, EPPPT, Prosody, MTMM

The role of fairness perceptions in patient and employee health: A multilevel, multisource investigation.

This research extends the group engagement model (GEM) to examine how fairness judgments implicate both organizational-level and individual-level outcomes, including patient health and satisfaction (i.e., patient health outcomes) and employee health. Based on the social identity arguments of the GEM, we argue that fair career advancement procedures at the organizational level and experiences of discrimination at the individual level are indicators of identity-based evaluations of fairness. Utilizing annual staff survey data from the National Health Service (NHS) in the U.K. (n = 147 hospitals with n = 60,602 employees), we observe that organizational fairness of career advancement procedures significantly relates to patient health through the hospital-level mediator, employee voice. Individual fairness of an employee's personal experience with discrimination significantly relates to employee health through the individual-level mediator, psychological safety. Results support the three-stage indirect effect from organizational-level fairness to employee health via individual-level fairness and individual-level psychological safety. In supplemental studies, measurement limitations are addressed through multitrait multimethod matrix and content validation approaches. These results indicate that the archival NHS measures sufficiently operationalize the constructs of interest providing further support for the hypothesized model. The theoretical and practical implications of this work for multilevel conceptualizations of fairness and healthcare organizations are presented. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Convergence Between Teacher Self-Report and School Administrator Observation Ratings Using the Classroom Strategies Assessment System

The Classroom Strategies Assessment System (CSAS) is a multi-rater, multi-method (direct observation and rating scale methodology) assessment of teachers’ use of research-based instructional and behavior management strategies. The present study investigated the association between teacher self-report and school administrator ratings using the CSAS Teacher (CSAS-T) and Observer (CSAS-O) Forms in 15 high-poverty U.S. charter schools. The CSAS-T and CSAS-O were designed to be used concurrently as a valid formative assessment of teacher practice. Findings include small, but statistically significant correlations between the CSAS-T and CSAS-O. Analysis of a multi-trait–multi-method (MTMM) matrix found teachers and observers to be measuring different constructs. No mean score differences were found between teacher self-reported instruction and behavior management strategy use compared with school administrators’ observed ratings. Furthermore, school administrators and teachers have similar ratings of overall effectiveness, with the majority of teachers in the sample being rated at or above effective. Overall, findings offer support for using the CSAS-O and CSAS-T for guiding professional development conversations.

Recognizing Post-Castration Pain in Piglets: A Survey of Swine Industry Stakeholders and the General Public

We need validated, practical methods for pain assessment in piglets. Pain assessment can help researchers, veterinarians and industry professionals identify the need for analgesia or other pain management approaches when applying painful procedures, such as surgical castration. A pain assessment tool, the Piglet Grimace Scale (PGS), was previously validated in this context, but it is not widely applied. It is important that the PGS can be applied by a range of people, not just pain assessment experts. Our objective was to study the validity and reliability of PGS ratings applied by swine industry professionals and the general public, to assess its potential utility in non-research applications. To do so, we conducted an online Qualtrics survey in which, after completing a brief online training module and a practice test, 119 respondents were asked to rate 9 piglet images showing facial expressions immediately after surgical castration or sham-handling. Respondents were provided information on the castration treatment for each image and had continuous access to the scale throughout the survey. The survey also contained demographic questions. Industry respondents were recruited through networking, and participants from the general public were recruited through Amazon Mechanical Turk. Four trained experts scored each image, and these scores were averaged to produce gold standard scores. Intraclass correlations indicated strong internal consistency among experts, industry and public. ANOVA demonstrated scoring to be moderately comparable between groups. Campbell and Fiske’s Multi-Trait Multi-Method framework provided qualified support for the internal validity and reliability of the PGS scale, even applied by non-experts (industry and public). Both response groups were able to recognize pain in castrated piglets. However, public respondents attributed higher levels of pain to sham-handled piglets than industry respondents (2.83 vs. 2.35; p=0.047), and both response groups systematically overestimated pain experience compared to the experts, suggesting more training may be necessary before using the scale in a diagnostic capacity. Nevertheless, overall findings support wide applicability of PGS, even with minimal training, to improve awareness, recognition and monitoring of swine pain among veterinarians, industry professionals and even members of the public.

Multi-trait, multi-method talent assessment through a digital platform: Relationship with job performance

This study investigated whether a multi-method assessment approach increases variance for predicting future job performance compared to a single-method approach measuring similar constructs. Participants were managers (n = 539, female = 29%, average age 38 years, SD = 7 years) across a variety of organisations, including the utilities (n = 145), mining (n= 83), telecommunications (n = 98), Information Technology and professional service sectors (n = 213), drawn from samples across Namibia (n = 54), Nigeria (n = 53), Saudi Arabia (n = 105), South Africa (n = 177), the United Arab Emirates (n = 88), and the United Kingdom (n = 62). The managers used a digital platform to complete multi-trait multi-method (MTMM) measures which included competency-based video interviewing, work-related self-report measures, work simulations and aptitude assessments, each measuring work-related competencies. Structural equation modelling was used to determine paths between measures and work performance. Acceptable model fit for the measurement model was established through confirmatory factor analysis. A constrained and baseline model was used to investigate predictive paths to job performance. The baseline model proved to explain greater variance between measures and job performance than the constrained model. This provides support for deploying a multi-method assessment approach, including both self-rated and expert rated content, in a modern organisational context; and using an integrated digital platform.

Convergent validity and sex invariant factor structure of the Movement Imagery Questionnaire-3 - Second version (MIQ-3S): Healthy, young adult reference data

Aligned with the approach that established the factor structure of the Movement Imagery Questionnaire-3 (MIQ-3), this study extended the two-factor structure of the Movement Imagery Questionnaire – Revised Second version (MIQ-RS). The extension involves assessment of both internal and external visual imagery abilities along with kinesthetic imagery ability. Participants (N = 396) completed the new Movement Imagery Questionnaire – 3 Second Version (MIQ-3S) along with the Vividness of Movement Imagery Questionnaire-2 (VMIQ-2) which measure the same three imagery abilities. Alpha coefficients and between scale Spearman correlations for internal, external, and kinesthetic abilities indicated items were internally consistent (α > 0.87) and established convergent validity (r > 0.69), respectively. MIQ-3S scale means ranged from 5.56 (SD = 1.10) to 5.98 (SD = 0.84), with no differences by sex. The three scales were not multicolinear as intra-scale correlations ranged from 0.47 to 0.61, supporting the three abilities were related, but separate constructs. A multi-trait multimethod confirmatory factor analysis (MTMM CFA), with sex invariance, was conducted to confirm the 3-factor structure of the MIQ-3S. Results from 396 healthy male (n = 200) and female (n = 196) adult college-aged students (M = 21.91, SD = 2.37) indicated a correlated-traits correlated-uniqueness model provided the best fit to the data (CFI = 0.99; SRMR = 0.05; RMSEA = 0.03), while displaying sex invariance. These findings provide baseline data on college-aged, healthy adult participants providing reference data to those investigating imagery abilities among injured populations and practitioners interested in tracking individuals in rehabilitation.

Parental stress among Norwegian fathers and mothers: Similarities and differences

Although fathers are increasingly involved in childrearing, less is known about how parental stress is perceived among mothers and fathers separately. The present study investigated the construct validity of the PSS in a community-based sample of Norwegian father–mother dyads with a child aged 4. A total of 1030 mothers and fathers ( N = 515 dyads) responded to the PSS, which is a much used instrument that assesses stress related to childrearing across 18 items. Construct validity was investigated using exploratory and confirmatory factor analyses and the multitrait–multimethod approach (MTMM). The results revealed that six of the items showed extensive ceiling-effects for both parents, and were excluded from further analysis. Dimensionality was examined for the remaining 12 PSS items. The results supported a two-factor solution for both fathers and mothers, reflecting parental stressors (9 items) and lack of rewards (3 items). The MTMM approach indicated convergent and discriminant validity across traits (Parental stressors and Lack of rewards) and methods (father vs mother rating). We also explored the network of PSS items for mothers and fathers, and found both similarities and differences. Overall, the results of the factor analyses have implications for the use of the PSS. The networks of PSS suggest that some items may be especially important when addressing parental stress with mothers and fathers.

Developing robust indicators of age-friendly neighborhood environments across urban-rural and socioeconomic gradients in 20 countries from five continents

BACKGROUND AND AIM: By 2050, the older-adult population (60+) will reach 2.1 billion, surging fastest in low- and middle-income countries (LMIC). In response, the World Health Organization (WHO) has developed a framework and indicators of age-friendly urban environments, but these criteria have been challenging to apply in rural areas and LMIC. This study fills this gap by adapting these indicators to such settings and assessing variation by country-level income and community-level urban-rural status. METHODS: First, a narrative review identified and described existing healthy-aging audit tools. Next, data were drawn from the Prospective Urban and Rural Epidemiology (PURE) study’s environmental-exposure assessment tools, which relied on systematic social observation and ecometrics to develop objective and subjective indicators relevant to cardiovascular disease. Following a multitrait, multimethod (MTMM) approach to assess convergent and discriminant validity, we specified a set of 23 indicators aligned with WHO’s framework and calculated their values across 496 communities in 20 countries, including 382 communities (77%) located in LMIC. RESULTS:Among the WHO’s eight environmental domains, six were represented in PURE’s environmental data. Overall, 91% of communities had access to parks/recreation and street lighting, while 45% had traffic lights. In addition, 85% of communities offered a public medical clinic, but only 15% a public hospital. Communities in high-income countries rated highest overall, although bus-transit connections were most common in low-income countries (95%). The greatest amount of variation by urbanness was seen in the number of streetscape-greenery elements (55.2 in rural areas vs. 32.5 in urban); traffic lights (17.8% vs. 67.1%); access to universities (32.9% vs. 90.3%); and home-internet availability (24.7% vs. 53.8%). CONCLUSIONS:This study indicates the extent to which environmental supports for healthy aging may be less readily available to older adults residing in rural areas and LMIC, highlighting a critical aspect of equity that is increasingly important to address in an aging world. KEYWORDS: Green space, Walkability, Healthy aging, Methodological study design, International collaboration

Multitrait Multimethod validation of Implicit Association Test as a measure of individual differences in personality: Is there personality at all?

The objective of the study was to investigate the validity of IAT in the assessment of the Big five personality traits. We employed the Multi-Trait Multi-Method (MTMM) approach on a sample of 146 respondents. We collected measures on IATs assessing Big five personality traits, self-reports, and two ratings of close others of different gender on NEO PI-R. Additionally, we assessed behavior related to the Big five personality traits in a structured interview where three experts independently provided rating measures. MTMM analyses suggest that personality IAT measures have little in common with personality traits measured traditionally using explicit methods. The evidence does not support the claim that personality IAT can serve as a valid measure of individual differences in personality traits.

The Fear of Being Laughed at, Social Anxiety, and Paranoid Ideation: A Multilevel Confirmatory Factor Analysis of Multitrait-Multimethod Data.

The GELOPH-15 is a self-report measure that assesses individual differences in the fear of being laughed at (i.e., gelotophobia), a relatively understudied but important trait that is closely related to social anxiety. Using a multitrait-multimethod (MTMM) approach, the convergent and discriminant validity of the GELOPH-15 scale was examined based on 217 self- and 651 peer ratings (of three close acquaintances per target) of the traits gelotophobia, social anxiety, and paranoid ideation. Participants completed the Spanish versions of the GELOPH-15, the Social Interaction Anxiety Scale, and the Paranoia Scale. Applying MTMM models of multilevel confirmatory factor analyses (ML-CFA-MTMM) revealed relatively high associations between the self- and peer ratings, supporting the convergent validity of the GELOPH-15. Discriminant validity analyses confirmed the expected relationship patterns of gelotophobia with social anxiety and paranoid ideation (i.e., strong, but not perfect associations). The results showed that the ML-CFA-MTMM models might be a useful tool for analyzing the convergent and discriminant validity based on self- and peer ratings.

Is there potential in assessing for high-potential? Evaluating the relationships between performance ratings, leadership assessment data, designated high-potential status and promotion outcomes in a global organization

While leadership is a critical component of organizational success, organizations need to be more effective at classifying talent based on future potential vs current performance. This need raises the question as to whether the assessment of leadership potential is truly adding value or enhancing the talent review and classification process. Data from 9,784 participants in PepsiCo’s Leadership Assessment and Development (LeAD) program—a multi-trait, multi-method (MTMM) assessment and development process—are used to examine relationships between individual performance, assessed potential, organizationally designated potential, and promotion rates post assessment. Results from the analyses indicate that assessed potential provides unique variance above performance alone in determining process outcomes (i.e., designated potential and promotions), and therefore can be useful for making more informed talent management decisions. The applications of these measures within the current organization as well as limitations of the study and areas for future research are discussed.

Cognitive screening with functional assessment improves diagnostic accuracy and attenuates bias.

IntroductionCognitive screening measures often lack sensitivity and are hampered by inequities across ethnoracial groups. A multitrait multimethod (MTMM) classification may attenuate these shortcomings.MethodsA sample of 7227 participants across the diagnostic spectrum were selected from the National Alzheimer's Coordinating Center cohort. Random forest ensemble methods were used to predict diagnosis across the sample and within Black American (n = 1025) and non‐Hispanic White groups (n = 5263) based on: (1) a demographically corrected Montreal Cognitive Assessment (MoCA), (2) MoCA and Functional Assessment Questionnaire (FAQ), (3) MoCA and FAQ with demographic correction.ResultsThe MTMM approach with demographic correction had the highest diagnostic accuracy for the cognitively unimpaired (area under curve [AUC] [95% confidence interval (CI)]): 0.906 [0.892, 0.920]) and mild cognitive impairment (AUC: 0.835 [0.810, 0.860]) groups and reduced racial disparities.DiscussionWith further validation, the MTMM approach combining cognitive screening and functional status assessment may serve to improve diagnostic accuracy and extend opportunities for early intervention with greater equity.

Hippocampal and Parahippocampal Gray Matter Structural Integrity Assessed by Multimodal Imaging Is Associated with Episodic Memory in Old Age

Maintained structural integrity of hippocampal and cortical gray matter may explain why some older adults show rather preserved episodic memory. However, viable measurement models for estimating individual differences in gray matter structural integrity are lacking; instead, findings rely on fallible single indicators of integrity. Here, we introduce multitrait–multimethod methodology to capture individual differences in gray matter integrity, based on multimodal structural imaging in a large sample of 1522 healthy adults aged 60–88 years from the Berlin Aging Study II, including 333 participants who underwent magnetic resonance imaging. Structural integrity factors expressed the common variance of voxel-based morphometry, mean diffusivity, and magnetization transfer ratio for each of four regions of interest: hippocampus, parahippocampal gyrus, prefrontal cortex, and precuneus. Except for precuneus, the integrity factors correlated with episodic memory. Associations with hippocampal and parahippocampal integrity persisted after controlling for age, sex, and education. Our results support the proposition that episodic memory ability in old age benefits from maintained structural integrity of hippocampus and parahippocampal gyrus. Exploratory follow-up analyses on sex differences showed that this effect is restricted to men. Multimodal factors of structural brain integrity might help to improve our biological understanding of human memory aging.

Examining building blocks of well-being beyond PERMA and self-report bias

ABSTRACT Recent debates in the Journal of Positive Psychology about the nature and usefulness of PERMA have created confusion about its contribution toward the understanding and prediction of well-being. This empirical study was designed to clarify several issues that have emerged in these recent articles. Using a multi-trait multi-method (MTMM) research design with 220 knowledable co-worker pairs (N = 440), it was found that the 5 PERMA building blocks of well-being (positive emotions, engagement, relationships, meaning, and accomplishment) and 4 additional potential building blocks of well-being (physical health, mindset, environment, and economic security) significantly predicted SWB above and beyond self-report and mono-method bias. This is one of the first empirical studies to test the PERMA building blocks of well-being beyond the sole use of self-reports, and illustrates that the building blocks can be strong predictors of well-being in some populations.