Evidence Of Measurement Equivalence Research Articles

Validating Dewey’s Notion of Genuine Interest

This investigation (1) elucidates genuine interest in the context of learning from Dewey’s perspective, (2) assesses construct validity evidence for a genuine interest conceptual model using data from the Factors Effecting Ethics Learning survey, and (3) assesses measurement invariance of the genuine interest constructs between male ( n = 352) and female ( n = 188) Taiwan high school students. Results provide support for genuine interest as a two-dimensional construct. Second-order factor analysis shows that latent variables of genuine interest in learning socio-scientific issues and interpersonal communication ethics-related issues are represented by sub-latent constructs of learning interest and learning enjoyment. Assessment of gender invariance in the conceptual model provides evidence of measurement equivalence.

Well-Being of College Students in China

In this study, we tested a modified academic satisfaction model based on social cognitive career theory in a sample of 757 college students in China. The hypothesized model included personality traits (extraversion and emotional stability), self-construals (independence and interdependence), environmental and person-cognitive variables (supports, self-efficacy, outcome expectations, and perceived goal progress) in the academic domain, and academic and global well-being outcomes. Pathways that consisted of academic supports, self-efficacy, and goal progress partially mediated the effects of personality traits and self-construals on academic satisfaction and/or stress, which were, in turn, predictive of life satisfaction. Although Chinese students perceived outcomes of completing a college degree as favorable, such outcome expectations did not predict progress made in the academic domain. Multigroup analyses showed that the modified academic satisfaction model was applicable to both male and female college students in two major metropolitan areas—Chongqing in Southwest China and Shanghai in East China. With evidence for measurement equivalence, full structural equivalence was present by gender, whereas two of the hypothesized paths differed by location. Results of the study suggest that outreach or intervention programs, which involve gathering supports, boosting self-efficacy, and facilitating goal progress in the academic domain, are particularly beneficial for promoting the well-being of Chinese college students.

External validity of a generic safety climate scale for lone workers across different industries and companies

PurposeThe goal of this study was to examine the external validity of a 12-item generic safety climate scale for lone workers in order to evaluate the appropriateness of generalized use of the scale in the measurement of safety climate across various lone work settings. External validity evidence was established by investigating the measurement equivalence (ME) across different industries and companies. MethodConfirmatory factor analysis (CFA)-based and item response theory (IRT)-based perspectives were adopted to examine the ME of the generic safety climate scale for lone workers across 11 companies from the trucking, electrical utility, and cable television industries. ResultsFairly strong evidence of ME was observed for both organization- and group-level generic safety climate sub-scales. Although significant invariance was observed in the item intercepts across the different lone work settings, absolute model fit indices remained satisfactory in the most robust step of CFA-based ME testing. IRT-based ME testing identified only one differentially functioning item from the organization-level generic safety climate sub-scale, but its impact was minimal and strong ME was supported. ImplicationsThe generic safety climate scale for lone workers reported good external validity and supported the presence of a common feature of safety climate among lone workers. The scale can be used as an effective safety evaluation tool in various lone work situations.

The Multidimensionality of Prosocial Behaviors and Evidence of Measurement Equivalence in Mexican American and European American Early Adolescents

There is growing recognition of the need to examine distinct forms of prosocial behaviors and to conduct research on prosocial behaviors among ethnic minorities. Middle school students (mean age=12.67 years; 54% girls; European American, n=290; Mexican American, n=152) completed a multidimensional measure of prosocial behavior and measures of parental monitoring, externalizing behaviors, and religiosity. Results yield supportive evidence that anonymous, dire, emotional, compliant, public, and altruism are different forms of prosocial behaviors. Moreover, the measure of prosocial behaviors shows measurement equivalence properties across ethnicity and gender. The discussion focuses on the multidimensional nature of prosocial behaviors and implications for future research on prosocial behaviors.

Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report

BackgroundPatient-reported outcomes (PROs) are the consequences of disease and/or its treatment as reported by the patient. The importance of PRO measures in clinical trials for new drugs, biological agents, and devices was underscored by the release of the US Food and Drug Administration's draft guidance for industry titled “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims.” The intent of the guidance was to describe how the FDA will evaluate the appropriateness and adequacy of PRO measures used as effectiveness end points in clinical trials. In response to the expressed need of ISPOR members for further clarification of several aspects of the draft guidance, ISPOR's Health Science Policy Council created three task forces, one of which was charged with addressing the implications of the draft guidance for the collection of PRO data using electronic data capture modes of administration (ePRO). The objective of this report is to present recommendations from ISPOR's ePRO Good Research Practices Task Force regarding the evidence necessary to support the comparability, or measurement equivalence, of ePROs to the paper-based PRO measures from which they were adapted. MethodsThe task force was composed of the leadership team of ISPOR's ePRO Working Group and members of another group (i.e., ePRO Consensus Development Working Group) that had already begun to develop recommendations regarding ePRO good research practices. The resulting task force membership reflected a broad array of backgrounds, perspectives, and expertise that enriched the development of this report. The prior work became the starting point for the Task Force report. A subset of the task force members became the writing team that prepared subsequent iterations of the report that were distributed to the full task force for review and feedback. In addition, review beyond the task force was sought and obtained. Along with a presentation and discussion period at an ISPOR meeting, a draft version of the full report was distributed to roughly 220 members of a reviewer group. The reviewer group comprised individuals who had responded to an emailed invitation to the full membership of ISPOR. This Task Force report reflects the extensive internal and external input received during the 16-month good research practices development process. Results/RecommendationsAn ePRO questionnaire that has been adapted from a paper-based questionnaire ought to produce data that are equivalent or superior (e.g., higher reliability) to the data produced from the original paper version. Measurement equivalence is a function of the comparability of the psychometric properties of the data obtained via the original and adapted administration mode. This comparability is driven by the amount of modification to the content and format of the original paper PRO questionnaire required during the migration process. The magnitude of a particular modification is defined with reference to its potential effect on the content, meaning, or interpretation of the measure's items and/or scales. Based on the magnitude of the modification, evidence for measurement equivalence can be generated through combinations of the following: cognitive debriefing/testing, usability testing, equivalence testing, or, if substantial modifications have been made, full psychometric testing. As long as only minor modifications were made to the measure during the migration process, a substantial body of existing evidence suggests that the psychometric properties of the original measure will still hold for the ePRO version. Hence, an evaluation limited to cognitive debriefing and usability testing only may be sufficient. However, where more substantive changes in the migration process has occurred, confirming that the adaptation to the ePRO format did not introduce significant response bias and that the two modes of administration produce essentially equivalent results is necessary. Recommendations regarding the study designs and statistical approaches for assessing measurement equivalence are provided. ConclusionsThe electronic administration of PRO measures offers many advantages over paper administration. We provide a general framework for decisions regarding the level of evidence needed to support modifications that are made to PRO measures when they are migrated from paper to ePRO devices. The key issues include: 1) the determination of the extent of modification required to administer the PRO on the ePRO device and 2) the selection and implementation of an effective strategy for testing the measurement equivalence of the two modes of administration. We hope that these good research practice recommendations provide a path forward for researchers interested in migrating PRO measures to electronic data collection platforms.

Measurement Equivalence of the Child Behavior Checklist among Parents of African American Adolescents

The Child Behavior Checklist (CBCL/6-18) is a widely used parent-report measure of youth internalizing and externalizing problems. Accumulating evidence, however, suggests that the CBCL may not adequately assess clinical problems in diverse populations. We investigated the measurement equivalence (ME) of the CBCL in an African American adolescent sample (N = 145). Confirmatory factor analysis indicated poor factor model fit; however, modification indices and an alternative two-factor model produced improved fit statistics. Internal consistency coefficients of CBCL scales ranged from α = .65 to .88 and were generally lower than those reported for the normative sample. The CBCL exhibited convergent validity with conceptually similar scales on the Parent Report Scale of the Behavior Assessment System for Children; however, the correlations were weaker than those reported for the normative sample. The CBCL Internalizing composite failed to predict clinician-rated Emotional problems, however, the Externalizing composite predicted clinician-rated behavior problems. In summary, the CBCL did not exhibit unequivocal evidence of ME, as it both possesses and lacks a number of statistical attributes that would demonstrate its equivalence with African Americans. Future empirical and theoretical work is discussed.

The Beck Depression Inventory-II: Testing for Measurement Equivalence and Factor Mean Differences Across Hong Kong and American Adolescents

Working within the framework of a confirmatory factor analytic (CFA) model, this study adds another dimension to construct validation of both the Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996) and a Chinese version of the BDI-II (C-BDI-II; Chinese Behavioral Sciences Society, 2000). Specifically, we tested for measurement equivalence of the C-BDI-II with the original BDI-II across Hong Kong (N = 1771) and American (N = 501) adolescents, respectively. Provided with evidence of measurement equivalence, we then tested for differences in the latent factor means (i.e., subscale levels) of three first-order factors of negative attitude, performance difficulty, and somatic elements and one second-order factor of general depression. All procedures were based on analyses of mean and covariance structures (MACS) that took into account both the incompleteness and non-normality of the data. Findings revealed sound evidence of measurement equivalence of the C-BDI-II and BDI-II factorial structures, and latent factor means that pointed to higher levels of depressive symptoms for Hong Kong adolescents than for their American counterparts.