Implications For Scale Development Research Articles

Developing and validating an L2 writing willingness to communicate scale: A sequential embedded mixed-methods approach

Current willingness to communicate (WTC) scales center on WTC in general second language (L2) learning, while L2 writing WTC is underrepresented. This study intended to close this gap by developing and validating an L2 writing WTC scale. A three-phase sequential embedded mixed-methods design was adopted to overcome the over-reliance on quantitative data and provide adequate evidence of validity. Nineteen items were generated based on our literature search and thematic analysis of the interview data ( n = 10). With quantitative data collected from 288 learners of English as a foreign language (EFL), the psychometric properties of the initial scale were examined by exploratory factor analysis. After that, the revised 17-item questionnaire was validated by confirmatory factor analysis and other validation methods with data from 224 EFL learners. The results indicated that the underlying structure involved writing task traits, English language ideology, writing teacher support, interest in English language, and self-perception of English language proficiency. The scale was further validated through factor analysis of the quantitative data ( n = 173) and thematic analysis of the immediate retrospective interview data ( n = 12) from EFL learners to test its generalizability in other L2 learning contexts and for face validity evidence. The findings showcased a promising mixed-methods design for scale development and clarified the underlying factors of L2 writing WTC. Implications for scale development and the teaching and learning of L2 writing were discussed.

Does the Five Facet Mindfulness Questionnaire measure the same constructs before and after ACT-based treatment? Examination of the longitudinal measurement invariance of the 24-item FFMQ-SF

Violations of longitudinal measurement invariance (LMI) suggest “response shifts” in individuals’ understanding and appraisal of items over time and result in misinterpretation of observed changes in outcome measures. These concerns are crucial to the validity of outcome measures and are particularly relevant in examining the effectiveness of mindfulness and other process-based interventions. In the current report from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we examine the LMI of the 24-item short form of the Five Factor Mindfulness Questionnaire from intake to discharge in a large, diagnostically heterogeneous sample of psychiatric patients (n = 2928) participating in an acceptance and commitment therapy-based partial hospitalization program. The original five-factor structure required correlated error terms for items with similar wording (Nonjudging of Experience and Describe scales) to achieve acceptable fit. Following these modifications, we found evidence of configural and weak invariance for all scales. We found evidence for strong invariance for Nonjudging of Experience, Nonreactivity to Inner Experiences, Describe, and Observing, but not for Acting with Awareness. Strict invariance was not supported. Results highlight the challenges of measuring constructs involving overt metacognitive processes via self-report. Implications for scale development and clinical practice are discussed.

A multidimensional zero-inflated graded response model for ordinal symptom data.

Zero responses and their equivalents-for example, never, none, not at all-are commonly observed on measures of psychopathology inquiring about symptom frequencies, particularly when these measures are administered to community samples. Measurement researchers typically accommodate multivariate zero inflation by including a nonpathological class of respondents who endorse zero for all symptoms. While this latent class approach accounts for test-level zero inflation (i.e., a proportion of individuals who do not experience any of the symptoms), it may be overly restrictive on questionnaires comprising items of differing severity. For example, an item about suicidal ideation is likely to exhibit a much higher degree of zero inflation than an item about low energy. Existing models do not account for this variability. We propose a multidimensional zero-inflated graded response model (MZI-GRM) as a more flexible approach for modeling zero inflation on questionnaires. According to the model, two distinct but correlated latent variables underlie ordinal item responses; one represents susceptibility to the construct, whereas the other represents severity. As a motivating example, we show how the MZI-GRM can be fit to data from the PHQ-9, a common depression screener. Results suggest that the MZI-GRM is better able to capture zero inflation across items than existing alternative models. Further, we find support for a multidimensional model that allows distinct but correlated latent variables to underlie each response process. Some items better measure susceptibility to depression (symptom presence), whereas others better capture severity of depression (symptom frequency). Implications for scale development and scoring are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Toward a multidimensional construct of racialized sexual discrimination (RSD): Implications for scale development.

Toward a multidimensional construct of racialized sexual discrimination (RSD): Implications for scale development.

Using Rasch Rating Scale Methodology to Examine a Behavioral Screener for Preschoolers At Risk

A screening instrument used to identify young children at risk for behavioral and emotional difficulties, the Behavioral and Emotional Screening System Teacher Rating Scale–Preschool was examined. The Rasch Rating Scale Method was used to provide additional information about psychometric properties of items, respondents, and the response scale. Using a representative U.S. sample of 1,485 children, the structure of the screener, performance of the rating scale, and coverage of the Maladaptive Behavior construct were investigated to identify children with emerging behavioral and emotional difficulties. Items measuring Internalizing and Externalizing Problems indicated greater efficiency for assessing at-risk behavioral and emotional problems, whereas items measuring emerging school problems or adaptive skills contributed little to the assessment of risk. Differential item functioning was examined for by age and found to be present for roughly one third of the items. Implications for scale development, screening young children, and use of screening information are discussed.

A More Nuanced View of Reliability

Retest reliability is a better predictor of validity than is internal consistency. One explanation for this is item-specific variance, which distinguishes different nuances of a facet and contributes to retest reliability but not internal consistency. Specific variance at the facet level is temporally stable, consensually validated, and heritable; a consideration of the role of specific variance in personality measures leads to a distinction between traits as the intersection (∩) versus the union (∪) of their constituents. I discuss specific variance at the item level and its implications for scale development and argue that retest reliability outpredicts internal consistency because item-specific variance has been shown to be observable, and is probably heritable and stable. I consider some implications of these ideas for the use of single-item scales, the causal interpretation of traits, and the notion of scalar equivalence. Finally, I note that the sources of random error in scales are still poorly understood.

Analysis of patient-reported anterior knee pain scale: implications for scale development in children and adolescents.

The purpose of this investigation was to estimate and document the reliability and validity of the Anterior Knee Pain Scale (AKPS) and to estimate its relative prediction accuracy of anterior knee pain in young females. Data from a prospective, epidemiologic study to diagnose patellofemoral knee pain among female athletes (n = 499) using the Anterior Knee Pain Scale (AKPS). Data were treated in 4 phases (descriptive phase, reliability phase, scale refinement phase) and a final validation stage that was focused on an effort to test and document the validation of the AKPS short form and perform head-to-head comparisons of the 6-item short form with the original, 13-item form. The AKPS was reduced from 13 items (αCoeff = 0.77, σSEM = 0.004) to 6 items (αCoeff = 0.78, σSEM = 0.004). Point-biserial correlations with patellofemoral pain diagnosis were comparable: r [498] = 0.70 (R(2) = 0.49, short form) and r [498] = 0.71 (R(2) = 0.51, long form), as was sensitivity: 84% (short form) and 80% (long form), and specificity: 89% (short form) and 90% (long form; AUC = 0.94 both). The current analyses indicate that a subset of measures from the AKPS is responsive to patellofemoral pain symptoms and may support screening for related diagnoses. A simpler and quicker scale optimized for diagnostic accuracy could reduce the demand on patients, clinicians and research teams focused on the identification and management of patellofemoral pain. III.

Developing Scales to Measure Consumer Dispositions: A New Approach Based on Predictive Validity

In the decades since publication of Churchill (1979), scale development in marketing has migrated in the direction of shorter scales and much smaller item pools. However, neither a justification for this trend, nor its implications for scale development, has been laid out. This paper focuses on the implications for measurement of consumer dispositions, with skepticism toward advertising serving as the running example. It identifies problematic aspects of the now obsolescent call for large item pools of 100, and clarifies the potential negative effects of pursuing internal consistency or clean factor structure in isolation from considerations of predictive validity. To address these concerns, an alternative procedure for generating and pruning item pools is offered. The core of the new approach is to apply well-accepted tools for scale validation at the level of individual items, thus moving the consideration of predictive validity forward to the item pruning stage.

Considerations for Measuring Functioning of the Elderly: IRM Dimensionality and Scaling Analysis

Modern measurement methods were applied in this study to examine the properties of a measure of functioning of the elderly. Measures of functioning form an essential element in health services and outcomes research. Several implications for scale development and improved score precision are presented in this case study. This study examined the structure of responses to the Older Americans Resources and Services (OARS) Activities of Daily Living (ADL) and Instrumental Activities of Daily Living (IADL) scales using item response model (IRM) analysis methods. The analysis illustrates the extension of IRM dimensionality and item analysis to health scales in general. Attention is given to the underlying theory and appropriate interpretation of these methods for health measurement. Data were taken from 1364 elderly Canadians participating in the caregiver component of the Canadian Study of Health and Aging (CSHA). The fit of a two-parameter logistic IRM was compared with a one-parameter (Rasch) model for these data. The dimensionality of responses to the scale was evaluated with an approximate χ2 test of residuals after fitting an IRM based on non-linear factor analysis. Results confirm that ADL and IADL item sets differ in the degree of disability they measure and are well represented as separate dimensions using a two-parameter IRM. Implications are drawn concerning the adequacy of the OARS disability measure for health surveys, while more general conclusions cover the precision of IRM based optimal scoring for functional disability measures.

Voluntary class absences and other behaviors in college students: an exploratory analysis.

In a survey of 120 sophomore and junior undergraduate college students, we found that those who admitted to cutting class for no reason during the previous month had a lower mean GPA than those who reported no voluntary absences. Cutting class was related to reports of getting drunk, speeding, breaking the law, and visiting home, although reports of speeding, getting drunk, and breaking the law were not related to GPA. The 11 students who reported feeling depressed had a lower GPA than those without depression, while the 18 students who had not visited their families in the previous month had higher GPAs than those who had visited their families. Implications for scale development are discussed.

UK Consumer Decision-Making Styles

The importance of market globalisation and the consequent interest in international marketing has prompted a surge of work examining cross-cultural consumer behaviour. As part of this trend, this paper examines the generalisability of Sproles and Kendall's (1986) Consumer Styles Inventory (CSI) in an extension of their work in the UK. Most of the original US traits were found in the UK, with the addition of new store-loyalty and time-energy saving traits. However, some problems were encountered with the CSI's reliability and validity. Extending the original study, the research also identified meaningful and distinct decision-making groups which suggests a possible use of the CSI in segmentation. Implications for scale development, segmentation and cross-cultural research are discussed.

Pain profiles of patients with nonorganic chest pain: A preliminary report of the multidimensional pain inventory

The primary purpose of this report is to extend the range of the Multidimensional Pain Inventory (MPI) to include patients with nonorganic chest pain. Previous research with the MPI has not included this patient population, although this instrument has been used to derive an empirically based taxonomy of patient responses to chronic pain. Scale scores are provided for a sample of 43 chest pain patients and compared with normative scores from samples of chronic lower back pain patients and patients suffering from temporomandibular disorder. The MPI taxonomy was applicable for only 34.8% ( N = 15) of this sample. Scale intercorrelations are examined and compared with those derived during development of the MPI, to explore reasons for this low classification rate. The results are discussed in light of cognitive-behavioral factors present in persistent chest pain, with implications for scale development and use of the MPI.