Item Response Scales Research Articles

Patient interpretations of patient-reported outcome measures to assess bowel urgency: qualitative interviews in ulcerative colitis

ObjectivesBowel urgency is an impactful core symptom of ulcerative colitis (UC). Patient-reported outcome (PRO) questionnaires have been developed and used to assess the patient experience of this important symptom. The objective of this paper is to present evidence from qualitative research conducted to support the use and interpretation of select PRO questionnaires to assess bowel urgency related to the UC patient experience.MethodsQualitative interviews were conducted with ten adults with a clinician-confirmed diagnosis of moderately to severely active UC. Interviews aimed to document patient interpretation of modified recall periods for the Urgency Numeric Rating Scale (Urgency NRS), two global assessments (i.e., the Patient Global Impression of Severity [PGIS] and Patient Global Impression of Change [PGIC]), and four items (Items 11, 16, 23, and 26) of the Inflammatory Bowel Disease Questionnaire (IBDQ), and explore the patient perspective of meaningful change on these questionnaires.ResultsBoth modified Urgency NRS versions (with 7-day or 3-day recall period) were interpreted as intended by most patients (≥ 88.9%), and slightly more than half of patients (60.0%) reported that the 7-day recall period was more relevant to their bowel urgency experience. Patients reported thinking of bowel urgency (≥ 80.0%) or bowel urgency-related accidents (70.0% of patients) when interpreting the global assessments and IBDQ items. Most patients reported a 1- to 3-point change as the smallest meaningful improvement that would be meaningful on the Urgency NRS (similar to findings on other questionnaires).ConclusionAdults with UC can understand and respond to the Urgency NRS with modified recall periods (i.e., 7-day or 3-day), interpret the conceptual content of the PGIS, PGIC, and select IBDQ items to be inclusive of bowel urgency and bowel urgency-related accidents, and select answers representing meaningful improvements on the Urgency NRS, PGIS, PGIC, and IBDQ item response scales. These results further contribute patient-centered data to existing UC and bowel urgency research.

Cognitive and cultural models in psychological science: A tutorial on modeling free-list data as a dependent variable in Bayesian regression.

Assessing relationships between culture and cognition is central to psychological science. To this end, free-listing is a useful methodological instrument. To facilitate its wider use, we here present the free-list method along with some of its many applications and offer a tutorial on how to prepare and statistically model free-list data as a dependent variable in Bayesian regression using openly available data and code. We further demonstrate the real-world utility of the outlined workflow by modeling within-subject agreement between a free-list task and a corollary item response scale on religious beliefs with a cross-culturally diverse sample. Overall, we fail to find a reliable statistical association between these two instruments, an original empirical finding that calls for further inquiry into identifying the cognitive processes that item response scales and free-list tasks tap into. Throughout, we argue that free-listing is an unambiguous measure of cognitive and cultural information and that the free-list method therefore has broad potential across the social sciences aiming to measure and model individual-level and cross-cultural variation in mental representations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Measuring residual limb health in persons with upper limb amputation: Modifications of the Prosthetic Evaluation Questionnaire residual limb health scale.

Assessing the user perspective on residual limb health problems is particularly important for amputation care, given the relationship between residual limb health and prosthetic satisfaction. Only 1 measure, the Residual Limb Health scale of the Prosthetic Evaluation Questionnaire (PEQ) has been validated for use in lower limb amputation, but not examined in persons with upper limb amputation (ULA). The objective of this study was to examine the psychometric properties of a modified PEQ Residual Limb Health scale in a sample of persons with ULA. The study involved a telephone survey of 392 prosthesis users with ULA, with a 40-person retest sample. The PEQ item response scale was modified to a Likert scale. The item set and instructions were refined in cognitive and pilot testing. Descriptive analyses characterized the prevalence of residual limb issues. Factor analyses and Rasch analyses evaluated unidimensionality, monotonicity, item fit, differential item functioning, and reliability. Test-retest reliability was assessed by an intraclass correlation coefficient. Sweating and prosthesis odor were prevalent at 90.7% and 72.5%, respectively; blisters/sores (12.1%) and ingrown hairs (7.7%) were the least prevalent problems. Response categories were dichotomized for 3 items and trichotomized for 3 items to improve monotonicity. After adjusting for residual correlations, confirmatory factor analyses showed acceptable fit (comparative fit index = 0.984, Tucker-Lewis index = 0.970, and root mean square error approximation = 0.032). Person reliability was 0.65. No items had moderate-to-severe differential item functioning by age or sex. Intraclass correlation coefficient for test-retest reliability was 0.87 (95% CI, 0.76-0.93). The modified scale had excellent structural validity, fair person reliability, very good test-retest reliability, and no floor or ceiling effects. The scale is recommended for use with persons with wrist disarticulation, transradial amputation, elbow disarticulation, and above-elbow amputation.

Psychometric validation and interpretation of the Nocturia Impact Diary in a clinical trial setting

PurposePsychometric evaluation of the Nocturia Impact (NI) Diary was conducted to support its use as a trial endpoint.MethodsAs part of a randomized, controlled Phase 2 clinical trial investigating a novel drug candidate for nocturnal polyuria, adult nocturia patients completed the NI Diary and a voiding diary for three nights preceding their clinic visit at Baseline and Weeks 1, 4, 8, and 12 (end of treatment). Exit interviews were conducted to obtain patient impressions of the NI Diary.ResultsA total of N = 302 participants were included. Confirmatory factor analysis (CFA) indicated that the 11-item measure is unidimensional with values of CFI, TLI, and RMSEA meeting relevant thresholds. Good internal consistency (Cronbach’s α 0.941) and test–retest reliability (intra-class correlation coefficients 0.730–0.880). Convergent validity with two reference measures was demonstrated with strong correlations of 0.573–0.730 were shown. Significant differences (P = 0.0018, standardized effect size = 0.372) between groups defined by number of night-time voids supported known-groups validity. Exit interviews in 66 patients indicated all participants experienced improvement in at least 1 NI Diary item and that a 1-point improvement on the item response scale and 1-void reduction per night (associated with an average best cut point on ROC analysis of − 11.6) constituted meaningful improvement. Anchor and distribution-based analyses identified a meaningful change threshold of − 15 to − 18 points on the NI Diary.ConclusionThe NI Diary is a reliable and valid patient-reported psychometric instrument which is fit-for-purpose to evaluate the impact of nocturia on patient quality of life in the clinical trial setting.Trial registration number and registration date NCT03201419; June 28, 2017.

“If only…”: customer counterfactual thinking in failed recovery

Purpose This paper aims to examine counterfactual thinking as a key mediator of the effects of failed recovery (vs. failed delivery) on negative electronic word-of-mouth (eWOM). The authors further investigate the effectiveness of using recovery co-creation in minimizing customers’ counterfactual thinking. Design/methodology/approach This research includes textual analysis of online reviews (Study 1) and three scenario-based experiments (Studies 2, 3a and 3b). In addition to using item-response scales, the authors analyze negative online reviews and participants’ open-ended responses to capture their counterfactual thinking. Findings Failed recovery (vs failed delivery) increases counterfactual thinking, which, in turn, increases negative eWOM. These mediating effects of counterfactual thinking are consistent across textual analyses and experimental studies, as well as across different measures of counterfactual thinking. Counterfactual thinking also impacts customer anger in experiments; however, anger alone does not explain the effects of failed recovery on negative eWOM. Counterfactual thinking can be minimized by co-created recovery, especially when it is used proactively. Practical implications The findings demonstrate the detrimental effects of counterfactual thinking and offer managerial insights into co-creation as a strategy to minimize customers’ counterfactual thinking. The authors also highlight the importance and ways of tracking counterfactual thinking in digital outlets. Originality/value The authors contribute to counterfactual thinking and service recovery research by demonstrating the effects of failed recovery on counterfactual thinking that, in turn, impacts negative eWOM and offering a novel way to measure its expression in online narratives. The authors provide guidance on how to use co-creation in the service recovery process to minimize counterfactual thinking.

Examining the factor structure of ICD-11 posttraumatic stress disorder (PTSD) and complex-PTSD among prison staff exposed to potentially traumatic experiences

The recently released 11th edition of the International Classification of Diseases (ICD-11) classifies posttraumatic stress disorder (PTSD) and complex PTSD (CPTSD) as distinct, yet related, disorders within the spectrum of trauma and stress-related disorders. This study aimed to explore the construct validity of the International Trauma Questionnaire (ITQ), a measure of ICD-11 PTSD and CPTSD symptoms among prison governors (i.e., wardens in the U.S. and Canada). Trauma-exposed prison governors (N = 409) aged 26 to 82 years (M = 50.04, SD = 7.97) provided their data online anonymously. Confirmatory factor analyses were performed to evaluate the construct validity of ITQ scores. When using the five-point item response scale, our five first-order factor model produced the best fit to the data; however, when using a dichotomous item scale, the first-order two-factor PTSD and disturbances of self-organization (DSO) model produced the best fit to the data. These results indicate that item-coding and estimation methods can significantly impact conclusions about the best-fitting model for this measure among trauma-exposed prison governors.

Un viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna

<p>Based on recent psychometric developments, this paper presents a conceptual and practical guide for estimating internal consistency reliability of measures obtained as item sum or mean. The internal consistency reliability coefficient is presented as a by-product of the measurement model underlying the item responses. A three-step procedure is proposed for its estimation, including descriptive data analysis, test of relevant measurement models, and computation of internal consistency coefficient and its confidence interval. Provided formulas include: (a) Cronbach’s alpha and omega coefficients for unidimensional measures with quantitative item response scales, (b) coefficients ordinal omega, ordinal alpha and nonlinear reliability for unidimensional measures with dichotomic and ordinal items, (c) coefficients omega and omega hierarchical for essentially unidimensional scales presenting method effects. The procedure is generalized to weighted sum measures, multidimensional scales, complex designs with multilevel and/or missing data and to scale development. Four illustrative numerical examples are fully explained and the data and the R syntax are provided.</p>

Item-level informant discrepancies between children and their parents on the PROMIS(®) pediatric scales.

The study objective was to describe the individual item-level discrepancies between children ages 8-17 years and their parents for the PROMIS(®) pediatric scales. Contextual effects on item-level informant discrepancies for the pediatric pain interference items were further analyzed conditional on whether the child, the parent, or anyone else in the household experienced chronic pain. Parallel pediatric self-report and parent proxy-report items were completed by approximately 300 parent-child dyads depending on form assignment and individual nonresponse. Agreement between parent and child responses to individual items was measured using the polychoric correlation coefficient and weighted κ. The Chi-square test of symmetry was utilized for a comparison of the pattern of parent-child item discrepancies on the response scales, and the differences between the child and parent responses on the 1-5 item response scale are summarized . A continuum of higher item-level parent-child discrepancies was demonstrated starting with peer relationships, anger, anxiety, and depressive symptoms, followed by progressively lower parent-child discrepancies for energy, fatigue, asthma impact, pain interference, upper extremity, and mobility items. Parent-child discrepancies for pain interference items were lower in the context of chronic pain either in the child or in the parent. Parent-child item-level discrepancies were lower for more objective or visible items than for items measuring internal states or less observable items measuring latent variables such as peer relationships and fatigue. Future research should focus on the child and parent characteristics that influence domain-specific item-level discrepancies, and under what conditions item-level parent-child discrepancies predict child health outcomes.

Development of the measure of ovarian symptoms and treatment concerns: aiming for optimal measurement of patient-reported symptom benefit with chemotherapy for symptomatic ovarian cancer.

The aim of this study was to determine the optimal patient-reported outcome measure (PROM) for assessing symptom benefit in trials of palliative chemotherapy for women with symptomatic ovarian cancer. Candidate PROMs were EORTC QLQ-C30 plus ovarian-specific QLQ-OV28, Functional Assessment of Cancer Therapy-Ovarian (FACT-O), FACT Ovarian Symptom Index (FOSI), and gynecologic cancer-specific Symptom Representation Questionnaire. Predefined optimality criteria were inclusion of all symptoms necessary for the specified purpose, recall period covering typical length of palliative chemotherapy, numerical item rating scales, and all necessary symptoms included in a single symptom index. Qualitative and quantitative methods were applied to data from stage 1 of the Gynecologic Cancer Intergroup Symptom Benefit Study to determine the set of necessary symptoms and to objectively assess candidate PROMs against the optimality criteria. Ten necessary symptoms were identified: pain, fatigue, abdominal bloating/discomfort, sleep disturbance, bowel disturbance, nausea and vomiting, shortness of breath, poor appetite, urinary symptoms, and weight changes. Although QLQ-C30 and QLQ-OV28 together cover all these symptoms, they split them into numerous scales, dissipating potential symptom-benefit signal. Conversely, FACT-O does not cover all necessary symptoms and contains many other HRQoL-related items and treatment side effects, diluting potential symptom-benefit signal when summed into scales. Item response scales and composite scoring of all candidate PROMs were suboptimal to our specific purpose. We therefore developed a new PROM, the Measure of Ovarian Symptoms and Treatment (MOST) concerns, to provide optimal measurement for the specified purpose. This article documents the development of the MOST, a new PROM designed to assess patient-reported benefits and burden as end points in clinical trials of palliative chemotherapy for women with symptomatic ovarian cancer. The validity, reliability, and statistical efficiency of the MOST, relative to the best candidate scales of existing PROMs, will be assessed in the stage 2 of Gynecologic Cancer Intergroup Symptom Benefit Study.

Predicting risk of airflow obstruction in primary care: Validation of the lung function questionnaire (LFQ)

The Lung Function Questionnaire (LFQ) is being developed as a case finding tool to identify patients who are appropriate for spirometry testing to confirm the diagnosis of chronic obstructive pulmonary disease (COPD). The cross-sectional study reported herein was conducted to validate the LFQ, to identify item-response scales associated with the best accuracy, and to determine the impact on accuracy of the addition of another item on activity limitations (AL). Patients >or= 40 years old seen at 2 primary care offices completed the LFQ, a demographic questionnaire followed by spirometry. Of the 837 evaluable patients, 18.6% had airflow obstruction (forced expiratory volume in 1 s/forced vital capacity [FEV(1)/FVC] < 0.70). The 5 items (age, wheeze, dyspnea, smoking, and cough) previously identified in initial LFQ development predicted airflow obstruction and showed good evidence of screening accuracy. Screening accuracy was significantly better with 5-point ordinal item-response scales (78%) than binary (yes/no) item-response scales (74%)(p < 0.05). Screening accuracy was good regardless of whether airflow obstruction was defined as FEV(1)/FVC < 0.70 or FEV(1)/FVC < 0.70 and FEV(1) < 80% of predicted. Based on <or=18 was selected to suggest presence of airflow obstruction with area under the receiver operating characteristic curve 0.652; sensitivity 82.6%; specificity 47.8%; 54.3% correctly classified. While the specificity of LFQ is low, its high sensitivity suggests that it can serve to identify patients who should be further assessed using spirometry. Our results confirm the screening accuracy of the LFQ, a simple and effective tool to facilitate early recognition and diagnosis of COPD.

DSM criteria for major depression: evaluating symptom patterns using latent-trait item response models

Expert committees of clinicians have chosen diagnostic criteria for psychiatric disorders with little guidance from measurement theory or modern psychometric methods. The DSM-III-R criteria for major depression (MD) are examined to determine the degree to which latent trait item response models can extract additional useful information. The dimensionality and measurement properties of the 9 DSM-III-R criteria plus duration are evaluated using dichotomous factor analysis and the Rasch and 2 parameter logistic item response models. Quantitative liability scales are compared with a binary DSM-III-R diagnostic algorithm variable to determine the ramifications of using each approach. Factor and item response model results indicated the 10 MD criteria defined a reasonably coherent unidimensional scale of liability. However, person risk measurement was not optimal. Criteria thresholds were unevenly spaced leaving scale regions poorly measured. Criteria varied in discriminating levels of risk. Compared to a binary MD diagnosis, item response model (IRM) liability scales performed far better in (i) elucidating the relationship between MD symptoms and liability, (ii) predicting the personality trait of neuroticism and future depressive episodes and (iii) more precisely estimating heritability parameters. Criteria for MD largely defined a single dimension of disease liability although the quality of person risk measurement was less clear. The quantitative item response scales were statistically superior in predicting relevant outcomes and estimating twin model parameters. Item response models that treat symptoms as ordered indicators of risk rather than as counts towards a diagnostic threshold more fully exploit the information available in symptom endorsement data patterns.

Short form of the Dizziness Handicap Inventory: construction and validation through Rasch analysis.

A new item response scale is presented, which measures the severity of self-reported balance deficits. The scale, DHIsf, is a short form of the Dizziness Handicap Inventory. The scale was constructed and validated by Rasch analysis. Rasch analysis was applied to rescore or remove any items misfitting, redundant, or off-target, until an optimal instrument was obtained. The 25-item, 3-level Dizziness Handicap Inventory was, thus, reduced to the 13-item, 2-level DHIsf. The retained items explore the domains of eye/head movements, full body activities, and mood alterations. Data were collected from 55 outpatients (63 +/- 13 yr; 43 females) attending otoneurological rehabilitation referral at a general hospital because of complaints of dizziness or imbalance. They were fully independent in ambulation and showed no evidence of major neurological or orthopedic diseases. Objective tests included brain computed tomography, sovraaorctic Doppler sonography, craniocorpography, static posturography, and nystagmography. The findings were categorized as pathologic, borderline, or normal. At least one examination was borderline or abnormal in 42 patients. The DHIsf was well targeted on this sample, with a mean score of 5.7/13 (standard deviation, 2.8; median, 5; range, 1-13). The Rasch statistics showed that the 13 items evenly fitted a hierarchy of difficulty within a homogeneous construct. A moderate but significant variance explanation of DHIsf measures was provided by a two-way analysis of variance model, with craniocorpography and nystagmography as independent categorical variables (r2 = 0.15; P = 0.018). When the clinical tests were individually taken into account, their outcome (dichotomized as abnormal v borderline or normal) could not be predicted by either of the DHIsf measures or raw scores (logistic regression). The DHIsf compares favorably with the original Dizziness Handicap Inventory, shows some consistency with the instrumental findings, and provides original information on the severity of imbalance syndromes, as it is seen from the patient's perspective.