Response Shift Effects Research Articles

Response-shift effect for self-assessment: Magnitude and accuracy within an interprofessional education course

ObjectiveResponse-shift has described a shift in respondents' self-assessment following an impactful experience—where a respondent may initially score themselves higher than their true ability on Pre-assessment, though shifted/lowered their initial ability-score (on Post-assessment) after they learned further. While the psychological literature had described this ‘response-shift’ effect decades ago, our objective at this time was to evaluate the magnitude that response-shift was occurring in our interprofessional education (IPE) context and using our own self-assessment instrument. MethodsUsing a standard response-shift study design, first-year health-professions students within a foundational IPE course completed beginning-of-course (Pre) and end-of-course (Post) self-assessments based on that IPE course's student-learning-objectives; additionally at course-end, students completed a retrospective-beginning-of-course (RetroPre) self-assessment (to reflect and score themselves retrospectively for course's beginning). Paired t-tests compared each student's Post minus Pre (Conventional) versus Post minus RetroPre (Adjusted-effect); effect-sizes used Cohen's-d. ResultsAmong 267 students, conventional change had a 0.5 effect-size (medium), while adjusted-effect change was 1.5 (very-large). Notably, some students’ conventional change was negative, though none of those same students reported negative-change in their adjusted-effect. ConclusionWe found a substantial response-shift in our educational context, and evidence for improved accuracy. Moreover, RetroPre/Post appeared more efficient for students (as they could complete both in one sitting at course-end, instead of twice with conventional Pre/Post), and staff (no one needed to match each student's Pre and Post assessments).

Response Shifts in the Canadian Occupational Performance Measure: A Convergent Mixed-Methods Study.

A response shift (RS) is a phenomenon in which there is an individual perceptual gap between pre and post assessments. RS effects were not considered in the Canadian Occupational Performance Measure (COPM) development process. To detect the effects of RS on the COPM. Convergent mixed-methods research. Subacute rehabilitation hospital in Japan. Nineteen adult patients with a range of neurological and musculoskeletal conditions recruited from a subacute rehabilitation hospital. In the qualitative analysis, patients' perceptions regarding occupation identified by the COPM were compared between the initial assessment (Time 1 [T1]) and a reassessment (Time 2 [T2]). In the quantitative study, patients were asked to re-rate the occupations in which the RS had occurred, giving feedback on their perceptions at T1 (T2'). The difference between T2 and T2' was calculated to clarify the magnitude of the RS. Of the 19 patients, 18 had an RS in at least one occupation. The RS effects were classified into five categories: Replacing, Adding, Reducing, Unspecified, and Embodiment. Ninety occupations were extracted from all the patients, and 46 (51.1%) were affected by RS. The percentages of occupations for which the change in score due to RS exceeded the minimal clinically important difference (±2 points) was 26.1% (12 of 46) for COPM-Performance scores and 30.4% (14 of 46) for COPM-Satisfaction scores. Diverse RS effects have been identified in the COPM, which also affect score interpretation. Plain-Language Summary: The Canadian Occupational Performance Measure has a potential measurement bias that is due to a response shift in which there is an individual perceptual gap between pre and post assessments. The results of this study reveal a need to establish more accurate measurement methods to reduce the impact of response shifts on COPM scores.

Response shift in coronary artery disease.

Patients with coronary artery disease (CAD) experience significant angina symptoms and lifestyle changes. Revascularization procedures can result in better patient-reported outcomes (PROs) than optimal medical therapy (OMT) alone. This study evaluates the impact of response shift (RS) on changes in PROs of patients with CAD across treatment strategies. Data were from patients with CAD in the Alberta Provincial Project on Outcome Assessment in Coronary Heart Disease (APPROACH) registry who completed the 16-item Canadian version of the Seattle Angina Questionnaire at 2weeks and 1year following a coronary angiogram. Multi-group confirmatory factor analysis (MG-CFA) was used to assess measurement invariance across treatment groups at week 2. Longitudinal MG-CFA was used to test for RS according to receipt of coronary artery bypass grafting (CABG), percutaneous coronary intervention (PCI), or optimal medical therapy (OMT) alone. Of the 3116 patients included in the analysis, 443 (14.2%) received CABG, 2049(65.8%) PCI, and the remainder OMT alone. The MG-CFA revealed a partial-strong invariance across the treatment groups at 2weeks (CFI = 0.98, RMSEA [90% CI] = 0.05 [0.03, 0.06]). Recalibration RS was detected on the Angina Symptoms and Burden subscale and its magnitude in the OMT, PCI, and CABG groups were 0.32, 0.28, and 0.53, respectively. After adjusting for RS effects, the estimated target changes were largest in the CABG group and negligible in the OMT group. Adjusting for RS is recommended in studies that use SAQ-CAN to assess changes in patients with CAD who have received revascularization versus OMT alone.

Well-Being, Mental Health, and Study Characteristics of Medical Students before and during the Pandemic.

Medical students report high demands, stressors, pressure to perform, and a lack of resources, and are therefore at a higher risk for mental strain and burnout. Due to the COVID-19 pandemic, study conditions have changed, and new challenges have emerged. Thus, this study aimed to examine medical students' well-being, mental health, and study characteristics before and during the pandemic. Data from 988 Austrian medical students were included into the cross-sectional comparisons, and 63 students were included into the longitudinal analyses (variance analyses/t-tests or appropriate non-parametric tests). Well-being before and during the pandemic did not differ significantly but the peri-pandemic cohort reported higher study satisfaction, more social support from lecturers, and less emotional exhaustion, cognitive demands, and stressors (information problems, organizational stressors, work overload). Longitudinally, work overload was also perceived to be higher before the pandemic; however, study satisfaction was lower. During the pandemic, approximately every seventh student exceeded the cut-off value for generalized anxiety disorder, and approximately every tenth student exceeded the cut-off value for major depression. These unexpected peri-pandemic results concerning constant high well-being, study satisfaction, and the perception of conditions may be based on response shift effects that require further exploration. The scores exceeding the reasonable cut-point for identifying probable cases of generalized anxiety disorder and depression may reflect medical students' needs, calling for an in-depth analysis if further health promotion is necessary.

A systematic review of the use of the then-test for evaluating health-related quality of life in cancer patients.

The then-test, also known as the retrospective pre- and post-test design method, is a measurement used to evaluate response shift. The method requires patients to assess their previous healthrelated quality of life (HRQoL) and provide a retrospective judgement based on their current perspectives. The then-test, however, has been criticized for its reliability and validity. The objective of this systematic review is to summarize the current literature for the use of the then-test for evaluating HRQoL in cancer patients and account for potential response shift effects. A literature search was conducted in May 2022 using MEDLINE, PubMed, and PsychINFO dated from 2005 to the time of the search. Studies were included if they (I) used the then-test and (II) involved a population of cancer patients (any cancer site). The literature search resulted in 16 studies, published between 2005 and 2020. All studies used the then-test to detect response shifts. The EORTC QLQ-C30 and EORTC QLQ-BR23 questionnaires were the most common assessments used to evaluate HRQoL. Of the 16 articles, 5 exclusively reported on breast cancer, 5 reported on prostate cancer, and the remaining included all cancer sites. Most studies looked at patients undergoing a combination of chemotherapy, radiotherapy, and hormonal therapy. The mean differences between the retrospective assessment at both 3 and 6 months were significant for various quality of life (QoL) dimensions. Patients in some studies recalled their pretreatment HRQoL (then-score) as better than the pretreatment baseline scores and others reported them as worse, both confirming the existence of a response shift. Our review demonstrates that the then-test measurement tool has been commonly used for the detection of response shift. Newer measures for response shift have become more accepted; the thentest, if used, should be used with caution considering its limitations and the emergence of more advanced methods.

Validation of the German language version of the Chronic Ear Survey and its psychometric comparison with an established German measurement instrument.

With the Chronic Ear Survey (CES), avalidated measurement instrument for the assessment of disease-specific health-related quality of life (HRQoL) has been available internationally since 2000. The aim of this study was to provide avalidated German version of this international instrument and to compare it with the German Chronic Otitis Media Outcome Test15 (COMOT-15). The CES was translated into German via aforward-backward translation process. For validation, 79patients with COM undergoing middle ear surgery were prospectively included. HRQoL was determined preoperatively and 6months postoperatively using the CES and the COMOT-15. Pure tone audiometry was also performed at both measurement time points. In the control examination, an additional retrospective assessment of the preoperative situation was additionally performed using the CES and the COMOT-15 to assess the response shift. The determined psychometric characteristics were internal consistency, test-retest reliability, discrimination validity, agreement validity, responsiveness, and response shift for both measurement instruments. Convergent validity of both measurement instruments was assessed using linear regression. On the basis of the CES, patients with COM could be reliably distinguished from patients with healthy ears. The CES showed satisfactory reliability with high internal consistency (Cronbachα 0.65-0.85) and high retest reliability (r > 0.8). The global assessment of HRQoL impairment correlated very well with the scores of the CES (r = 0.51). In addition, it showed ahigh sensitivity to change (standardized response mean -0.86). Compared to the COMOT-15, it showed alower response shift (effect size -0.17 vs. 0.44). Both measurement instruments correlated only slightly with air conduction hearing threshold (r = 0.29 and r = 0.24, respectively). The concordant validity of both measurement instruments was high (r = 0.68). The German version of the CES shows satisfactory psychometric characteristics, so that its use can be recommended. The CES focuses on the influence of ear symptoms on HRQoL, whereas the COMOT-15 also includes functional and psychological aspects. Due to only minor response shift effects, the CES is particularly suitable for studies with multiple repeat measurements.

Validation of the German language version of the Chronic Ear Survey and its psychometric comparison with an established German language measurement instrument. German version

With the Chronic Ear Survey (CES), avalidated measurement instrument for the assessment of disease-specific health-related quality of life (HRQoL) has been available internationally since 2000. The aim of this study was to provide avalidated German version of this international instrument and to compare it with the German Chronic Otitis Media Outcome Test15 (COMOT-15). The CES was translated into German via aforward-backward translation process. For validation, 79patients with COM undergoing middle ear surgery were prospectively included. HRQoL was determined preoperatively and 6months postoperatively using the CES and the COMOT-15. Pure tone audiometry was also performed at both measurement time points. In the control examination, an additional retrospective assessment of the preoperative situation was additionally performed using the CES and the COMOT-15 to assess the response shift. The determined psychometric characteristics were internal consistency, test-retest reliability, discrimination validity, agreement validity, responsiveness, and response shift for both measurement instruments. Convergent validity of both measurement instruments was assessed using linear regression. On the basis of the CES, patients with COM could be reliably distinguished from patients with healthy ears. The CES showed satisfactory reliability with high internal consistency (Cronbachα 0.65-0.85) and high retest reliability (r > 0.8). The global assessment of HRQoL impairment correlated very well with the scores of the CES (r = 0.51). In addition, it showed ahigh sensitivity to change (standardized response mean -0.86). Compared to the COMOT-15, it showed alower response shift (effect size -0.17 vs. 0.44). Both measurement instruments correlated only slightly with air conduction hearing threshold (r = 0.29 and r = 0.24, respectively). The concordant validity of both measurement instruments was high (r = 0.68). The German version of the CES shows satisfactory psychometric characteristics, so that its use can be recommended. The CES focuses on the influence of ear symptoms on HRQoL, whereas the COMOT-15 also includes functional and psychological aspects. Due to only minor response shift effects, the CES is particularly suitable for studies with multiple repeat measurements.

Depression trajectories during the COVID-19 pandemic: a secondary analysis of the impact of cognitive-appraisal processes

PurposeThis study characterized depression trajectories during the COVID pandemic and investigated how appraisal and changes in appraisal over time related to these depression trajectories.MethodsThis longitudinal study of the psychosocial impact of the COVID-19 pandemic included 771 people with data at three timepoints over 15.5 months. The depression index was validated using item-response-theory methods and receiver-operating-characteristic curve analysis. The Quality of Life (QOL) Appraisal Profilev2 Short-Form assessed cognitive-appraisal processes. Sequence analysis characterized depression-trajectory groups, and random effects models examined appraisal main effects, appraisal-by-group, and appraisal-by-group-by-time interactions.ResultsSequence analysis generated six trajectory groups: Stably Well (n = 241), Stably Depressed (n = 299), Worsening (n = 79), Improving (n = 83), Fluctuating Pattern 1 (No–Yes–No; n = 41), and Fluctuating Pattern 2 (Yes–No–Yes; n = 28). While all groups engaged in negative appraisal processes when they were depressed, the Stably Depressed group consistently focused on negative aspects of their life. Response-shift effects were revealed such that there were differences in the appraisal-depression relationship over time for standards of comparison and recent changes for the Stably Depressed, and in health goals for those Getting Better.ConclusionThe present work is, to our knowledge, the first study of response-shift effects in depression. During these first 15.5 pandemic months, group differences highlighted the connection between negative appraisals and depression, and response-shift effects in these relationships over time. Egregious life circumstances may play a lesser role for the Stably Depressed but a greater role for people who have transient periods of depression as well as for those with improving trajectories (i.e., endogenous vs. reactive depression). How one thinks about QOL is intrinsically linked to mental health, with clear clinical implications.

Response shift in parent-reported psychopathology in children with chronic physical illness.

Because physical-mental comorbidity in children is relatively common, this study tested for response shift (RS) in children with chronic physical illness using a parent-reported measure of child psychopathology. Data come from Multimorbidity in Children and Youth across Life-course (MY LIFE), a prospective study of n = 263 children aged 2-16years with physical illness in Canada. Parents provided information on child psychopathology using the Ontario Child Health Study Emotional Behavioral Scales (OCHS-EBS) at baseline and 24months. Oort's structural equation modeling was used to test for different forms of RS in parent-reported assessments between baseline and 24months. Model fit was evaluated using root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardized root mean residual (SRMR). There were n = 215 (81.7%) children with complete data and were included in this analysis. Of these, n = 105 (48.8%) were female and the mean (SD) age was 9.4 (4.2) years. A two-factor measurement model provided good fit to the data [RMSEA (90% CI) = 0.05 (0.01, 0.10); CFI = 0.99; SRMR = 0.03]. Non-uniform recalibration RS was detected on the conduct disorder subscale of the OCHS-EBS. This RS effect had negligible impact on the longitudinal change in externalizing and internalizing disorders construct over time. Response shift detected on the conduct disorder subscale of the OCHS-EBS, indicated that parents of children with physical illness may recalibrate their responses on child psychopathology over 24months. Researchers and health professionals should be aware of RS when using the OCHS-EBS to assess child psychopathology over time.

17th International Conference on Malignant Lymphoma, Palazzo dei Congressi, Lugano, Switzerland, 13 - 17 June, 2023.

Introduction: Few quality of life (QoL) data are available for diffuse large B-cell lymphoma (DLBCL) patients during the first year of diagnosis. Due to the disease and treatment, patients can adapt to their condition and then change their criteria to assess QoL over time, resulting in a response shift (RS) effect. This study aims to provide QoL level for DLBCL patients at diagnosis and after 1 year, and to assess the occurrence of a RS effect. Methods: Data from DLBCL patients prospectively included in the French multicentric cohort REALYSA were used. QoL was collected at diagnosis and at 1 year using the EORTC QLQ-C30 and QLQ-NHL-HG29 (non-Hodgkin high-grade lymphoma) questionnaires. The recently validated QLQ-NHL-HG29 specific questionnaire assesses 5 symptomatic dimensions: symptom burden (SB), neuropathy (NP), physical condition/fatigue (PC), emotional impact (EI), worries/fears about health and functioning (WF). A score is generated per dimension on a 0 to 100 scale. Analyses were done on patients who received standard first-line immunochemotherapy regimen, with QoL questionnaire at diagnosis and at 1 year, and without relapse before 1 year questionnaire completion to avoid bias related to second line therapy. Scores were reported according to age, gender, performance status (PS) and disease stage. We examined the prevalence of clinically important problems/symptoms on QLQ-C30 scales at patient level using validated thresholds (Giesinger et al. J Clin Epidemiol. 2020). The potential occurrence of a RS effect was explored using the Oort procedure. Results: Between Nov 2018 and Dec 2021, 3116 patients were enrolled in REALYSA including 1198 DLBCL. Among 964 patients without relapse at 1 year, 523 had completed questionnaires at baseline and 1 year. For Global QoL, 57% of patients presented clinically important problems at baseline and 42% at 1 year. Regarding the QLQ-NHL-HG29, the mean PC score was 32 (SD 25) [PS 0–1 vs. 2: 29 (SD 24) vs. 47 (SD 25)] at baseline and 23 (SD 21) [PS 0–1 vs. 2: 22 (SD 20) vs. 27 (SD 22)] at 1 year; the mean NP score was 12 (SD 20) [PS 0–1 vs. 2: 11 (SD 19) vs. 16 (SD 26)] at baseline and 23 (SD 28) [PS 0–1 vs. 2: 23 (SD 27) vs. 24 (SD 29)] at 1 year. Among the QLQ-NHL-HG29, a RS effect was observed for NP, PC and WF. The highest effect was observed for NP (Figure 1), suggesting that there is no increase in NP level at 1 year compared to baseline (mean change of -1 points (SD 1)). If no RS effect was considered, it would have been concluded that the NP level had increased at 1 year compared to diagnosis (mean change of 12 points (SD 11)). The research was funded by: Institut National du Cancer (INCa, N°2021-130) Keyword: Aggressive B-cell non-Hodgkin lymphoma No conflicts of interests pertinent to the abstract.

Response-shift effects in childhood cancer survivors: Aprospective study.

Treatment-related late effects can worsen over time among cancer survivors. Such worsening health states may trigger changes in internal standards, values, or conceptualization of quality-of-life (QOL). This "response-shift" phenomenon can jeopardize the validity of QOL assessment, and misrepresent QOL comparisons over time. This study tested response-shift effects in reporting future-health concerns among childhood cancer survivors who experienced progression in chronic health conditions (CHCs). 2310 adult survivors of childhood cancer from St. Jude Lifetime Cohort Study completed a survey and clinical assessment at two or more timepoints. Based on 190 individual CHCs graded for adverse-event severity, global CHC burden was classified as "progression" or "non-progression". QOL was assessed using the SF-36TM eight domains and physical- and mental-component summary scores (PCS, MCS). A single global item measured concerns about future health. Random-effects models comparing survivors with and without progressive global CHC burden (progressors vs. non-progressors) evaluated response-shift effects (recalibration, reprioritization, reconceptualization) in reporting future-health concerns. Compared with non-progressors, progressors were more likely to de-emphasize (or downplay) overall physical and mental health in evaluating future-health concerns (p-values<0.05), indicating recalibration response-shift, and more likely to de-emphasize physical health earlier rather than later in follow-up (p-value<0.05), indicating reprioritization response-shift. There was evidence for a reconceptualization response-shift with progressor classification associated with worse-than-expected future-health concerns and physical health, and better-than-expected pain and role-emotional functioning (p-values<0.05). We identified three types of response-shift phenomena in reporting concerns about future health among childhood cancer survivors. Survivorship care or research should consider response-shift effects when interpreting changes in QOL over time.

Response-shift effects in childhood cancer survivors: A prospective study

Response-shift effects in childhood cancer survivors: A prospective study

Impact of response shift effects in the assessment of self-reported depression during treatment: Insights from a rTMS versus Venlafaxine randomized controlled trial

PurposePatient-Reported Outcomes are essential to properly assess treatment effectiveness in randomized clinical trial (RCT) for Major Depressive Disorder (MDD). MDD self-assessment may vary over time depending on change in the meaning of patients’ self-evaluation of depression, i.e. Response Shift (RS). Our aim was to investigate RS and its impact on different depression domains in a clinical trial comparing rTMS versus Venlafaxine. MethodsThe occurrence and type of RS was determined using Structural Equation Modeling applied to change over time in 3 domains (Sad Mood, Performance Impairment, Negative Self-Reference) of the short-form Beck Depression Inventory (BDI-13) in a secondary analysis of a RCT on 170 patients with MDD treated by rTMS, venlafaxine or both. ResultsRS was evidenced in the venlafaxine group in the Negative Self-Reference and Sad Mood domains. ConclusionRS effects differed between treatment arms in self-reported depression domains in patients with MDD. Ignoring RS would have led to a slight underestimation of depression improvement, depending on treatment group. Further investigations of RS and advancing new methods are needed to better inform decision making based on Patient-Reported Outcomes.

Listening to the elephant in the room: response-shift effects in clinical trials research

BackgroundWhile a substantial body of work postulates that adaptation (response-shift effects) may serve to hide intervention benefits, much of the research was conducted in observational studies, not randomized-controlled trials. This scoping review identified all clinical trials that addressed response shift phenomena, and characterized how response-shift effects impacted trial findings.MethodsA scoping review was done of the medical literature from 1968 to 2021 using as keywords “response shift” and “clinical trial.” Articles were included if they were a clinical trial that explicitly examined response-shift effects; and excluded if they were not a clinical trial, a full report, or if response shift was mentioned only in the discussion. Clinical-trials papers were then reviewed and retained in the scoping review if they focused on randomized participants, showed clear examples of response shift, and used reliable and valid response-shift detection methods. A synthesis of review results further characterized the articles’ design characteristics, samples, interventions, statistical power, and impact of response-shift adjustment on treatment effect.ResultsThe search yielded 2148 unique references, 25 of which were randomized-controlled clinical trials that addressed response-shift effects; 17 of which were retained after applying exclusion criteria; 10 of which were adequately powered; and 7 of which revealed clinically-important response-shift effects that made the intervention look significantly better.ConclusionsThese findings supported the presumption that response shift phenomena obfuscate treatment benefits, and revealed a greater intervention effect after integrating response-shift related changes. The formal consideration of response-shift effects in clinical trials research will thus not only improve estimation of treatment effects, but will also integrate the inherent healing process of treatments.Key pointsThis scoping review supported the presumption that response shift phenomena obfuscate treatment benefits and revealed a greater intervention effect after integrating response-shift related changes.The formal consideration of response-shift effects in clinical trials research will not only improve estimation of treatment effects but will also integrate the inherent healing process of treatments.

Midwives’ empathy and shared decision making from women’s perspective - sensitivity of an assessment to compare quality of care in prenatal and obstetric care

BackgroundFor quality-oriented evaluation of prenatal and obstetric care, it is important to systematically consider the perspective of the women receiving care in order to comprehensively assess and optimize quality in a woman-centered manner. Empathy and Shared Decision Making (SDM) are essential components of woman-centered midwifery care. The aim of the study was to analyze measurement invariance of the items of the Consultation and Relational Empathy (CARE) and Shared Decision Making-Questionnaire (SDM-Q-9) scales depending on the prenatal versus obstetric care setting.MethodsOne hundred fifty women retrospectively assessed aspects of woman-centered midwifery care in both prenatal and obstetric care setting. The birth of the child was a maximum of 12 months ago. A structural equation modelling approach was adopted to separate true effects from response shift (RS) effects depending on care setting. The latter were analyzed in terms of recalibration (changing women’s internal measurement standards), Reprioritization (changing associations of items and construct) as well as Reconceptualization (redefining the target construct).ResultsA response shift model was identified for both assessments (pregnancy/birth: CFI = .96/.96; SRMR = .046/.051). At birth, both scales indicated lower quality of care compared with prenatal care (SDM-Q-9-M/CARE-8-M:|d| = 0.190/0.392). Although no reconceptualization is required for the items of both scales, RS effects are evident for individual items. Due to recalibration and reprioritization effects, the true differences in the items are partly underestimated (SDM-Q-9-M/CARE-8-M: 3/2 items) or overestimated (4/2 items).ConclusionThe structure of the constructs SDM and Empathy, indicating woman-centered midwifery care, are moderated by the care settings. To validly assess midwives’ empathy and shared decision making from women’s perspective, setting-dependent response shift effects have to be considered. The proven item-specific response effects contribute to a better understanding of construct characteristics in woman-centered care by midwives during pregnancy and childbirth.

Response-Shift Effects in the Future-Health Expectations Among Adult Survivors of Childhood Cancer: A Prospective Study from the St. Jude Lifetime Cohort

Response-Shift Effects in the Future-Health Expectations Among Adult Survivors of Childhood Cancer: A Prospective Study from the St. Jude Lifetime Cohort

If it's information, it's not "bias": a scoping review and proposed nomenclature for future response-shift research.

The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as "bias," "noise," "nuisance," or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings. A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles). Of 10 methods used, papers using four of them utilized terms like "bias" and aimed to remove response-shift effects to reveal "true change." Yet, the investigators' reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods. It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.

Norm-based comparison of the quality-of-life impact of ravulizumab and eculizumab in paroxysmal nocturnal hemoglobinuria

AimsParoxysmal nocturnal hemoglobinuria (PNH) is a rare and life-threatening intravascular hematologic disorder with significant morbidity and premature mortality. Clinical trials (NCT02946463 and NCT03056040) comparing ravulizumab with eculizumab for PNH have supported the non-inferiority of the former and similar safety and tolerability. This secondary analysis compared PNH trial participants after 26 weeks on either treatment (n = 438) to a general-population sample (GenPop) (n = 15,386) and investigated response-shift effects.MethodsMultivariate analysis of covariance (MANCOVA) investigated function and symptom scores on the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30 of people with PNH as compared to GenPop, after covariate adjustment. Risk-factor groups were created based on clinical indicators known to be associated with worse PNH outcomes, and separate MANCOVAs were computed for lower- and higher-risk-factor groups. Differential item functioning (DIF) analyses examined whether item response varied systematically (1) by treatment, (2) compared to GenPop, and (3) over time, the latter two suggesting and reflecting response-shift effects, respectively. DIF analyses examined 24 items from scales with at least two items. Recalibration response shift was operationalized as uniform DIF over time, reflecting the idea that, for a given group, the difficulty of endorsing an item changes over time, after adjusting for the total subscale score. Reprioritization response shift was operationalized as non-uniform DIF over time, i.e., the relative difficulty of endorsing an item over time changes across the total domain score.ResultsAcross PNH risk-factor levels, people who had been on either treatment for 26 weeks reported better-than-expected functioning and lower symptom burden compared to GenPop. Ravulizumab generally showed larger effect sizes. Results were similar for lower and higher PNH risk factors, with slightly stronger effects in the former. DIF analyses revealed no treatment DIF, but did uncover group DIF (9 items with uniform DIF, and 11 with non-uniform) and DIF over time (7 items with uniform DIF, and 3 with non-uniform).ConclusionsThis study revealed that people with PNH on ravulizumab or eculizumab for 26 weeks reported QOL levels better than those of the general population. Significant effects of DIF by group and DIF over time support recalibration and reprioritization response-shift effects. These findings suggest that the treatments enabled adaptive changes.

Creating idiometric short-form measures of cognitive appraisal: balancing theory and pragmatics

BackgroundThe Rapkin and Schwartz appraisal theory and measure provided a path toward documenting response-shift effects and describing individual differences in ways of thinking about quality of life (QOL) that distinguished people in different circumstances. Recent work developed and validated the QOL Appraisal Profileversion 2 (QOLAPv2), an 85-item measure that taps response-shift-detection domains of Frame of Reference, Standards of Comparison, Sampling of Experience, and Combinatory Algorithm. Recent theoretical work proposed that appraisal measurement constitutes a new class of measurement (idiometric), distinct from psychometric and clinimetric. To validate an idiometric measure, one would document that its items reflect different circumstances and population characteristics, and explain variance in QOL. The present work sought to develop idiometric short-forms of the QOLAPv2 item bank by examining which items were most informative, retaining the appraisal-domain structure.MethodsThis secondary analysis (n = 1481) included chronically-ill patients and their caregivers from a longitudinal web-based survey (mean follow-up 16.6 months). Data included the QOLAPv2, the Center for Disease Control Healthy Days Core Module, the PROMIS-10 Global Health, and demographic/medical variables. Appraisal items were measured at baseline (relevant to understanding cognitive appraisal processes); and with change scores (sensitive to response-shift effects). Multivariate analysis of covariance examined what demographic and health-status change variables were reflected by each of 85 appraisal items (in five sets), as dependent variables, and other demographic/medical variables. Multiple linear regression examined how appraisal items explained variance in global physical- and mental-health change, after covariate adjustment. A tally summarized item performance across all five sets of cross-sectional and longitudinal analyses.ResultsThe vast majority (i.e., 80%) of the QOLAPv2 items performed well across the analyses presented. Using a relatively strict criterion of explaining meaningful variance across 60% of analyses, one would retain 68 items. A more lenient criterion (40%) would retain 71.ConclusionsThe present study provides heuristics to support investigators’ creating ‘discretionary’ QOLAPv2 short-forms to fit their study aim and amplifying individual differences in the cognitive processes underlying QOL. This approach enables adapting the measure to the study population, as per the expectation that respondent populations differ in the predominant cognitive processes used.

Response shift after coronary revascularization

PurposeThe aims of this study were to investigate (1) the extent to which response shift occurs among patients with coronary artery disease (CAD) after coronary revascularization, (2) whether the assessment of changes in health-related quality of life (HRQoL), controlled for response shift, yield more valid estimates of changes in HRQoL, as indicated by stronger associations with criterion measures of change, than without controlling for response shift, and (3) if occurrences of response shift are related to patient characteristics.MethodsPatients with CAD completed the SF-36 and the Seattle Angina Questionnaire (SAQ7) at baseline and 3 months after coronary revascularization. Sociodemographic, clinical and psychosocial variables were measured with the patient version of the New York Heart Association-class, Subjective Significance Questionnaire, Reconstruction of Life Events Questionnaire (RE-LIFE), and HEXACO personality inventory. Oort’s Structural Equation Modeling (SEM) approach was used to investigate response shift.Results191 patient completed questionnaires at baseline and at 3 months after treatment. The SF-36 showed recalibration and reprioritization response shift and the SAQ7 reconceptualization response shift. Controlling for these response shift effects did not result in more valid estimates of change. One significant association was found between reprioritization response shift and complete integration of having CAD into their life story, as indicated by the RE-LIFE.ConclusionResults indicate response shift in HRQoL following coronary revascularization. While we did not find an impact of response shift on the estimates of change, the SEM approach provides a more comprehensive insight into the different types of change in HRQoL following coronary revascularization.