Re-evaluating randomized clinical trials of psychological interventions:Impact of response shift on the interpretation of trial results.

J. Lancee,T. van der Zweerde,I. M. Verdonck-de Leeuw,F. J. Oort,M. G. E. Verdam,W. van Ballegooijen,C. J. M. Holtmaat,H. Knoop,H. Riper,M. de Wit,M. A. G. Sprangers,A. van Straten

doi:10.1371/journal.pone.0252035

Abstract

Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. However, comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called 'response shift'. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients' frames of reference. The overall aim is to gain insight into the general relevance of response shift for psychological health intervention research. Specifically, the aim is to re-analyse data of published randomized controlled trials (RCTs) investigating the effectiveness of psychological interventions targeting different health aspects, to assess (1) the occurrence of response shift, (2) the impact of response shift on interpretation of treatment effectiveness, and (3) the predictive role of clinical and background variables for detected response shift. We re-analysed data from RCTs on guided internet delivered cognitive behavioural treatment (CBT) for insomnia in the general population with and without elevated depressive symptoms, an RCT on meaning-centred group psychotherapy targeting personal meaning for cancer survivors, and an RCT on internet-based CBT treatment for persons with diabetes with elevated depressive symptoms. Structural equation modelling was used to test the three objectives. We found indications of response shift in the intervention groups of all analysed datasets. However, results were mixed, as response shift was also indicated in some of the control groups, albeit to a lesser extent or in opposite direction. Overall, the detected response shifts only marginally impacted trial results. Relations with selected clinical and background variables helped the interpretation of detected effects and their possible mechanisms. This study showed that response shift effects can occur as a result of psychological health interventions. Response shift did not influence the overall interpretation of trial results, but provide insight into differential treatment effectiveness for specific symptoms and/or domains that can be clinically meaningful.

Highlights

Psychological interventions play an important role in the promotion of good health, which The World Health Organization (WHO) defines as a ‘state of complete physical, mental and social well-being and not merely the absence of disease or infirmity’
In both the treatment- and control groups there was a small but significant association of religiousness with the detected reprioritization response shift in the subscale relation with God/higher order (RG) (MCGP-CS group: r = .17, p = .007; control group: r = .19, p = .026;). These results indicate that being religious before the start of treatment is associated with more reprioritization response shift in the treatment group, and with less reprioritization response shift in the control group
In the personal meaning for cancer survivors dataset, we found that relation with god or a higher order became more important to the measurement of personal meaning for patients in the treatment group, for those who indicated to be religious, whereas it became less important to the measurement of personal meaning in the control group, for those who indicated to be non-religious

Summary

Background

Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. Comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called ‘response shift’. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients’ frames of reference. Peer Review History: PLOS recognizes the benefits of transparency in the peer review process; we enable the publication of all of the content of peer review and author responses alongside final, published articles. Data Availability Statement: The data that were used for the analyses presented in the current

Aims

Method

Results

Conclusion

Introduction

Objective

Discussion

Strengths and limitations

Conclusions