Guidelines for secondary analysis in search of response shift

Abstract

Response shift methods have developed substantially in the past decade, with a notable emphasis on model-based methods for response shift detection that are appropriate for the analysis of existing data sets. These secondary data analyses have yielded useful insights and motivated the continued growth of response shift methods. However, there are also challenges inherent to the successful use of secondary analysis for response shift detection. Based on our experience with a number of secondary analyses, we propose guidelines for the optimal implementation of secondary analysis for detecting response shift. We review the definition of response shift and recent advances in response shift theory. We describe current statistical methods that have been developed for or applied to response shift detection. We then discuss lessons learned when using these methods to test specific hypotheses about response shift in existing data and of the features of a data set that could guide early decision-making about undertaking a secondary analysis. A checklist is provided that includes guidelines for secondary analyses focusing on: (1) selecting an appropriate data set to investigate response shift; (2) prerequisites of data sets and their preparation for analysis; (3) managing missing data; (4) confirming that the data fit the requirements and assumptions of the selected response shift detection technique; (5) model fit evaluation; (6) interpreting results/response shift effect sizes; and (7) comparing findings across methods. The guidelines-checklist has the potential to stimulate rigorous and replicable research using existing data sets and to assist investigators in assessing the appropriateness and potential of a data set and model-based methods for response shift research.