Visualizing meaningful change in small sample sizes.

Abstract

e18656 Background: Patient Reported Outcomes (PROs) are included in trials to incorporate patient voice into the drug development process. In oncology, PROs typically evaluate patient physical function and/or symptom burden. Clinicians need to be able to describe expected patient outcomes in terms of meaningful change in conversations with patients. FDA guidances outline analysis methods for measuring meaningful change, and the results are increasingly used to inform regulatory decisions and in drug packaging. Improving best practices in measuring meaningful change is crucial for regulatory review and to help clinicians communicate effectively with patients. Current best practice under FDA guidance is to compute meaningful within-patient change (MWPC) using an anchor-based approach. The relationship between change scores and anchor groups is visualized via empirical cumulative distribution functions (eCDFs). However, when sample sizes within anchor groups are small, it may be difficult to compute the distribution function. In contrast, a scatterplot approach does not require any computation. The objective was to evaluate under what study conditions (i.e., sample sizes and magnitude of meaningful change) would it be advantageous to interpret meaningful change with a scatterplot approach. Methods: Data simulated to emulate real PRO data from a trial, including attrition due to death, were used to highlight shortcomings in current practice and propose a robust alternative. A simulation study permitted a variety sample sizes and magnitudes of meaningful change to be evaluated. The eCDF was implemented in accordance with current guidance and compared to scatterplots. The scatterplot approach was operationalized as follows: the subject scores were plotted with the baseline scores on the x-axis and the follow-up scores on the y-axis. The scatterplot was stratified by color to represent anchor group-specific scores. Subjects who did not respond at follow-up due to progression or death were plotted in a separate color, using the baseline response and an assigned follow-up score reflecting maximum severity. A 45-degree line represented No Change. The vertical distance between the subject score and the 45-degree line represented the observed change. Results: When sample size was below n = 10 subjects per anchor group, the eCDF yielded uninterpretable visualizations. At sample sizes n = 1-20 per anchor group, the scatterplot approach consistently yielded interpretable visualizations of meaningful change. Conclusions: The scatterplot approach avoids methodological complications and thus ensures transparency in the evaluation of meaningful change. Recommendations to supplement current best practice with the scatterplot approach described above should be considered.