Comparative Effectiveness of Supplementation with a Juice Powder Concentrate: The Challenge of Multiplicity and Propensity Score Matching (P06-073-19)

Abstract

ObjectivesThere is a need for real-world evaluation of dietary supplementation in complex scenarios. We have designed an observational study intended to assess the comparative effectiveness of supplementation with a juice powder concentrate foreseeing multiple exposure groups, multiple primary endpoints and prospective use of propensity score matching. MethodsThree different exposure groups (Control, “Capsules” and “Shake”) and three primary endpoints have been identified (TNF-α, Homocysteine and Vitamin C). This study design presents a multiplicity problem with two sources of multiplicity. The two supplementation-control comparisons (Capsules vs Control and Shake vs Control) and the between-supplementation comparison (Capsules vs Shake) will serve as the first source of multiplicity. In addition to that, three multiple primary endpoints are tested in each comparison.A formal power analysis is carried out through simulations assuming: i) specific effect sizes per comparison per endpoint; ii) t-test with parallel gatekeeping with truncated Holm test as the multiplicity adjustment procedure; iii) control of the familywise error rate (FWER) at a pre-specified 0.05 level.For a proper comparative analysis of effectiveness, we foresee the use of matching on a propensity score (PS) prospectively. PSs are typically applied in retrospective cohort studies. However we initially developed a Random Forest-based PS model on historical data. The PSs predicted by this tool are used to match patients on an ongoing basis to evaluate the comparative effectiveness of supplementation. ResultsWith a sample size of 20 subjects per exposure group, the disjunctive power for testing each primary endpoint (i.e., the probability of establishing a significant effect in Capsules vs. Control or Shake vs. Control or Capsules vs. Shake) is 78%, 51%, 97% for TNF-α, Homocysteine and Vitamin C, respectively. ConclusionsThe increasing availability of computational resources and methods allows researchers to conduct comparative effectiveness cohort studies that require prospective data collection, by transporting Pss to new patients. Moreover, simulations allow examining the operating characteristics of complex testing frameworks. Funding SourcesZETA RESEARCH S.r.l.