Leveraging the Variability of Pharmacovigilance Disproportionality Analyses to Improve Signal Detection Performances.

Charles Khouri,Marion Lepelley,Matthieu Roustit,Jean-Luc Cracowski,Thuy Nguyen,Bruno Revol,Antoine Pariente

doi:10.3389/fphar.2021.668765

Abstract

Background: A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results. We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances. Methods: We used a validated reference set containing 399 true and false drug-event pairs and performed, with a frequentist and a Bayesian disproportionality method, seven types of analyses (model) for which the results were very unlikely to be related to actual differences in absolute risks of ADR. We calculated sensitivity, specificity and plotted ROC curves for each model. We then evaluated the predictive capacities of all models and assessed the impact of combining such models with the number of positive SDR for a given drug-event pair through binomial regression models. Results: We found considerable variability in disproportionality analysis results, both positive and negative SDR could be generated for 60% of all drug-event pairs depending on the model used whatever their truthfulness. Furthermore, using the number of positive SDR for a given drug-event pair largely improved the signal detection performances of all models. Conclusion: We therefore advocate for the pre-registration of protocols and the presentation of a set of secondary and sensitivity analyses instead of a unique result to avoid selective outcome reporting and because variability in the results may reflect the likelihood of a signal being a true adverse drug reaction.

Highlights

Spontaneous reports are a valuable data source to complete the pre-marketing safety profile of medical products, notably for rare or long latency adverse drug reactions (ADRs) (Hartmann et al, 1999; Hauben and Bate, 2009)
Several models can be used to overcome the multitude of biases related to spontaneous reporting rates of ADR, such as media alerts, selective reporting according to ADR severity, or time since the drug was first marketed (Seabroke et al, 2016; Wisniewski et al, 2016; Sandberg et al, 2020)
We identified the four outcomes in Vigibase by using a collection of MedDRA Preferred Terms (PT) or standardized MedDRA queries (SMQ) to match the broader definitions used in the reference set (Supplementary Table S1). (Reich et al, 2013; Ryan et al, 2013)

Summary

Introduction

Spontaneous reports are a valuable data source to complete the pre-marketing safety profile of medical products, notably for rare or long latency adverse drug reactions (ADRs) (Hartmann et al, 1999; Hauben and Bate, 2009). Automated screening tools using quantitative methods have been developed to detect signals of disproportionate reporting (SDR), i.e. a higher proportions of reporting of ADRs for a studied drug as compared to the other drugs in the database They allow broad screening to trigger further investigations, to detect more complex dependencies and to prioritize potential signals (Bate and Evans, 2009; Hauben and Aronson, 2009). A plethora of methods and models of disproportionality analyses for safety surveillance have been developed to date without consensus nor a gold standard, leading to methodological heterogeneity and substantial variability in results We hypothesized that this variability is inversely correlated to the robustness of a signal of disproportionate reporting (SDR) and could be used to improve signal detection performances

Methods

Results

Discussion

Conclusion