A Decade of Data Mining and Still Counting

Abstract

The introduction of database-wide disproportionality screening for signal detection in spontaneous reporting systems (SRS) sparked a renaissance in pharmacovigilance research notable for numerous peer reviewed research articles, three expert working groups/white papers, countless meetings, symposia, workshops, graduate school theses and aggressive promotion of proprietary software. In addition to expanding the pharmacovigilance toolkit, this research has yielded ancillary benefits beyond patient safety, including an increased awareness of data quality issues such as case report duplication, the importance of adverse event coding terminology, the proper definition of signal in drug safety, the logic of signal detection and an admonition that conflicts of interest, both intellectual and financial, may not only involve the ‘usual suspects’ such as software vendors, but also other stakeholders that may not normally come tomind, such as regulatory authorities. The absence of standard test beds for systematic evaluation of signal detection methodologies is an impediment to progress. A fundamental question is what steps of a comprehensive screening strategy to test and what to compare performance against. Herein, we discuss the value of data mining evaluation studies that continue to appear, and related issues, in the hope of facilitating pharmacovigilance system benchmarking and optimization. As a basis for discussion we focus on three recent performance evaluations published in Drug Safety (see table I).