Abstract
Introduction: Diverse algorithms for signal detection exist. However, inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. We aimed to explore potential safety signals for two pneumococcal vaccines in a spontaneous reporting database and compare the results and performances among the algorithms. Methods: Safety surveillance was conducted using the Korea national spontaneous reporting database from 1988 to 2017. Safety signals for pneumococcal vaccine and its subtypes were detected using the following the algorithms: disproportionality methods comprising of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC); empirical Bayes geometric mean (EBGM); and tree-based scan statistics (TSS). Moreover, the performances of these algorithms were measured by comparing detected signals with the known AEs or pneumococcal vaccines (reference standard). Results: Among 10,380 vaccine-related AEs, 1135 reports and 101 AE terms were reported following pneumococcal vaccine. IC generated the most safety signals for pneumococcal vaccine (40/101), followed by PRR and ROR (19/101 each), TSS (15/101), and EBGM (1/101). Similar results were observed for its subtypes. Cellulitis was the only AE detected by all algorithms for pneumococcal vaccine. TSS showed the best balance in the performance: the highest in accuracy, negative predictive value, and area under the curve (70.3%, 67.4%, and 64.2%). Conclusion: Discrepancy in the number of detected signals was observed between algorithms. EBGM and TSS calibrated noise better than disproportionality methods, and TSS showed balanced performance. Nonetheless, these results should be interpreted with caution due to a lack of a gold standard for signal detection.
Highlights
The World Health Organization-Adverse Reaction Terminology (WHO-ART) is constructed as a tree structure [18], and we used the level of preferred terms (PTs) that represent the principal terminology used for documentation
From a total of 1,341,724 reports in the Korea Adverse Event Reporting System (KAERS), we identified 30,062 (2.2%) reports involving vaccination (Figure 1)
Among the signal detection algorithms, both the tree-based scan statistic and the disproportionality methods generated a comparable number of signals, whereas empirical Bayes geometric mean (EBGM) generated the least number of signals
Summary
Inconsistent results are often encountered among the algorithms due to different levels of specificity used in defining the adverse events (AEs) and signal threshold. Safety signals for pneumococcal vaccine and its subtypes were detected using the following the algorithms: disproportionality methods comprising of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC); empirical Bayes geometric mean (EBGM); and tree-based scan statistics (TSS). Drug Administration (FDA) uses the empirical Bayes geometric mean (EBGM) [2,3] These algorithms have been previously validated, inconsistent signal detection results are often encountered within and between the algorithms due to different levels of specificity in defining adverse events (AEs) and signal score threshold [4,5]. As signal detection through this method is fundamentally based on a pre-defined tree structure constructed with the variables of interest grouped together at different specificity levels, it can both evaluate a variable alone and a group of related variables simultaneously [9]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
