Change Point Analysis for Detecting Vaccine Safety Signals.

Sun-Young Jung,Seung-Hun You,Jae-Eun Lee,Ye-Jin Kang,Eun Jin Jang,Min-Taek Lee,Myo-Song Kim,Joo-Hyeon Eom

doi:10.3390/vaccines9030206

Sun-Young Jung, Seung-Hun You + Show 6 more

Open Access

https://doi.org/10.3390/vaccines9030206

Copy DOI

Journal: Vaccines	Publication Date: Mar 2, 2021
Citations: 9	License type: CC BY 4.0

Affiliation: Chung-Ang University, Andong National University

Abstract

It is important to detect signals of abrupt changes in adverse event reporting in order to notice public safety concerns and take prompt action, especially for vaccines under national immunization programs. In this study, we assessed the applicability of change point analysis (CPA) for signal detection in vaccine safety surveillance. The performances of three CPA methods, namely Bayesian change point analysis, Taylor’s change point analysis (Taylor-CPA), and environmental time series change point detection (EnvCpt), were assessed via simulated data with assumptions for the baseline number of events and degrees of change. The analysis was validated using the Korea Adverse Event Reporting System (KAERS) database. In the simulation study, the Taylor-CPA method exhibited better results for the detection of a change point (accuracy of 96% to 100%, sensitivity of 7% to 100%, specificity of 98% to 100%, positive predictive value of 25% to 85%, negative predictive value of 96% to 100%, and balanced accuracy of 53% to 100%) than the other two CPA methods. When the CPA methods were applied to reports of syncope or dizziness following human papillomavirus (HPV) immunization in the KAERS database, Taylor-CPA and EnvCpt detected a change point (Q2/2013), which was consistent with actual public safety concerns. CPA can be applied as an efficient tool for the early detection of vaccine safety signals.

Highlights

Vaccines are generally administered to healthy individuals and a high standard of safety is expected for vaccines
We considered Taylor’s change point analysis (TaylorCPA), a method based on the nonparametric approach; environmental time series change point detection (EnvCpt) [21], based on the frequentist method; and the Bayesian change point (BCP) method [22].We first generated simulated data based on the framework of the real data reported to the Korea Adverse Event Reporting System (KAERS) in order to assess the performance of CPA
For all baselines with 1.5- and 3-fold increases, the three methods performed with accuracies from 95% to 100%, balanced accuracies from 50% to 100%, and positive predictive value (PPV) and negative predictive value (NPV) from 8% to 100% and 96% to 100%, respectively (Table 1, Table S3)

Summary

Introduction

Vaccines are generally administered to healthy individuals and a high standard of safety is expected for vaccines. It is essential to maintain public confidence through post-licensure vaccine safety monitoring, as clinical trials might not have a large enough sample size to detect rare adverse events (AEs), which may occur in large post-licensure populations [1,2,3]. An additional feature of post-licensure vaccine safety monitoring is that it requires timely assessment in order to help distinguish true vaccine adverse reactions from coincidental unrelated events, as potential health risks associated with vaccines are drawing increasing public attention [1]. Because disproportionality analysis is an approach that analyzes AEs for a vaccine of interest in comparison to the same event for all other vaccines, the method may not be sufficient for detecting abrupt increases in safety reports. To enable early response, including epidemic investigations, causality assessment, and appropriate decision-making, early detection of a cluster of AEs following vaccination is important [8]

Objectives

Methods

Results

Conclusion