Estimating the effectiveness of tetanus-diphtheria-acellular pertussis vaccine.

Abstract

To The Editor—We read with interest Koepke et al's analysis [1] of Wisconsin Immunization Registry (WIR) and pertussis surveillance databases to determine effectiveness of tetanus-diphtheria-acellular pertussis (Tdap) vaccines. We are familiar with the challenges involved in using such data sources to evaluate the performance of pertussis vaccines, as we have been collaborating since 2008 in a multiyear study to evaluate the effectiveness of pediatric pertussis vaccines in Wisconsin (clinical trials registration NCT01129362). The authors are to be commended both for their report and for their enumeration therein of many of the limitations of this nonrandomized, nonprospective study that relies on data collected for administrative purposes. Although their analyses controlled for a number of known confounding factors, we are concerned that their brand-specific analyses were unable to control for potent confounding factors, rendering those analyses unreliable. The full study cohort consisted of WIR clients born during 1998–2000; cases during 2012 were identified and rates calculated by year of Tdap vaccination. However, as shown in Figure 1 of their article, the number of Wisconsin organizations contributing data to the WIR rose from <1000 in 2008 to <1400 in 2011, compared with nearly 2000 in 2013. In addition, brand names plus lot numbers were provided for only 53% of those records (for 20% of Tdap vaccinees, no brand information was available). Moreover, because immigrants to Wisconsin were added to the WIR but emigrants from Wisconsin commonly were not deleted, the WIR database cohort was 21% larger than the actual Wisconsin population size. To provide an alternate analytical population that avoided some of these limitations, a Tdap cohort was constructed, consisting of members of the full cohort documented to have received Tdap during 2008–2011. For sensitivity analyses, missing brand information was imputed on the basis of available information under the missing-at-random assumption. Confounding occurs when a third, unmeasured variable is associated both with the risk factor of interest (eg, Tdap brand information) and the outcome (eg, pertussis) [2]. For example, the association between sex and lung cancer is confounded by the fact that men smoke more than women; smoking correlates with both sex (the putative risk factor) and lung cancer (the outcome). Because of the particular circumstances of Wisconsin with respect to both Tdap brand use and WIR participation and accuracy, the attempt to use the WIR to evaluate pertussis incidence by Tdap brand suffers from multiple confounders. Wisconsin is characterized by many small practices, along with a few very large institutions that predominate in a given subregional or cross-regional area; these providers typically use only 1 brand of Tdap. The likelihood of participating in the WIR and the likelihood of accurate data entry varied by institution and thus varied by geographic location. Relative market share of the 2 Tdap brands varied markedly during the study period and between private and public markets. There were issues for both private and public supplies of Tdap during the study period that led to unplanned changes in the Tdap brand used at various practices, increasing the likelihood of misreporting and miscategorization. A partial list of the confounding correlations includes geographic location and Tdap brand use, geographic location and accuracy and completeness of recording, practice and Tdap brand use, practice and likelihood or accuracy of WIR reporting, year and accuracy and completeness of WIR reporting, Tdap brand use and year, and likelihood of having left Wisconsin and year (and thus Tdap brand). The authors' analyses controlled for Wisconsin region, but these variations in practice behavior (eg, Tdap choice and WIR participation) did not respect regional boundaries, and many of these other confounders are beyond adequate control on the basis of available WIR data. Why does this matter? Consider that if a large institution used only a single brand of pertussis vaccine and that the large institution also was an early and faithful user of WIR (and we know that both of these suppositions are valid), then the WIR size (the denominator for rate calculations) is overrepresented with reports of use of that brand, thus lowering the apparent relative risk of pertussis for that brand. And this is only one of many such examples. Moreover, it is clear that brand misclassification might indeed be differential and that the missing-at-random assumption, used to impute the missing brand information, was not valid. Many of the considerations raised above do not apply to the primary study result, which describes the overall effectiveness of Tdap vaccine against pertussis. Regarding the brand-specific estimates, however, we strongly agree with the authors that their “VE estimates may be biased or confounded by uncontrolled client-level factors” (pp. 949–950 [1]).