The Methodologic Ozone Effect

Abstract

rhe U.S. Environmental Protection Agency (EPA) has some nerve in the sense of scientific courage. That is a trait not often associated with government agencies, but the risk the EPA took in commissioning the 3 analyses published in this issue of the journall-3 must be recognized and applauded. The implications of this exercise go far beyond the question of the ozone mortality effect, the ostensible focus of these papers. In commissioning this examination in triplicate, the agency was testing not just the ozone mortality hypothesis, but the methods of science itself-methods the EPA and others have used to justify regulations in many areas. If these methods had failed this test, there would have been broad repercussions for the entire field of environmental risk assessment, not to mention the field of evidence synthesis. The results of this exercise will undoubtedly elicit a substantial sigh of relief in many quarters and perhaps an equal degree of consternation. The 3 groups used a wide diversity of methods and assumptions as outlined in Table 1. There were differences in the studies selected, the estimates used, the numbers abstracted, the confounders considered, the models used, the conversion factors applied, the subgroups investigated, and the alternatives explored. These many methodologic approaches could be contrasted and critiqued, but the bottom lines were remarkably consistent, within a fraction of a percent-a 0.8% increase in immediate mortality per 10-ppb increase in average daily ozone over the year, with most or all of this risk concentrated in the warmer months. However, behind the sigh of relief must be some discomfiture. Agreement aside, we are also given a glimpse of that Holy Grail-truth that looks somewhat different than suggested by the meta-analyses. The meta-analyses depended on published single-city analyses, each of which used different kinds of data, different analytic techniques, and different reporting-severely limiting the meta-analysts' ability to control for confound ing effects in a sophisticated and uniform manner. However, 2 of the meta-analyses threw in, as a bonus, a primary analysis of independent, multicity air-quality data, with the National Morbidity and Mortality Air Pollution Study (NMMAPS) representing the mother lode of such information: a longitudinal study of air quality in 95 U.S. cities over a 14-year period. Although the NMMAPS analysis' does not qualitatively contravene the meta analytic results, in that it still shows an ozone hazard, it does point strongly to a smaller effect-less than one third of the risk. Ito et a13 contrasted their meta-analysis with a reanalysis of primary air quality data from 7 U.S. cities. The weather-modeling approach closest to NMMAPS (four-smoother) produced a point estimate approximately 40% lower than their meta-analysis. Both the NMMAPS and 7-city contrasts send a strong message that depending on published, single-estimate, single-site analyses is an invitation to bias. This is not the first time that such bias has been demonstrated,4'5 but it may be the