National Air Quality Forecast Capability Research Articles

Evaluation of modeled surface ozone biases as a function of cloud cover fraction

Abstract. A regional air-quality forecast system's model of surface ozone variability based on cloud coverage is evaluated using satellite-observed cloud fraction (CF) information and a surface air-quality monitoring system. We compared CF and daily maximum ozone from the National Oceanic and Atmospheric Administration's National Air Quality Forecast Capability (NOAA NAQFC) with CFs from the Moderate Resolution Imaging Spectroradiometer (MODIS) and the US Environmental Protection Agency's AirNow surface ozone measurements during May to October 2014. We found that observed surface ozone shows a negative correlation with the MODIS CFs, showing around 1 ppb decrease for 10 % MODIS CF change over the contiguous United States, while the correlation of modeled surface ozone with the model CFs is much weaker, showing only −0.5 ppb per 10 % NAQFC CF change. Further, daytime CF differences between MODIS and NAQFC are correlated with modeled surface-ozone biases between AirNow and NAQFC, showing −1.05 ppb per 10 % CF change, implying that spatial and temporal misplacement of the modeled cloud field might have biased modeled surface ozone level. Current NAQFC cloud fields seem to have fewer CFs compared to MODIS cloud fields (mean NAQFC CF = 0.38 and mean MODIS CF = 0.55), contributing up to 35 % of surface-ozone bias in the current NAQFC system.

Long-term NOx trends over large cities in the United States during the great recession: Comparison of satellite retrievals, ground observations, and emission inventories

National emission inventories (NEIs) take years to assemble, but they can become outdated quickly, especially for time-sensitive applications such as air quality forecasting. This study compares multi-year NOx trends derived from satellite and ground observations and uses these data to evaluate the updates of NOx emission data by the US National Air Quality Forecast Capability (NAQFC) for next-day ozone prediction during the 2008 Global Economic Recession. Over the eight large US cities examined here, both the Ozone Monitoring Instrument (OMI) and the Air Quality System (AQS) detect substantial downward trends from 2005 to 2012, with a seven-year total of −35% according to OMI and −38% according to AQS. The NOx emission projection adopted by NAQFC tends to be in the right direction, but at a slower reduction rate (−25% from 2005 to 2012), due likely to the unaccounted effects of the 2008 economic recession. Both OMI and AQS datasets display distinct emission reduction rates before, during, and after the 2008 global recession in some cities, but the detailed changing rates are not consistent across the OMI and AQS data. Our findings demonstrate the feasibility of using space and ground observations to evaluate major updates of emission inventories objectively. The combination of satellite, ground observations, and in-situ measurements (such as emission monitoring in power plants) is likely to provide more reliable estimates of NOx emission and its trend, which is an issue of increasing importance as many urban areas in the US are transitioning to NOx-sensitive chemical regimes by continuous emission reductions.

Evaluation of the United States National Air Quality Forecast Capability experimental real-time predictions in 2010 using Air Quality System ozone and NO<sub>2</sub> measurements

Abstract. The National Air Quality Forecast Capability (NAQFC) project provides the US with operational and experimental real-time ozone predictions using two different versions of the three-dimensional Community Multi-scale Air Quality (CMAQ) modeling system. Routine evaluation using near-real-time AIRNow ozone measurements through 2011 showed better performance of the operational ozone predictions. In this work, quality-controlled and -assured Air Quality System (AQS) ozone and nitrogen dioxide (NO2) observations are used to evaluate the experimental predictions in 2010. It is found that both ozone and NO2 are overestimated over the contiguous US (CONUS), with annual biases of +5.6 and +5.1 ppbv, respectively. The annual root mean square errors (RMSEs) are 15.4 ppbv for ozone and 13.4 ppbv for NO2. For both species the overpredictions are most pronounced in the summer. The locations of the AQS monitoring sites are also utilized to stratify comparisons by the degree of urbanization. Comparisons for six predefined US regions show the highest annual biases for ozone predictions in Southeast (+10.5 ppbv) and for NO2 in the Lower Middle (+8.1 ppbv) and Pacific Coast (+7.1 ppbv) regions. The spatial distributions of the NO2 biases in August show distinctively high values in the Los Angeles, Houston, and New Orleans areas. In addition to the standard statistics metrics, daily maximum eight-hour ozone categorical statistics are calculated using the current US ambient air quality standard (75 ppbv) and another lower threshold (70 ppbv). Using the 75 ppbv standard, the hit rate and proportion of correct over CONUS for the entire year are 0.64 and 0.96, respectively. Summertime biases show distinctive weekly patterns for ozone and NO2. Diurnal comparisons show that ozone overestimation is most severe in the morning, from 07:00 to 10:00 local time. For NO2, the morning predictions agree with the AQS observations reasonably well, but nighttime concentrations are overpredicted by around 100%.

Ensemble statistical post-processing of the National Air Quality Forecast Capability: Enhancing ozone forecasts in Baltimore, Maryland

An ensemble statistical post-processor (ESP) is developed for the National Air Quality Forecast Capability (NAQFC) to address the unique challenges of forecasting surface ozone in Baltimore, MD. Air quality and meteorological data were collected from the eight monitors that constitute the Baltimore forecast region. These data were used to build the ESP using a moving-block bootstrap, regression tree models, and extreme-value theory. The ESP was evaluated using a 10-fold cross-validation to avoid evaluation with the same data used in the development process. Results indicate that the ESP is conditionally biased, likely due to slight overfitting while training the regression tree models. When viewed from the perspective of a decision-maker, the ESP provides a wealth of additional information previously not available through the NAQFC alone. The user is provided the freedom to tailor the forecast to the decision at hand by using decision-specific probability thresholds that define a forecast for an ozone exceedance. Taking advantage of the ESP, the user not only receives an increase in value over the NAQFC, but also receives value for costly decisions that the NAQFC couldn't provide alone.

Evaluation of NAQFC model performance in forecasting surface ozone during the 2011 DISCOVER-AQ campaign.

The National Air Quality Forecast Capability (NAQFC) and an experimental version of the NAQFC (NAQFC-β) provided flight decision support during the July 2011 NASA DISCOVER-AQ field campaign around Baltimore, Maryland. Ozone forecasts from the NAQFC and NAQFC-β were compared to surface observations at six air quality monitoring stations in the DISCOVER-AQ domain. A bootstrap algorithm was used to test for significant bias and error in the forecasts from each model. Both models produce significant positively biased forecasts in the morning while generally becoming insignificantly biased in the afternoon during peak ozone hours. The NAQFC-β produces higher forecast bias, higher forecast error, and lower correlations than the NAQFC. Forecasts from the two models were also compared to each other to determine the spatial and temporal extent of significant differences in forecasted ozone using a bootstrap algorithm. The NAQFC-β tends to produce an average background ozone mixing ratio of at least 3.51 ppbv greater than the NAQFC throughout the domain at 95 % significance. The difference between the two models is significant during the overnight and early morning hours likely due to the way the Carbon Bond 5 mechanism in the NAQFC-β handles reactive nitrogen recycling and organic peroxide species. The value of information each model provides was tested using a static cost-loss ratio model. By standard measures of forecast skill, the NAQFC generally outperforms the NAQFC-β; however, the NAQFC-β provides greater value of information. This is because standard measures of forecast skill often hide the sensitivity of end users’ needs to forecast error.

Identifying the causes of differences in ozone production from the CB05 and CBMIV chemical mechanisms

Abstract. An investigation was conducted to identify the mechanistic differences between two versions of the carbon bond gas-phase chemical mechanism (CB05 and CBMIV) which consistently lead to larger ground-level ozone concentrations being produced in the CB05 version of the National Air Quality Forecasting Capability (NAQFC) modeling system even though the two parallel forecast systems utilize the same meteorology and base emissions and similar initial and boundary conditions. Box models of each of the mechanisms as they are implemented in the NAQFC were created and a set of 12 sensitivity simulations was designed. The sensitivity simulations independently probed the conceptual mechanistic differences between CB05 and CBMIV and were exercised over a 45-scenario simulation suite designed to emulate the wide range of chemical regimes encountered in a continental-scale atmospheric chemistry model. Results of the sensitivity simulations indicate that two sets of reactions that were included in the CB05 mechanism, but which were absent from the CBMIV mechanism, are the primary causes of the greater ozone production in the CB05 version of the NAQFC. One set of reactions recycles the higher organic peroxide species of CB05 (ROOH), resulting in additional photochemically reactive products that act to produce additional ozone in some chemical regimes. The other set of reactions recycles reactive nitrogen from less reactive forms back to NO2, increasing the effective NOx concentration of the system. In particular, the organic nitrate species (NTR), which was a terminal product for reactive nitrogen in the CBMIV mechanism, acts as a reservoir species in CB05 to redistribute NOx from major source areas to potentially NOx-sensitive areas where additional ozone may be produced in areas remote from direct NOx sources.

The Value of Air Quality Forecasting in the Mid-Atlantic Region

Abstract Air quality forecasts produced by the National Air Quality Forecast Capability (NAQFC), human air quality forecasters, and persistence are evaluated for predictive skill and economic value when used to inform decisions regarding pollutant emission and exposure. Surface ozone forecasts and observations were collected from 40 monitors representing eight forecast regions throughout Washington, D.C.; Virginia; and Maryland over the 2005–09 ozone seasons (April–October). The skill of the forecasts are quantified using discrete statistics, such as correlation, mean bias, and root-mean-square error, and categorical statistics, such as exceedance hit rate, false alarm rate, and critical success index. The value of the forecasts are quantified using a decision model based on costs to protect the public against a poor air quality event and the losses incurred if no protective measures are taken. The results indicate that the most skillful forecast method is not necessarily the most valuable forecast method. Air shed managers need to consider multiple forecast methods when deciding on multiple protective measures, because a single measure of forecast skill can often hide the user’s sensitivity to forecast error for a specific decision.

Assessment of bias-adjusted PM<sub>2.5</sub> air quality forecasts over the continental United States during 2007

Abstract. To develop fine particulate matter (PM2.5) air quality forecasts for the US, a National Air Quality Forecast Capability (NAQFC) system, which linked NOAA's North American Mesoscale (NAM) meteorological model with EPA's Community Multiscale Air Quality (CMAQ) model, was deployed in the developmental mode over the continental United States during 2007. This study investigates the operational use of a bias-adjustment technique called the Kalman Filter Predictor approach for improving the accuracy of the PM2.5 forecasts at monitoring locations. The Kalman Filter Predictor bias-adjustment technique is a recursive algorithm designed to optimally estimate bias-adjustment terms using the information extracted from previous measurements and forecasts. The bias-adjustment technique is found to improve PM2.5 forecasts (i.e. reduced errors and increased correlation coefficients) for the entire year at almost all locations. The NAQFC tends to overestimate PM2.5 during the cool season and underestimate during the warm season in the eastern part of the continental US domain, but the opposite is true for the Pacific Coast. In the Rocky Mountain region, the NAQFC system overestimates PM2.5 for the whole year. The bias-adjusted forecasts can quickly (after 2–3 days' lag) adjust to reflect the transition from one regime to the other. The modest computational requirements and systematic improvements in forecast outputs across all seasons suggest that this technique can be easily adapted to perform bias adjustment for real-time PM2.5 air quality forecasts.

Using National Air Quality Forecast Guidance to Develop Local Air Quality Index Forecasts

The National Air Quality Forecast Capability (NAQFC) currently provides next-day forecasts of ozone concentrations over the contiguous United States. It was developed collaboratively by NOAA and Environmental Protection Agency (EPA) in order to provide state and local agencies, as well as the general public, air quality forecast guidance. As part of the development process, the NAQFC has been evaluated utilizing strict monitor-to-gridcell matching criteria, and discrete-type statistics of forecast concentrations. While such an evaluation is important to the developers, it is equally, if not more important, to evaluate the performance using the same protocol as the model's intended application. Accordingly, the purpose of this article is to demonstrate the efficacy of the NAQFC from the perspective of a local forecaster, thereby promoting its use. Such an approach has required the development of a new evaluation protocol: one that examines the ability of the NAQFC to forecast values of the EPA's Air Qualit...

A performance evaluation of the National Air Quality Forecast Capability for the summer of 2007

This paper provides a performance evaluation of the real-time, CONUS-scale National Air Quality Forecast Capability (NAQFC) that supported, in part, its transition into operational status. This evaluation focuses primarily on discrete forecasts for the maximum 8-h O 3 concentrations covering the 4-month period, June through September, 2007, using measurements obtained from EPA's AIRNow network. Results indicate that the 2007 NAQFC performed as well or better than previous configurations, despite the expansion of the forecast domain into the western half of the nation that is dominated by complex terrain. The mean, domain-wide, season-long correlation was 0.70. When examined over time, the domain-wide correlations exhibit a fairly consistent nature, with values exceeding 0.60 (0.70) over 90% (55%) of the days. The NAQFC systematically over-predicted the 8-h O 3 concentrations, continuing a trend established by earlier NAQFC configurations, though to a lesser degree. The summer-long mean forecast value of 53.2 ppb was 4.2 ppb higher than the observed value, resulting in a domain-wide Normalized Mean Bias (NMB) of 8.7%. Most of the over-prediction is associated with observed concentrations less than 50 ppb. In fact the model tends to under-predict when concentrations exceed 70 ppb. As with the bias, the error associated with the latest configuration was also lower. The summer-long Root Mean Square Error of 13.0 ppb (Normalized Mean Error (NME) = 20.4%) represented marked improvements over earlier forecasts. Examination of the spatial distribution of both the NMB and NME reveals that the NAQFC was generally within 25% for the NME and 25% for the NMB over a majority of the domain. Several areas of poorer performance, where the NMB and NME often exceed 25% and in some cases 50%, were noted. These areas include southern California, where the NAQFC tended to under-predict concentrations (especially on weekends) and the southeast Atlantic and Gulf coasts regions, where the model over-predicted. Subsequent analysis revealed that the incorrect temporal allocation of precursor emissions was likely the source of the under-prediction in southern California, while inaccurate simulation of PBL heights likely contributed to the over-prediction in the coastal regions.

Impact of consistent boundary layer mixing approaches between NAM and CMAQ

Discrepancies in grid structure, dynamics and physics packages in the offline coupled NWS/NCEP NAM meteorological model with the U.S. Environmental Protection Agency Community Multiscale Air Quality (CMAQ) model can give rise to inconsistencies. This study investigates the use of three vertical mixing schemes to drive chemistry tracers in the National Air Quality Forecast Capability (NAQFC). The three schemes evaluated in this study represent various degrees of coupling to improve the commonality in turbulence parameterization between the meteorological and chemistry models. The methods tested include: (1) using NAM predicted TKE-based planetary boundary height, h, as the prime parameter to derive CMAQ vertical diffusivity; (2) using the NAM mixed layer depth to determine h and then proceeding as in (1); and (3) using NAM predicted vertical diffusivity directly to parameterize turbulence mixing within CMAQ. A two week period with elevated surface O3 concentrations during the summer 2006 has been selected to test these schemes in a sensitivity study. The study results are verified and evaluated using the EPA AIRNow monitoring network and other ozonesonde data. The third method is preferred a priori as it represents the tightest coupling option studied in this work for turbulent mixing processes between the meteorological and air quality models. It was found to accurately reproduce the upper bounds of turbulent mixing and provide the best agreement between predicted h and ozonesonde observed relative humidity profile inferred h for sites investigated in this study. However, this did not translate into the best agreement in surface O3 concentrations. Overall verification results during the test period of two weeks in August 2006, did not show superiority of this method over the other 2 methods in all regions of the continental U.S. Further efforts in model improvement for the parameterizations of turbulent mixing and other surface O3 forecast related processes are warranted.

Performance evaluation of NOAA-EPA developmental aerosol forecasts

To aid air quality model development and assess air quality forecasts, the Meteorological Development Laboratory (MDL) provided categorical verification metrics for developmental aerosol predictions. The National Air Quality Forecasting Capability (NAQFC) generated 48 h (of) gridded hourly developmental predictions for the lower 48 states (CONUS) domain in 12 km horizontal spacing. The NAQFC uses the North American Mesoscale (NAM) model with EPA’s Community Multiscale Air Quality (CMAQ) model to produce predictions of ground level aerosol concentrations. We used bilinear interpolation to calculate predicted daily maximum values at the location of the observation sites. We compared these interpolated predicted values to the observed daily maximum to produce 2 × 2 contingency tables, with a threshold of 40 μg/m3 during the months of March–August, 2007. The model showed some degree of skill in predicting aerosol exceedances. These results are preliminary as the NAQFC model for aerosol prediction is in the developmental stage. A more comprehensive performance evaluation will be accomplished in 2008, when more data become available. Our verification metrics included categorical analyses for Fraction Correct (FC) or percent correct (FC × 100), Threat Score (TS) or Critical Success Index (CSI), Probability of Detection (POD), and the False Alarm Rate (FAR), Mean Absolute Error (MAE) and mean algebraic error or bias, where bias is forecast minus observation. Graphic products included weekly statistics for the CONUS displayed in the form of bar charts, scatterplots, and graphs. In addition, we split the CONUS into six geographic regions and provided regional statistics on a monthly basis. MDL produced spatial maps of daily 1-h maximum predicted aerosol values overlaid with the corresponding point observations. MDL also provided spatial maps of the daily maximum of the 24-h running average. We derived the 24-h running average from the 1-h average predicted aerosol values and observations.