Variational Bias Correction Research Articles

Data assimilation of GNSS zenith total delays from a Nordic processing centre

Abstract. Atmospheric moisture-related information estimated from Global Navigation Satellite System (GNSS) ground-based receiver stations by the Nordic GNSS Analysis Centre (NGAA) have been used within a state-of-the-art kilometre-scale numerical weather prediction system. Different processing techniques have been implemented to derive the moisture-related GNSS information in the form of zenith total delays (ZTDs) and these are described and compared. In addition full-scale data assimilation and modelling experiments have been carried out to investigate the impact of utilizing moisture-related GNSS data from the NGAA processing centre on a numerical weather prediction (NWP) model initial state and on the ensuing forecast quality. The sensitivity of results to aspects of the data processing, station density, bias-correction and data assimilation have been investigated. Results show benefits to forecast quality when using GNSS ZTD as an additional observation type. The results also show a sensitivity to thinning distance applied for GNSS ZTD observations but not to modifications to the number of predictors used in the variational bias correction applied. In addition, it is demonstrated that the assimilation of GNSS ZTD can benefit from more general data assimilation enhancements and that there is an interaction of GNSS ZTD with other types of observations used in the data assimilation. Future plans include further investigation of optimal thinning distances and application of more advanced data assimilation techniques.

Characterisation of Special Sensor Microwave Water Vapor Profiler (SSM/T-2) radiances using radiative transfer simulations from global atmospheric reanalyses

The near-global and all-sky coverage of satellite observations from microwave humidity sounders operating in the 183GHz band complement radiosonde and aircraft observations and satellite infrared clear-sky observations. The Special Sensor Microwave Water Vapor Profiler (SSM/T-2) of the Defense Meteorological Satellite Program began operations late 1991. It has been followed by several other microwave humidity sounders, continuing today. However, expertise and accrued knowledge regarding the SSM/T-2 data record is limited because it has remained underused for climate applications and reanalyses. In this study, SSM/T-2 radiances are characterised using several global atmospheric reanalyses. The European Centre for Medium-Range Weather Forecasts (ECMWF) Interim Reanalysis (ERA-Interim), the first ECMWF reanalysis of the 20th-century (ERA-20C), and the Japanese 55-year Reanalysis (JRA-55) are projected into SSM/T-2 radiance space using a fast radiative transfer model. The present study confirms earlier indications that the polarisation state of SSM/T-2 antenna is horizontal (not vertical) in the limit of nadir viewing. The study also formulates several recommendations to improve use of the SSM/T-2 measurement data in future fundamental climate data records or reanalyses. Recommendations are (1) to correct geolocation errors, especially for DMSP 14; (2) to blacklist poor quality data identified in the paper; (3) to correct for inter-satellite biases, estimated here on the order of 1K, by applying an inter-satellite recalibration or, for reanalysis, an automated (e.g., variational) bias correction; and (4) to improve precipitating cloud filtering or, for reanalysis, consider an all-sky assimilation scheme where radiative transfer simulations account for the scattering effect of hydrometeors.

Variational Bias Correction of GNSS ZTD in the HARMONIE Modeling System

AbstractTo fill the gap in the observation system for humidity, the HIRLAM–ALADIN Research on Mesoscale Operational NWP in Euromed (HARMONIE) limited-area high-resolution kilometer-scale model has been prepared for assimilation of Global Navigation Satellite System (GNSS) zenith total delay (ZTD) observations. The observation-processing system includes data selection, bias correction, quality control, and a GNSS observation operator for data assimilation. A large part of the bias between observations and model equivalents comes from the relatively low model top used in the HARMONIE experiments. The functionality of the different observation-processing components was investigated in detail as was the overall performance of the GNSS ZTD data assimilation. This paper contains an extensive description of the GNSS ZTD observation-processing system and a comparison of a newly introduced variational bias correction for GNSS ZTD data with an alternative static bias correction, as well as a detailed analysis of the impact of GNSS ZTD data, both in terms of statistical evaluations over a longer period and in terms of individual case studies. Assimilation of the GNSS ZTD observations with a variational bias correction has improved the quality of short-range weather forecasts for the moisture-related parameters in particular, both in a statistical sense and in individual case studies. The paper also discusses further improvements in the HARMONIE variational data-assimilation system that are needed to fully utilize the potential of high-resolution GNSS ZTD observations.

Predictors and grouping for bias correction of radiosonde temperature observations

Due to various causes, artificial biases can be found in meteorological observations. The use of biased observations in the assimilation process leads to systematic errors in the resulting analysis unless special measures are taken during the data assimilation. One such method that avoids the assumption of unbiased input observations is the variational bias correction (VarBC), which is used successfully within the (European Centre for Medium-range Weather Forecasts) operational system, mainly to deal with satellite radiance data. In VarBC the bias of the given observation is estimated using a linear predictor model based on a small number of predefined predictors and the corresponding unknown bias parameters. These are estimated together with the model state by including a bias term in the cost function of the variational analysis. The radiosonde temperature biases depend mainly on pressure, on solar elevation, and on the instrumentation used. The optimal choice of the grouping of radiosonde stations (to get larger samples) and of the bias models is not obvious. While the method should be used in a 4D-VAR setting, its properties can be estimated off-line with much less computational effort. In this paper different methods for the grouping and the bias model are investigated, both using and not using metadata. At the same time the statistics are compared with the output of two independent homogeneity adjustment algorithms. The major outcome of this work is, apart from the development of predictors model suitable for VarBC, the detection of the high variability in the bias using grouping based on metadata.

Impact of the assimilation of INSAT‐3D radiances on short‐range weather forecasts

The present study concentrates on the assimilation of clear‐sky radiances from the recently launched INSAT‐3D satellite. The imager and sounder are two primary meteorological instruments aboard the INSAT‐3D satellite. Pre‐assimilation monitoring of the radiances has been carried out from April to July 2014. A double‐difference technique using the High Resolution Infrared Sounder (HIRS) on MetOp‐A is employed to remove the effect of deficiencies in the model‐analysed profiles which can contribute to the biases. Observed radiances of both instruments appear to be colder (∼1.5–2 K) with respect to radiative transfer model (RTM) simulated radiances, except water vapour channel radiances, which are warmer (0.5–1 K). The standard deviations of observed minus RTM‐simulated radiances are between 0.5 and 1.5 K. Prior to data assimilation, these biases are corrected using a variational bias correction scheme. The largest impacts on the analyses, when assimilating imager radiances, are found in the mid‐ and upper‐tropospheric moisture, while assimilation of sounder radiances impacted both moisture and temperature throughout the troposphere. The more accurate analyses with the INSAT‐3D radiance assimilation lead to improved moisture, wind, temperature and precipitation forecasts compared to the control case in which only conventional observations were assimilated. The results demonstrate the ability of temperature‐ and water vapour‐sensitive radiances to improve not only the temperature and moisture fields, but also the wind fields, enhancing their importance, particularly over tropical regions where wind observations are more important.

Satellite Radiance Assimilation in the JMA Operational Mesoscale 4DVAR System

Abstract The direct radiance assimilation scheme used in the Japan Meteorological Agency (JMA) global analysis system is applied to the JMA mesoscale four-dimensional variational data assimilation (4DVAR) system with two modifications. First, the data-thinning distance is shortened, and, second, the atmospheric profiles are extrapolated from the mesoscale model top to the radiative transfer model top using the U.S. Standard Atmosphere lapse rate. Although the variational bias correction method is widely used in many numerical weather prediction centers for global radiance assimilations, a radiance bias correction method for regional models has not been established because of difficulties in estimating the biases within limited regions and times. This paper examined the use of the bias correction coefficients estimated in the global system for the mesoscale system when the radiance data were introduced instead of the retrievals. It was found that the profile extrapolation was necessary to reduce the biases. Moreover, the use of common bias coefficients enables the use of the radiance data in the same way as the global system. The radiance data assimilation experiments in the mesoscale system demonstrated considerable improvements to the tropospheric geopotential height forecasts and precipitation forecasts. The improvements resulted from the introduction of radiance data from multiple satellites into data-sparse regions and times. However, the major effect of the radiance assimilation on the precipitation forecasts was limited to weak precipitation areas over oceans; the effects on deep convective areas and over land were relatively small.

Surface pressure information retrieved from GPS radio occultation measurements

AbstractNumerical weather prediction (NWP) experiments have been performed to investigate the impact of surface pressure information retrieved from Global Positioning System radio occultation (GPSRO) measurements. The GPSRO measurements are able to reduce the impact of removing all conventional surface pressure observations from the NWP system. The GPSRO measurements can constrain global mean surface pressure errors in analyses and short‐range forecasts to the −1 to −1.5 hPa level over a 3‐month period, whereas they can be as large as −4 hPa when both GPSRO observations and conventional surface pressure measurements are removed from the NWP system. The large mean errors that arise when both GPSRO and conventional surface pressure measurements are removed are caused by an interaction between the assimilation of the conventional upper‐air measurements and the bias correction of radiances with the variational bias correction system (VARBC). In the absence of GPSRO, the VARBC amplifies and spreads errors that are introduced by the conventional upper‐air data. However, the GPSRO measurements cannot fully compensate for the loss of the conventional data, particularly in the Northern Hemisphere. It is also shown that the surface pressure results obtained when GPSRO are assimilated are sensitive to biases introduced by other observations, such as aircraft temperature measurements, because these biases can be aliased into surface pressure increments. Experiments designed to compare the amount of information provided by other parts of the global observing system highlight the complementary nature of GPSRO and scatterometer wind retrievals, and show that reasonable surface pressure analyses, with an uncertainty of ∼ 1 hPa, can now be obtained by assimilating just satellite data, without using any conventional data to ‘anchor’ the NWP system. However, the full observing system, including both satellite and conventional observations, remains superior to assimilating just satellite observations.

Challenges for data assimilation – from convective‐scale to climate

Challenges for data assimilation – from convective‐scale to climate

Integrating NWP Forecasts and Observation Data to Improve Nowcasting Accuracy

Abstract This study addresses the issue of improving nowcasting accuracy by integrating several numerical weather prediction (NWP) model forecasts with observation data. To derive the best algorithms for generating integrated forecasts, different integration methods were applied starting with integrating the NWP models using equal weighting. Various refinements are then successively applied including dynamic weighting, variational bias correction, adjusted dynamic weighting, and constraints using current observation data. Three NWP models—the Canadian Global Environmental Multiscale (GEM) regional model, the GEM Limited Area Model (LAM), and the American Rapid Update Cycle (RUC) model—are used to generate the integrated forecasts. Verification is performed at two Canadian airport locations [Toronto International Airport (CYYZ), in Ontario, and Vancouver International Airport (CYVR), in British Columbia] over the winter and summer seasons. The results from the verification for four weather variables (temperature, relative humidity, and wind speed and gust) clearly show that the integrated models with new refinements almost always perform better than each of the NWP models individually and collectively. When the integrated model with innovative dynamic weighting and variational bias correction is further updated with the most current observation data, its performance is the best among all models, for all the selected variables regardless of location and season. The results of this study justify the use of integrated NWP forecasts for nowcasting provided they are properly integrated using appropriate and specifically designed rules and algorithms.

The use of variable CO2 in the data assimilation of AIRS and IASI radiances

AbstractAn important component of the assimilation of radiance observations from the AIRS and IASI satellite instruments is the radiative transfer modelling. Currently, the RTTOV model used in the ECMWF IFS system uses a fixed value for CO2. Neglecting the spatio‐temporal variability of CO2 introduces an error in the simulation of the satellite radiances, which could affect the quality of the analyses and forecasts. The current assumption is that variational bias correction corrects most of this error and therefore minimizes the impact on the forecast scores. This paper investigates the possibility of modelling CO2 within the IFS to improve the radiative transfer modelling. Results show that the required bias correction is significantly reduced when using more realistic CO2 values. The impact on the analysis quality and forecast scores is mostly neutral with some indication of improvement in the Tropics and the stratosphere.

An evaluation of FY‐3A satellite data for numerical weather prediction

AbstractFY‐3A, launched in May 2008, is the first in a series of seven polar‐orbiting meteorological satellites due to be launched by China's Meteorological Administration in the period leading up to 2020. The FY‐3A payload includes four instruments of particular interest for numerical weather prediction (NWP): microwave temperature and humidity sounders, a microwave imager, and an infrared sounder. The main features of these instruments are described. Data from the calibration–validation phase of the FY‐3A mission were introduced into the ECMWF Integrated Forecasting System in order to assess the data quality and the influence of the data on analyses and forecasts. An analysis of first‐guess departures has shown the data to be of good quality overall. Several issues with instrument performance and ground segment processing have been identified. The most serious of these are: uncertainties in the temperature sounder passbands on‐orbit, orbital biases in the infrared instrument affecting the highest peaking channels, and scan biases in the microwave humidity sounder. Variational bias correction partially corrects for these errors, but more work remains to be done to correct the problems before the full benefit of the data is realised. In observing system experiments, the FY‐3A instruments, both individually and as a package, show considerable skill when added to observation depleted control experiments. When added to a full observing system, the impacts are neutral to slightly positive, as expected. These initial results are encouraging and build confidence that the following series of FY‐3 instruments will be widely used in NWP data assimilation systems. Copyright © 2011 Royal Meteorological Society

The GRAPES variational bias correction scheme and associated preliminary experiments

The variational assimilation theory is generally based on unbiased observations. In practice, however, almost all observations suffer from biases arising from observational instruments, radiative transfer operator, precondition of data, and so on. Therefore, a bias correction scheme is indispensable. The current scheme for radiance bias correction in the GRAPES 3DVar system is an offline scheme. It is actually a static correction for the radiance bias before the process of cost function minimization. In consideration of its effects on forecast results, this kind of scheme has some shortcomings. Thus, this study provides a variational bias correction (VarBC) scheme for the GRAPES 3DVar system following Dee’s idea. In the VarBC scheme, the observation operator is modified and a new control variable is defined by taking the predictor coefficients as the control parameters. According to the feature of the GRAPES-3DVAR, an incremental formulation is applied and the original bias correction scheme is maintained in the actual process of observations. The VarBC is designed to co-exist with the original scheme, because it is a dynamic revision to the observational operator on the basis of the old method, i.e., it adjusts the model state vector along with the control parameters to an unbiased state in the process of minimization and the assimilation system remains consistent with available information automatically. Preliminary experimental results show that the mean departures of background-minus-observation and analysis-minus-observation are reduced as expected. In a case study of the heavy rainfall that happened in South China on 11–13 June 2008, the 500-hPa geopotential height is better simulated using the analyzed field from the VarBC as the initial condition.

Assimilation of Global Positioning System radio occultation data in the ECMWF ERA–Interim reanalysis

AbstractThis article presents results of the assimilation of Global Positioning System (GPS) radio occultation (RO) bending‐angle data from CHAMP, FORMOSAT‐3/COSMIC and MetOp‐A GRAS in the European Centre for Medium‐range Weather Forecasts (ECMWF) global reanalysis ERA–Interim. We find that GPSRO data present the highest daily assimilation percentage rate among all the various types of observations, suggesting that these data are readily usable by today's reanalysis systems. Over time, except when additional GPSRO data are introduced, the ERA–Interim short‐term forecasts (background) are found to be stable compared with GPSRO data, and so are the ERA–Interim temperatures compared with radiosondes. This suggests that the GPSRO data are potentially as stable as the verification used here, at least to the extent that the possible quality variations in GPSRO data assimilated in ERA–Interim appear invisible when compared with radiosondes. We observe very good consistency between data from all six COSMIC receivers. Small differences between COSMIC and CHAMP (GRAS) data are observed in the lower troposphere (stratosphere). The mean effect of adding GPSRO data in ERA–Interim is to reduce temperature biases with respect to radiosondes in the ERA–Interim background by warming the tropopause region and lower stratosphere by about 0.1–0.2 K in all hemispheres. We also find hints of a drying effect in the mean water‐vapour content in the Tropics in ERA–Interim when GPSRO data are introduced. The fit to upper tropospheric wind observations in the Southern Hemisphere and the Tropics is also improved when GPSRO data are present. Overall the GPSRO data act as references via variational bias correction to correct satellite radiances. Removing the GPSRO data from the ERA–Interim system leaves the latter more prone to fitting the warmly biased aircraft data. All these effects mostly became apparent when large amounts of GPSRO data started being assimilated, at the end of 2006, with the introduction of COSMIC data. Copyright © 2010 Royal Meteorological Society

Ensemble Kalman Filter and 4D-Var Intercomparison with the Japanese Operational Global Analysis and Prediction System

Abstract The local ensemble transform Kalman filter (LETKF) is implemented and assessed with the experimental operational system at the Japanese Meteorological Agency (JMA). This paper describes the details of the LETKF system and verification of deterministic forecast skill. JMA has been operating a four-dimensional variational data assimilation (4D-Var) system for global numerical weather prediction since 2005. The main purpose of this study is to make a reasonable comparison between the LETKF and the operational 4D-Var. Several forecast–analysis cycle experiments are performed to find sensitivity to the parameters of the LETKF. The difference between additive and multiplicative error covariance inflation schemes is investigated. Moreover, an adaptive bias correction method for satellite radiance observations is proposed and implemented, so that the LETKF is equipped with functionality similar to the variational bias correction used in the operational 4D-Var. Finally, the LETKF is compared with the operational 4D-Var. Although forecast verification scores of the two systems relative to each system’s own analyses and to radiosonde observations show some disagreement, the overall conclusion indicates that the LETKF and 4D-Var have essentially comparable performance. The LETKF shows larger temperature bias in the lower troposphere mainly over the ocean, which is related to a well-known JMA model bias that plays an important role in the significant degradation of the forecast scores in the SH. The LETKF suffers less of a performance degradation than 4D-Var in the absence of satellite radiance assimilation. This suggests that better treatment of satellite radiances would be important in future developments toward operational use of the LETKF. Developing both LETKF and 4D-Var at JMA has shown significant benefits by the synergistic effect and is the recommended strategy for the moment.

Variational bias correction of satellite radiance data in the ERA‐Interim reanalysis

AbstractThis article describes the performance of the variational bias correction system for satellite radiance data in ERA‐Interim, and considers implications for the representation of climate signals in reanalysis. We briefly review the formulation of the method and its ability to automatically develop bias estimates when radiance measurements from newly available satellite sensors are first introduced in the reanalysis. We then present several results obtained from the first 19 years (1989–2007) of ERA‐Interim. These include the identification of Microwave Sounding Unit (MSU) instrument calibration errors, the response of the system to the Pinatubo eruption in 1991, and the detection of a long‐term drift in biases of tropospheric AMSU‐A data. We find that our results support the notion that global reanalysis provides an appropriate framework for climate monitoring. Copyright © 2009 Royal Meteorological Society

4D-Var assimilation of MERIS total column water-vapour retrievals over land

Experiments with the active assimilation of total column water-vapour retrievals from Envisat MERIS observations have been performed at the European Centre for Medium-Range Weather Forecasts (ECMWF), focusing on the summer 2006 African Monsoon Multidisciplinary Analysis (AMMA) field campaign period. A mechanism for data quality control, observation error definition and variational bias correction has been developed so that the data can be safely treated within 4D-Var, like other observations that are currently assimilated in the operational system. While data density is limited due to the restriction to daylight and cloud-free conditions, a systematic impact on mean moisture analysis was found, with distinct regional and seasonal features. The impact can last 1--2 days into the forecast but has little effect on forecast accuracy in terms of both moisture and dynamics. This is mainly explained by the weak dynamic activity in the areas of largest data impact. Analysis and short-range forecast evaluation with radiosonde observations revealed a strong dependence on radiosonde type. Compared with Vaisala RS92 observations, the addition of MERIS total column water-vapour observations produced neutral to positive impact, while contradictory results were obtained when all radiosonde types were used in generating the statistics. This highlights the issue of radiosonde moisture biases and the importance of sonde humidity bias correction in numerical weather prediction (NWP). Copyright © 2009 Royal Meteorological Society

Interaction between bias correction and quality control

AbstractAn interaction between the quality control (QC) and the bias correction of satellite radiances has been identified. If the bias correction is recalibrated intermittently, or if it is adaptive, a feedback process is possible. Indeed, the bias is calculated over a population of quality‐controlled observations. Since QC usually acts upon bias‐corrected observed‐minus‐first‐guess departures, the value of the bias correction influences the next population that passes the QC, and so on.Two situations that can trigger a feedback are described: residual outliers that have not been detected by the QC; and an asymmetric QC that selects a sub‐population of the good dataset. In both cases, the bias correction is strongly influenced by the feedback, and the performance of the QC is also degraded.The use of a new metric for the bias correction (called the ‘pseudo‐mode’), approximating the mode of the radiance‐departure distribution, significantly reduces the feedback due to outliers and asymmetric QC. The variational bias correction scheme VarBC, which updates the bias inside the analysis, also constrains the feedback triggered by an asymmetric QC, but it has limited skill for outliers in a population of infrared window channel observations. A combination of VarBC with the pseudo‐mode benefits from the advantages of both approaches. The bias correction is less sensitive to the QC, and more robust with respect to residual outliers. Copyright © 2007 Royal Meteorological Society

Adaptive bias correction for satellite data in a numerical weather prediction system

AbstractAdaptive bias corrections for satellite radiances need to separate the observation bias from the systematic errors in the background in order to prevent the analysis from drifting towards its own climate. The variational bias correction scheme (VarBC) is a particular adaptive scheme that is embedded inside the assimilation system.VarBC is compared with an offline adaptive and a static bias correction scheme. In simulation, the three schemes are exposed to artificial shifts in the observations and the background. The VarBC scheme can be considered as a good compromise between the static and the offline adaptive schemes. It shows some skill in distinguishing between the background‐error and the observation biases when other unbiased observations are available to anchor the system. Tests of VarBC in a real numerical weather prediction (NWP) environment show a significant reduction in the misfit with radiosonde observations (especially in the stratosphere) due to NWP model error. The scheme adapts to an instrument error with only minimal disruption to the analysis.In VarBC, the bias is constrained by the fit to observations—such as radiosondes—that are not bias‐corrected to the NWP model. In parts of the atmosphere where no radiosonde observations are available, the radiosonde network still imposes an indirect constraint on the system, which can be enhanced by applying a mask to VarBC. Copyright © 2007 Royal Meteorological Society