Variational Data Assimilation Algorithm Research Articles

Adjoint-based sensitivity analysis and assimilation of multi-source data for the inference of spatio-temporal parameters in a 2D urban flood hydraulic model

This contribution presents a novel approach for the calibration of distributed parameters in a 2D urban flood hydraulic model. It focuses on the challenging issue of inferring distributed friction parameters from multi-source heterogeneous spatio-temporal observations of their hydraulic signatures in the context of an urban flash flood in a complex street network. A variational data assimilation algorithm is used to infer high-dimensional multi-variate parameters (spatialized friction and inflow discharge time series) using multi-source observations. This method relies on a differentiable 2D shallow water hydraulic model which enables to generate high-resolution sensitivity maps of local gradients and Derivative-based Global Sensitivity Measures (DGSM), enabling to guide adequate definition of parameter spatialization for the data assimilation process. Assimilated data include real local limnigraphic measurements and high-water marks collected after a major flood event, as well as modeled flow velocity used in twin experiments setups. This study is the first to leverage high-water marks with a variational method for the calibration of distributed parameters in an urban flood model. The multi-source data is used to infer inflow hydrographs and distributed friction parameters in setups of varying complexity. In the main setup, the complex structure of the street network, along with the sensitivity maps and hydraulics expertise led to define a model configuration with 45 friction patches. A high-dimensional parameter vector composed of these friction values and an upstream inflow is inferred simultaneously by assimilating real limnigraphic data and high-water marks. This leads to an increase in model fit to observations and satisfying parameter estimates.

A data-assimilative modeling investigation of Gulf Stream variability

An advanced data-assimilative ocean circulation model is used to investigate Gulf Stream (GS) variability during 2017–2018. The modeling system applies a strong-constraint, 4D variational data assimilation algorithm. It assimilates satellite-based sea surface height and sea surface temperature measurements and in situ temperature and salinity profiles. Model skill assessment metrics along with comparisons of GS position and GS's three-dimensional mean kinetic energy with historical observations are applied to validate the data-assimilative model. The resulting time- and space-continuous ocean state estimates are used to diagnose eddy kinetic energy conversion and cross-stream eddy heat and salt fluxes over the two-year study period. The processes leading to kinetic energy conversion are primarily due to GS meanders. Significant inverse energy cascading (EKE→MKE and EKE→EPE) can occur during GS-eddy interactions, particularly during onshore intrusions or offshore meanderings of the GS. Throughout the two-year study period, the cross-stream eddy heat and salt fluxes off Cape Hatteras were predominantly positive (onshore). Both GS offshore meandering (occurring 44% of the time and associated with shelf/slope water export) and GS intrusion (occurring 56% of the time) contribute to onshore heat and salt transport. Improved understanding of these processes and dynamics requires strong integration of an advanced observational infrastructure that combines remote sensing; fixed, mobile, and shore-based observing components; and high-resolution data assimilative models.

Hybrid Neural Network - Variational Data Assimilation algorithm to infer river discharges from SWOT-like data

Hybrid Neural Network - Variational Data Assimilation algorithm to infer river discharges from SWOT-like data

Signatures-and-sensitivity-based multi-criteria variational calibration for distributed hydrological modeling applied to Mediterranean floods

Classical calibration methods in hydrology typically rely on a single cost function computed on long-term streamflow series. Even when hydrological models achieve acceptable scores in NSE and KGE, imbalances can still arise between overall model performance and its ability to simulate flood events, particularly flash floods. Multi-scale signatures, which refer to hydrological signatures computed at different temporal and/or spatial scales, and distributed flood modeling, which accounts for spatial variability in input variables and model parameters, are important concepts in hydrological modeling. In this study, the potential of using multi-scale signatures is explored to enhance multi-criteria calibration methods for spatially distributed flood modeling, which remains considerable challenges. We present a novel signatures and sensitivity-based calibration approach implemented into a variational data assimilation algorithm capable to deal with high dimensional spatially distributed hydrological optimization problems. It is tested on 141 flash flood pronecatchments mostly located in the French Mediterranean region. Our approach involves computing several signatures, including flood event signatures, using an automated flood segmentation algorithm. We select suitable signatures for constraining the model based on their global sensitivity with the input parameters through global signature-based sensitivity analysis (GSSA). We then perform two multi-criteria calibration strategies using the selected signatures, including a single-objective optimization approach, which transforms the multi-criteria problem into a single-objective function, and a multi-objective optimization approach, which uses a simple additive weighting method to select an optimal solution from the Pareto set. Our results show significant improvements in both calibration and temporal validation metrics, especially for flood signatures, demonstrating the robustness and delicacy of our signatures-based calibration framework for enhancing flash flood forecasting systems.

Improving sea surface temperature in a regional ocean model through refined sea surface temperature assimilation

Abstract. Infrared (IR) and passive microwave (PMW) satellite sea surface temperature (SST) retrievals are valuable to assimilate into high-resolution regional ocean forecast models. Still, there are issues related to these SSTs that need to be addressed to achieve improved ocean forecasts. Firstly, satellite SST products tend to be biased. Assimilating SSTs from different providers can thus cause the ocean model to receive inconsistent information. Secondly, while PMW SSTs are valuable for constraining models during cloudy conditions, the spatial resolution of these retrievals is rather coarse. Assimilating PMW SSTs into high-resolution ocean models will spatially smooth the modeled SST and consequently remove finer SST structures. In this study, we implement a bias correction scheme that corrects satellite SSTs before assimilation. We also introduce a special observation operator, called the supermod operator, into the Regional Ocean Modeling System (ROMS) four-dimensional variational data assimilation algorithm. This supermod operator handles the resolution mismatch between the coarse observations and the finer model. We test the bias correction scheme and the supermod operator using a setup of ROMS covering the shelf seas and shelf break off Norway. The results show that the validation statistics in the modeled SST improve if we apply the bias correction scheme. We also find improvements in the validation statistics when we assimilate PMW SSTs in conjunction with the IR SSTs. However, our supermod operator must be activated to avoid smoothing the modeled SST structures on spatial scales smaller than twice the PMW SST footprint. Both the bias correction scheme and the supermod operator are easy to apply, and the supermod operator can easily be adapted for other observation variables.

Impact of spatially-dense in-situ observations on ocean forecasts of mixed layer and thermocline depth

ABSTRACT Ocean models, such as the Navy Coastal Ocean Model (NCOM), rely on ocean observations through data assimilation to predict the evolution of ocean phenomena accurately. The accuracy is highly dependent on the type and quantity of observations available. This work attempts to examine the impact of spatially dense in-situ profile observations on the forecasts of mixed layer depth (MLD) and thermocline depth (TD), using a three-dimensional variational data assimilation algorithm. To do this, a twin data assimilation experiment is used to simulate the impact of float deployments on the forecast of MLD and TD in a specific area of interest (AOI) of the North Atlantic. It is found that during the northern hemisphere summer the model MLD is sensitive to the sea surface temperature (SST) observations rather than the profiling floats. TD is more strongly impacted by the assimilation of float observations, and the experiment with the most floats produces the best results. During the northern hemisphere winter the profiling floats have a greater impact on the model simulation of MLD. Finally, the overall lack of substantial improvement in the winter months for the highest density of profiling floats is determined to be due to the data assimilation methodology itself.

Rapid reconstruction of temperature and salinity fields based on machine learning and the assimilation application

Satellite observations play important roles in ocean operational forecasting systems, however, the direct assimilation of satellite observations cannot provide sufficient constraints on the model underwater structure. This study adopted the indirect assimilation method. First, we created a 3D temperature and salinity reconstruction model that took into account the advantage of the nonlinear regression of the generalized regression neural network with the fruit fly optimization (abbreviated as FOAGRNN). Compared with the reanalysis product and the WOA13 climatology data, the synthetic T/S (temperature and salinity) profiles had sufficient accuracy and could better describe the characteristics of mesoscale eddies. Then, the synthetic T/S profiles were assimilated into the Regional Ocean Model System (ROMS) using the Incremental Strong constraint 4D Variational (I4D-Var) data assimilation algorithm. The quantitative and qualitative analysis results indicated that compared with the direct assimilation of satellite observations, the root mean square errors (RMSEs) of temperature and salinity were reduced by 26.0% and 23.1% respectively by assimilating the synthetic T/S profiles. Furthermore, this method significantly improved the simulation effect of the model underwater structure, especially in the 300 m to 500 m water layer. Compared with the National Marine Data Center’s real-time analysis data, the machine learning-based assimilation system demonstrated a significant advantage in the simulation of underwater salinity structure, while showing a similar performance in the simulation of underwater temperature structure.

Assessment of the data assimilation framework for the Rapid Refresh Forecast System v0.1 and impacts on forecasts of a convective storm case study

Abstract. The Rapid Refresh Forecast System (RRFS) is currently under development and aims to replace the National Centers for Environmental Prediction (NCEP) operational suite of regional- and convective-scale modeling systems in the next upgrade. In order to achieve skillful forecasts comparable to the current operational suite, each component of the RRFS needs to be configured through exhaustive testing and evaluation. The current data assimilation component uses the hybrid three-dimensional ensemble–variational data assimilation (3DEnVar) algorithm in the Gridpoint Statistical Interpolation (GSI) system. In this study, various data assimilation algorithms and configurations in GSI are assessed for their impacts on RRFS analyses and forecasts of a squall line over Oklahoma on 4 May 2020. A domain of 3 km horizontal grid spacing is configured, and hourly update cycles are performed using initial and lateral boundary conditions from the 3 km grid High-Resolution Rapid Refresh (HRRR). Results show that a baseline RRFS run is able to represent the observed convection, although with stronger cells and large location errors. With data assimilation, these errors are reduced, especially in the 4 and 6 h forecasts using 75 % of the ensemble background error covariance (BEC) and 25 % of the static BEC with the supersaturation removal function activated in GSI. Decreasing the vertical ensemble localization radius from 3 layers to 1 layer in the first 10 layers of the hybrid analysis results in overall less skillful forecasts. Convection is greatly improved when using planetary boundary layer pseudo-observations, especially at 4 h forecast, and the bias of the 2 h forecast of temperature is reduced below 800 hPa. Lighter hourly accumulated precipitation is predicted better when using 100 % ensemble BEC in the first 4 h forecast, but heavier hourly accumulated precipitation is better predicted with 75 % ensemble BEC. Our results provide insight into the current capabilities of the RRFS data assimilation system and identify configurations that should be considered as candidates for the first version of RRFS.

Developing a Land Continuous Variable Estimator to Generate Daily Land Products From Landsat Data

Generating spatially and temporally consistent biophysical products at the global scale from Landsat data for monitoring and assessing surface change dynamics remains a challenge. This article presents an inversion framework called Land continuous Variable Estimator (LoVE)–Landsat for estimating a group of spatiotemporal continuous land surface variables with daily temporal resolution from Landsat 5, 7, and 8 top-of-atmosphere (TOA) data. LoVE–Landsat adopts a data assimilation approach originally developed for coarse-resolution satellite data, such as Moderate Resolution Imaging Spectroradiometer (MODIS) and Visible Infrared Imaging Radiometer Suite (VIIRS). Major improvements to the approach include constructing empirical dynamic equations based on MODIS retrievals and other ancillary information, developing an artificial neural networks (ANN)-based emulator of the coupled radiative transfer (RT) model of atmosphere and land surface (vegetation, soil, and snow) as the observation operator, and designing a hybrid four-dimensional variational (4DVar) and ensemble Kalman filter (EnKF) data assimilation algorithm. The approach starts with generating the initial cloud-free regularly distributed (every 16 days) time series of Landsat data. The 4DVar is then used to assimilate clear-sky snow-free Landsat TOA observations over one year into the empirical dynamic evolution models of the land surface variables (e.g., leaf area index—LAI). The EnKF is then used to further adjust the state vector at the actual Landsat acquisition times. After determining a core set of variables (e.g., LAI), other variables, such as broadband albedo, emissivity, and fraction of absorbed photosynthetically active radiation (FAPAR), are calculated by the coupled RT model. Several experimental cases are presented to demonstrate that the proposed LoVE–Landsat framework is effective to estimate daily land surface variables.

Estimating Nearshore Waves by Assimilating Buoy Directional Spectrum Data in SWAN

AbstractThis paper describes a variational data assimilation algorithm based on the Simulating WAves Nearshore (SWAN) wave spectrum model. The approach allows single-point wave spectrum observations to be used to estimate the wave field for a nearshore region under stationary conditions, assuming a spatially uniform incident wave spectrum at the offshore boundary. The assimilated data are in the form of Fourier directional coefficients, the standard output from operational wave buoys, and are used directly by incorporating the relationship between directional spectrum and the Fourier coefficients into the formulation. The algorithm was tested on data from nearshore buoys deployed off the coast of North Carolina in May 2012, and the estimated wave field is compared to both the input data and independent observation data. The results compare favorably to the independent data with overall RMS errors of 10%–20% for significant wave height, about half a second for mean wave period, and as much as three to four SWAN spectral grid cells for mean direction. Overall, the results show that the algorithm can be effectively used to estimate the offshore boundary spectrum and accurately reproduce wave conditions in the domain.

A Constrained Data Assimilation Algorithm Based on GSI Hybrid 3D-EnVar and Its Application

AbstractData assimilation (DA) at mesoscales is important for severe weather forecasts, yet the techniques of data assimilation at this scale remain a challenge. This study introduces dynamical constraints in the Gridpoint Statistical Interpolation (GSI) three-dimensional ensemble variational (3D-EnVar) data assimilation algorithm to enable the use of high-resolution surface observations of precipitation to improve atmospheric analysis at mesoscales. The constraints use the conservations of mass and moisture. Mass constraint suppresses the unphysical high-frequency oscillation, while moisture conservation constrains the atmospheric states to conform with the observed high-resolution precipitation. We show that the constrained data assimilation (CDA) algorithm significantly reduced the spurious residuals of the mass and moisture budgets compared to the original data assimilation (ODA). A case study is presented for a squall line over the Southern Great Plains on 20 May 2011 during Midlatitude Continental Convective Clouds Experiment (MC3E) of the Atmospheric Radiation Measurement (ARM) program by using ODA or CDA analysis as initial condition of forecasts. The state variables, and the location and intensity of the squall line are better simulated in the CDA experiment. Results show how surface observation of precipitation can be used to improve atmospheric analysis through data assimilation by using the dynamical constraints of mass and moisture conservations.

Strong Solvability of a Variational Data Assimilation Problem for the Primitive Equations of Large-Scale Atmosphere and Ocean Dynamics

For the primitive equations of large-scale atmosphere and ocean dynamics, we study the problem of determining by means of a variational data assimilation algorithm initial conditions that generate strong solutions which minimize the distance to a given set of time-distributed observations. We suggest a modification of the adjoint algorithm whose novel elements is to use norms in the variational cost functional that reflects the H^1-regularity of strong solutions of the primitive equations. For such a cost functional, we prove the existence of minima and a first-order adjoint condition for strong solutions that provides the basis for computing these minima. We prove the local convergence of a gradient-based descent algorithm to optimal initial conditions using the second-order adjoint primitive equations. The algorithmic modifications due to the H^1-norms are straightforwardly to implement into a variational algorithm that employs the standard L^2-metrics.

Reconstruction of turbulent flow fields from lidar measurements using large-eddy simulation

Abstract

Multi‐resolution incremental 4D‐Var for WRF: Implementation and application at convective scale

AbstractThe current four‐dimensional variational (4D‐Var) data assimilation (DA) algorithm, available in the community Weather Research and Forecasting (WRF) model's DA (WRFDA) system, can run only at the same resolution as that of the WRF model forecast, which makes it computationally prohibitive for operational applications at convective scale. The Multi‐Resolution Incremental 4D‐Var (MRI‐4DVar) has been developed in this study in order to speed up WRF 4D‐Var through a three‐stage procedure in each outer loop. One key aspect of WRF MRI‐4DVar is the introduction of the inverse control variable transform within WRFDA that allows the proper resolution change between different outer loops for the control variables projected in the vertical empirical orthogonal function (EOF) space. MRI‐4DVar's computational efficiency and forecast performance are demonstrated by applying it to an afternoon thunderstorm event over northern Taiwan with a 2 km model resolution setting. Comparing to the full‐resolution 4D‐Var experiment, two MRI‐4DVar configurations with a speed‐up of 4.5 and 7.5 times performed similarly well in terms of Fractions Skill Score (FSS) of 6‐hr accumulated rainfall forecasts and hourly variation of total rainfall amount, indicating MRI‐4DVar's potential for operational applications at convective scale.

Improving Background Error Covariances in a 3D Ensemble–Variational Data Assimilation System for Regional NWP

Following the recent development of a three-dimensional ensemble–variational (3DEnVar) data assimilation algorithm for the AROME-France NWP system, this paper examines different approaches to reduce the sampling noise in the ensemble-derived background error covariances in this new scheme without modifying the background ensemble generation strategy. We first examine two variants of scale-dependent localization: one method consists of applying different amounts of localization to different ranges of background error covariance spatial scales, while simultaneously assimilating all of the available observations. Another separate approach uses time-lagged forecasts in order to increase the effective ensemble size, up to a factor of 3 here. This approach of time-lagged forecasts is considered both on its own and together with scale-dependent localization. When the background error covariances are derived from the most recent 25-member ensemble forecasts, the results from data assimilation cycles over a 33-day winter period show that avoiding cross covariances between scales in the scale-dependent localization formulation first proposed by Buehner performs better than the more recent formulation of Buehner and Shlyaeva. However, when increasing the effective ensemble size to 75 members with time-lagged forecasts, the two scale-dependent formulations provide similar forecast improvements overall. It is also found that the lagged-members approach outperforms scale-dependent localization on its own. The largest forecast improvements are obtained when combining the two approaches.

Numerical Investigation of the Direct Variational Algorithm of Data Assimilation in the Urban Scenario

The performance of a direct variational data assimilation algorithm with quasi-independent data assimilation at individual steps of the splitting scheme has been studied in a realistic scenario of air pollution assessment in the city of Novosibirsk by monitoring system data. For operation under conditions of a sparse monitoring network, an algorithm with minimization of the spatial derivative of the uncertainty (control) function adjusted to data assimilation is proposed. The use of the spatial derivative minimization increases the smoothness of the uncertainty (control functions) reconstructed, which has a positive effect on the reconstruction quality in the scenario considered.

Numerical study of direct variational data assimilation algorithm in the urban scenario

Numerical study of direct variational data assimilation algorithm in the urban scenario

An inverse method to estimate emission rates based on nonlinear least-squares-based ensemble four-dimensional variational data assimilation with local air concentration measurements

An inverse source estimation method is proposed to reconstruct emission rates using local air concentration sampling data. It involves the nonlinear least squares-based ensemble four-dimensional variational data assimilation (NLS-4DVar) algorithm and a transfer coefficient matrix (TCM) created using FLEXPART, a Lagrangian atmospheric dispersion model. The method was tested by twin experiments and experiments with actual Cs-137 concentrations measured around the Fukushima Daiichi Nuclear Power Plant (FDNPP). Emission rates can be reconstructed sequentially with the progression of a nuclear accident, which is important in the response to a nuclear emergency. With pseudo observations generated continuously, most of the emission rates were estimated accurately, except under conditions when the wind blew off land toward the sea and at extremely slow wind speeds near the FDNPP. Because of the long duration of accidents and variability in meteorological fields, monitoring networks composed of land stations only in a local area are unable to provide enough information to support an emergency response. The errors in the estimation compared to the real observations from the FDNPP nuclear accident stemmed from a shortage of observations, lack of data control, and an inadequate atmospheric dispersion model without improvement and appropriate meteorological data. The proposed method should be developed further to meet the requirements of a nuclear emergency response.

Towards Estimating the Solar Meridional Flow and Predicting the 11-yr Cycle Using Advanced Variational Data Assimilation Techniques

AbstractWe present in this work the development of a solar data assimilation method based on an axisymmetric mean field dynamo model and magnetic surface data. Our mid-term goal is to predict the solar quasi cyclic activity. We focus on the ability of our variational data assimilation algorithm to constrain the deep meridional circulation of the Sun based on solar magnetic observations. Within a given assimilation window, the assimilation procedure minimizes the differences between data and the forecast from the model, by finding an optimal meridional circulation in the convection zone, and an optimal initial magnetic field, via a quasi-Newton algorithm. We demonstrate the capability of the technique to estimate the meridional flow by a closed-loop experiment involving 40 years of synthetic, solar-like data. We show that the method is robust in estimating a (stochastic) time-varying flow fluctuating 30% about the average, and that the horizon of predictability of the method is ~ 1 cycle length.

Regional Ensemble–Variational Data Assimilation Using Global Ensemble Forecasts

Abstract At the National Centers for Environmental Prediction, the global ensemble forecasts from the ensemble Kalman filter scheme in the Global Forecast System are applied in a regional three-dimensional (3D) and a four dimensional (4D) ensemble–variational (EnVar) data assimilation system. The application is a one-way variational method using hybrid static and ensemble error covariances. To enhance impact, three new features have been added to the existing EnVar system in the Gridpoint Statistical Interpolation (GSI). First, the constant coefficients that assign relative weight between the ensemble and static background error are now allowed to vary in the vertical. Second, a new formulation is introduced for the ensemble contribution to the analysis surface pressure. Finally, in order to make use of the information in the ensemble mean that is disregarded in the existing EnVar in GSI, the trajectory correction, a novel approach, is introduced. Relative to the application of a 3D variational data assimilation algorithm, a clear positive impact on 1–3-day forecasts is realized when applying 3DEnVar analyses in the North American Mesoscale Forecast System (NAM). The 3DEnVar DA system was operationally implemented in the NAM Data Assimilation System in August 2014. Application of a 4DEnVar algorithm is shown to further improve forecast accuracy relative to the 3DEnVar. The approach described in this paper effectively combines contributions from both the regional and the global forecast systems to produce the initial conditions for the regional NAM system.