Lagrangian Data Assimilation Research Articles

Sequential Markov chain Monte Carlo for Lagrangian data assimilation with applications to unknown data locations

AbstractWe consider a class of high‐dimensional spatial filtering problems, where the spatial locations of observations are unknown and driven by the partially observed hidden signal. This problem is exceptionally challenging, as not only is it high‐dimensional, but the model for the signal yields longer‐range time dependences through the observation locations. Motivated by this model, we revisit a lesser‐known and provably convergent computational methodology from Berzuini et al. (1997, Journal of the American Statistical Association, 92, 1403–1412); Centanniand Minozzo (2006, Journal of the American Statistical Association, 101, 1582–1597); Martin et al. (2013, Annals of the Institute of Statistical Mathematics, 65, 413–437) that uses sequential Markov Chain Monte Carlo (MCMC) chains. We extend this methodology for data filtering problems with unknown observation locations. We benchmark our algorithms on linear Gaussian state‐space models against competing ensemble methods and demonstrate a significant improvement in both execution speed and accuracy. Finally, we implement a realistic case study on a high‐dimensional rotating shallow‐water model (of about – dimensions) with real and synthetic data. The data are provided by the National Oceanic and Atmospheric Administration (NOAA) and contain observations from ocean drifters in a domain of the Atlantic Ocean restricted to the longitude and latitude intervals , , respectively.

Improvements of Lagrangian Data Assimilation Tested in the Gulf of Mexico

Abstract This study extends initial work by Sun and Penny and Sun et al. to explore the inclusion of path information from surface drifters using an augmented-state Lagrangian data assimilation based on the local ensemble transform Kalman filter (LETKF-LaDA) with vertical localization to improve analysis of the ocean. The region of interest is the Gulf of Mexico during the passage of Hurricane Isaac in the summer of 2012. Results from experiments with a regional ocean model at eddy-permitting and eddy-resolving model resolutions are used to quantify improvements to the analysis of sea surface velocity, sea surface temperature, and sea surface height in a data assimilation system. The data assimilation system assimilates surface drifter positions, as well as vertical profiles of temperature and salinity. Data were used from drifters deployed as a part of the Grand Lagrangian Deployment beginning 20 July 2012. Comparison of experiment results shows that at both eddy-permitting and eddy-resolving horizontal resolutions Lagrangian assimilation of drifter positions significantly improves analysis of the ocean state responding to hurricane conditions. These results, which should be applicable to other tropical oceans such as the Bay of Bengal, open new avenues for estimating ocean initial conditions to improve tropical cyclone forecasting.

Uncertainty quantification of nonlinear Lagrangian data assimilation using linear stochastic forecast models

Lagrangian data assimilation exploits the trajectories of moving tracers as observations to recover the underlying flow field. One major challenge in Lagrangian data assimilation is the intrinsic nonlinearity that impedes using exact Bayesian formulae for the state estimation of high-dimensional systems. In this paper, an analytically tractable mathematical framework for continuous-in-time Lagrangian data assimilation is developed. It preserves the nonlinearity in the observational processes while approximating the forecast model of the underlying flow field using linear stochastic models (LSMs). A critical feature of the framework is that closed analytic formulae are available for solving the posterior distribution, which facilitates mathematical analysis and numerical simulations. First, an efficient iterative algorithm is developed in light of the analytically tractable statistics. It accurately estimates the parameters in the LSMs using only a small number of the observed tracer trajectories. Next, the framework facilitates the development of several computationally efficient approximate filters and the quantification of the associated uncertainties. A cheap approximate filter with a diagonal posterior covariance derived from the asymptotic analysis of the posterior estimate is shown to be skillful in recovering incompressible flows. It is also demonstrated that randomly selecting a small number of tracers at each time step as observations can reduce the computational cost while retaining the data assimilation accuracy. Finally, based on a prototype model in geophysics, the framework with LSMs is shown to be skillful in filtering nonlinear turbulent flow fields with strong non-Gaussian features.

A Web Portal For Automatic Performance Evaluation Of Lagrangian Particle Tracking And Data Assimilation Algorithms

We introduce a web portal dedicated to benchmarking Lagrangian Particle Tracking (LPT) and Data Assimilation (DA) processing algorithms. LPT is here understood in a wide sense, as we consider acquisition situations ranging from only two illuminations (i.e. Particle Tracking Velocimetry - PTV) to a large number of images. The website, https://www.onera.fr/en/flow-benchmarks allows not only to download datasets, but also to upload results and automatically obtain their performance evaluation; in its principle, it is thus similar to a number of already existing benchmark websites in the computer vision community, while none existed so far in the community of optical metrology for fluid mechanics. On the basis of the obtained performance, a user has the choice to publish their results or not on the public page of the dataset. The present work presents the website structure, its rules for participation, and a description of the first dataset available at opening, namely the two-pulse (PTV) case of the First LPT and DA Challenge dataset (Sciacchitano et al. 2021).

Main Results Of The Analysis Of The HOMER Lagrangian Particle Tracking And Data Assimilation Database

In this work, we extend the analysis of Lagrangian Particle Tracking (LPT) and Data Assimilation (DA) algorithms, presented at the ISPIV21 conference, to the case of a turbulent wall-bounded flow interacting with a moving panel. The work is conducted within the European project HOMER and the data is processed by the partners of the project consortium. A synthetic experiment is performed that reproduces the air flow around a cylinder in ground effect, whereby a flexible panel is placed undergoing periodic oscillations. The database comprises two different panel's materials, namely metal and rubber, different marker densities on the panel’s surface and different concentrations of tracer particles in the flow. Images of the flow tracers and of the surface markers are acquired either with a four-camera volumetric imaging setup or with a dual setup composed of four LPT cameras and two Digital Image Correlation (DIC) cameras. Additionally, three image acquisition modes are simulated, namely time-resolved, two-pulse and four-pulse. The data are analyzed in terms of percentages of correctly reconstructed, missed and ghost particles, errors of the measured particles’ positions, velocities and accelerations (LPT analysis) and errors of the reconstructed flow fields, panel’s positions and surface pressures (DA analysis).

Impacts of the Lagrangian Data Assimilation of Surface Drifters on Estimating Ocean Circulation during the Gulf of Mexico Grand Lagrangian Deployment

Abstract Satellite-tracked in situ surface drifters, providing measurements of near-surface ocean quantities, have become increasingly prevalent in the global ocean observation system. However, the position data from these instruments are typically not leveraged in operational ocean data assimilation (DA) systems. In this work, the impact of an augmented-state Lagrangian data assimilation (LaDA) method using the local ensemble Kalman transform filter is investigated within a realistic regional ocean DA system. Direct positioning data of surface drifters released by the Consortium for Advanced Research on Transport of Hydrocarbon in the Environment during the summer 2012 Grand Lagrangian Deployment Experiment are assimilated using a Gulf of Mexico (GoM) configuration of the Modular Ocean Model, version 6, of the Geophysical Fluid Dynamics Laboratory. Multiple cases are tested using both 1/4° eddy-permitting and 1/12° eddy-resolving model resolutions: 1) a free running model simulation, 2) a conventional assimilation of temperature and salinity profile observations, 3) an assimilation of profiles and Lagrangian surface drifter positions, and 4) an assimilation of the profiles and derived Eulerian velocities. LaDA generally produces more accurate estimates of all fields compared to the assimilation of derived Eulerian velocities, with estimates of surface currents notably improving, when transitioning to 1/12° model resolution. In particular, LaDA produces the most accurate estimates of sea surface velocities under tropical cyclone conditions when Hurricane Isaac (2012) impacted the GoM. Further experiments applying a vertical localization while assimilating surface drifter positions improve the estimates of temperature and salinity below the mixed layer depth. Cases including the surface drifter positions in the DA show better Lagrangian predictability than the conventional DA.

An efficient and statistically accurate Lagrangian data assimilation algorithm with applications to discrete element sea ice models

Lagrangian data assimilation of complex nonlinear turbulent flows is an important but computationally challenging topic. In this article, an efficient data-driven statistically accurate reduced-order modeling algorithm is developed that significantly accelerates the computational efficiency of Lagrangian data assimilation. The algorithm starts with a Fourier transform of the high-dimensional flow field, which is followed by an effective model reduction that retains only a small subset of the Fourier coefficients corresponding to the energetic modes. Then a linear stochastic model is developed to approximate the nonlinear dynamics of each Fourier coefficient. Effective additive and multiplicative noise processes are incorporated to characterize the modes that exhibit Gaussian and non-Gaussian statistics, respectively. All the parameters in the reduced order system, including the multiplicative noise coefficients, are determined systematically via closed analytic formulae. These linear stochastic models succeed in forecasting the uncertainty and facilitate an extremely rapid data assimilation scheme. The new Lagrangian data assimilation is then applied to observations of sea ice floe trajectories that are driven by atmospheric winds and turbulent ocean currents. It is shown that observing only about 30 non-interacting floes in a 200 km×200 km domain is sufficient to recover the key multi-scale features of the ocean currents. The additional observations of the floe angular displacements are found to be suitable supplements to the center-of-mass positions for improving the data assimilation analysis. In addition, the observed large and small floes are more useful in recovering the large- and small-scale features of the ocean, respectively. The Fourier domain data assimilation also succeeds in recovering the ocean features in the areas where cloud cover obscures the observations. Finally, the multiplicative noise is shown to be crucial in recovering extreme events.

Lagrangian Data Assimilation for Improving Model Estimates of Velocity Fields and Residual Currents in a Tidal Estuary

Numerical models are associated with uncertainties that can be reduced through data assimilation (DA). Lower costs have driven a recent tendency to use Lagrangian instruments such as drifters and floats to obtain information about water bodies. However, difficulties emerge in their assimilation, since Lagrangian data are set out in a moving frame of reference and are not compatible with the fixed grid locations used in models to predict flow variables. We applied a pseudo-Lagrangian approach using OpenDA, an open-source DA tool to assimilate Lagrangian drifter data into an estuarine hydrodynamic model. Despite inherent challenges with using drifter datasets, the work showed that low-cost, low-resolution drifters can provide a relatively higher improvement over the Eulerian dataset due to the larger area coverage of the drifter. We showed that the assimilation of Lagrangian data obtained from GPS-tracked drifters in a tidal channel for a few hours can significantly improve modelled velocity fields (up to 30% herein). A 40% improvement in residual current direction was obtained when assimilating both Lagrangian and Eulerian data. We conclude that the best results are achieved when both Lagrangian and Eulerian datasets are assimilated into the hydrodynamic model.

Lagrangian Data Assimilation and Parameter Estimation of an Idealized Sea Ice Discrete Element Model

AbstractSea ice is a complex media composed of discrete interacting elements of various sizes and thicknesses (floes), and at sufficiently small lengthscales it can not be approximated as a continuous media as routinely done at large scales. While the Eulerian data assimilation is a relatively mature field, techniques for assimilation of satellite‐derived Lagrangian trajectories of sea ice floes remain poorly explored. Here, an idealized discrete element sea ice model is developed and used as a testbed to quantify the efficacy of the minimum approximation for the Lagrangian data assimilation in an one‐way coupled ice‐ocean system. First, it is shown that observations of O(100) floes in a 50 km by 50 km domain are needed to achieve a high data assimilation accuracy, with a large observational timestep of 1 day being sufficient to recover the geophysically balanced part of the unobserved ocean flow, while about a 2‐h timestep is necessary to recover the unbalanced flows. Second, a simple stochastic parameterization is shown to improve the assimilation accuracy when only a small subset of floes is observed or there is a significant model error resulting for example from simplifying the collision laws between floes. Finally, an efficient expectation‐maximization algorithm is developed that succeeds in assimilating the ocean flow and simultaneously estimating individual floe thicknesses and the overall thickness distribution function. Our study implies that the minimum approximation with its closed analytical formulae could potentially provide an efficient data assimilation scheme for satellite observations of sea ice floes.

Assimilation of GPS-tracked drifter data to improve the Eulerian velocity fields in an estuary

Numerical models are invaluable for the provision of real-time and forecasting information that can be used to examine estuarine hydrodynamics, particularly during times of flood or contaminant release. However, model outputs are associated with uncertainty; this necessitates the use of data assimilation (DA) techniques to improve model accuracy. We used an open-source DA tool to effectively assimilate Lagrangian drifter data into an estuarine hydrodynamic model using an ensemble Kalman filter (EnKF) algorithm. Our aims were to (i) evaluate the potential of drifter data for improving the accuracy of model estimates, and (ii) reduce the challenge and programming effort required for assimilation of such datasets, to make this technique accessible, for a broader range of users. We showed that assimilation of Lagrangian data obtained from prompt deployment of drifters in estuaries can lead to significant improvement (here, up to 54%) in modelled velocity fields.

First challenge on Lagrangian Particle Tracking and Data Assimilation: datasets description and planned evolution to an open online benchmark

In the last decade, Lagrangian Particle Tracking (LPT) has emerged as one of the leading measurement techniques for the quantitative determination of fluid flows in three-dimensional domains (see e.g. Schanz et al., 2016), due to its accuracy in reconstructing particles velocities and material accelerations. Due to the scattered nature of the obtained result, at the particles positions only, significant research efforts have also been placed in the development of dedicated Data Assimilation (DA) techniques, aiming at finally reconstructing full 3D velocity and pressure fields on regular Cartesian grids (see, e.g., Schneiders et al. 2016).

Recovering the Eulerian energy spectrum from noisy Lagrangian tracers

We consider the time-sequential state estimation of a flow field given a stream of noisy measurements that are provided by instruments advected by the flow, known as Lagrangian tracers or drifters. Lagrangian drifters collect real-time data as they move through the velocity field and are an important data collection method for atmospheric and oceanic measurements. We quantify the recovery of the Eulerian energy spectra from observations of Lagrangian drifters. This is performed by utilizing special Lagrangian data assimilation algorithms, known as conditionally Gaussian nonlinear filters. We address the following questions: how much of the turbulent Eulerian energy spectra can be recovered from assimilation of Lagrangian trajectory data and how accurately are the various energetic scales recovered relative to the truth. These issues are primarily studied in the perfect model scenario, but we quantify recovery due to model error by reduced order models via spectral truncation of the forecast model. We demonstrate high recovery skill of the two-dimensional turbulent energy spectra for both an exact filter and an imperfect filter, based on extreme localization of the covariance matrix, which is vastly cheaper than the exact filter, for both an inverse cascade spectrum with slope k−5∕3 and a direct cascade spectrum with slope k−3. The dependence of the spectral energy recovery skill on the number of tracers and the spectral truncation grid size is also studied.

Lagrangian Data Assimilation of Surface Drifters in a Double-Gyre Ocean Model Using the Local Ensemble Transform Kalman Filter

Abstract The assimilation of position data from Lagrangian observing platforms is underdeveloped in operational applications because of two main challenges: 1) nonlinear growth of model and observation error in the Lagrangian trajectories, and 2) the high dimensionality of realistic models. In this study, we propose a localized Lagrangian data assimilation (LaDA) method that is based on the local ensemble transform Kalman filter (LETKF). The algorithm is tested with an “identical twin” approach in observing system simulation experiments (OSSEs) using a simple double-gyre configuration of the Geophysical Fluid Dynamics Laboratory (GFDL) Modular Ocean Model. Results from the OSSEs show that with a proper choice of localization radius, the LaDA can outperform conventional assimilation of surface in situ temperature and salinity measurements. The improvements are seen not only in the surface state estimate, but also throughout the ocean column to 1000 m depth. The impacts of localization radius and model error in estimating accuracy of both fluid and drifter states are further investigated.

Eulerian and Lagrangian statistics in an exactly solvable turbulent shear model with a random background mean

We study the Lagrangian statistics of passively advected particles in an elementary velocity model for turbulent shear. The stochastic velocity model is exactly solvable and includes features that highlight the important differences between Lagrangian and Eulerian velocity statistics, which are not equal in the present context. A major element of the velocity model is the presence of a random, spatially uniform background mean, which is superimposed on a turbulent shear with a spectrum that typically follows a power law. We directly solve for the Eulerian and Lagrangian statistics and show how the sweeping motion of the background mean affects the Lagrangian velocity statistics with faster decaying correlations that oscillate more rapidly compared to the Eulerian velocity. This arises due to interaction of the cross-sweeps of the mean flow with the shear component, which determines Lagrangian tracer transport rates. We derive explicit expressions for the tracer dispersion that demonstrate how the dispersion rate depends on model parameters. We validate the predictions with numerical experiments in various test regimes that also highlight the behavior of Lagrangian particles in space. The proposed exactly solvable model serves as a test problem for Eulerian spectral recovery via Lagrangian data assimilation and parameter estimation methods.

A data assimilation framework for data-driven flow models enabled by motion tomography

Autonomous underwater vehicles (AUVs) have become central to data collection for scientific and monitoring missions in the coastal and global oceans. To provide immediate navigational support for AUVs, computational data-driven flow models described as generic environmental models (GEMs) construct a map of the environment around AUVs. This paper proposes a data assimilation framework for the GEM to update the map using data collected by the AUVs. Unlike Eulerian data, Lagrangian data along the AUV trajectory carry time-integrated flow information. To facilitate assimilation of Lagrangian data into the GEM, the motion tomography method is employed to convert Lagrangian data of AUVs into an Eulerian spatial map of a flow field. This process allows assimilation of both Eulerian and Lagrangian data into the GEM to be incorporated in a unified framework, which introduces a nonlinear filtering problem. Considering potential complementarity of Eulerian and Lagrangian data in estimating spatial and temporal characteristics of flow, we develop a filtering method for estimation of the spatial and temporal parameters in the GEM. The observability is analyzed to verify the convergence of our filtering method. The proposed data assimilation framework for the GEM is demonstrated through simulations using two flow fields with different characteristics: (i) a double-gyre flow field and (ii) a flow field constructed by using real ocean surface flow observations from high-frequency radar.

Model parameter estimation using coherent structure colouring

Lagrangian data assimilation is a complex problem in oceanic and atmospheric modelling. Tracking drifters in large-scale geophysical flows can involve uncertainty in drifter location, complex inertial effects and other factors which make comparing them to simulated Lagrangian trajectories from numerical models extremely challenging. Temporal and spatial discretisation, factors necessary in modelling large scale flows, also contribute to separation between real and simulated drifter trajectories. The chaotic advection inherent in these turbulent flows tends to separate even closely spaced tracer particles, making error metrics based solely on drifter displacements unsuitable for estimating model parameters. We propose to instead use error in the coherent structure colouring (CSC) field to assess model skill. The CSC field provides a spatial representation of the underlying coherent patterns in the flow, and we show that it is a more robust metric for assessing model accuracy. Through the use of two test cases, one considering spatial uncertainty in particle initialisation, and one examining the influence of stochastic error along a trajectory and temporal discretisation, we show that error in the coherent structure colouring field can be used to accurately determine single or multiple simultaneously unknown model parameters, whereas a conventional error metric based on error in drifter displacement fails. Because the CSC field enhances the difference in error between correct and incorrect model parameters, error minima in model parameter sweeps become more distinct. The effectiveness and robustness of this method for single and multi-parameter estimation in analytical flows suggest that Lagrangian data assimilation for real oceanic and atmospheric models would benefit from a similar approach.

A coherent structure approach for parameter estimation in Lagrangian Data Assimilation

We introduce a data assimilation method to estimate model parameters with observations of passive tracers by directly assimilating Lagrangian Coherent Structures. Our approach differs from the usual Lagrangian Data Assimilation approach, where parameters are estimated based on tracer trajectories. We employ the Approximate Bayesian Computation (ABC) framework to avoid computing the likelihood function of the coherent structure, which is usually unavailable. We solve the ABC by a Sequential Monte Carlo (SMC) method, and use Principal Component Analysis (PCA) to identify the coherent patterns from tracer trajectory data. Our new method shows remarkably improved results compared to the bootstrap particle filter when the physical model exhibits chaotic advection.

Resolving Temporal Variations in Data-Driven Flow Models Constructed by Motion Tomography

Abstract: Modeling and predicting ocean flow are great challenges in physical oceanography. To answer such challenges, mobile sensing platforms have been an effective tool for providing Lagrangian flow information. Such information is typically incorporated into ocean models using Lagrangian data assimilation which requires significant amount of computing power and time. Motion tomography (MT) constructs generic environmental models (GEMs) that combine computational ocean models with real-time data collected from mobile platforms to provide high-resolution predictions near the mobile platforms. MT employs Lagrangian data from mobile platforms to create a spatial map of flow in the region traversed by the mobile platforms. This paper extends the MT method to resolve the coupling between temporal variations and spatial variations in flow modeling. Along with Lagrangian data from a mobile sensor, Eulerian data are collected from a stationary sensor deployed in the region where the mobile sensor collects data. Assimilation of these two data sets into GEMs introduces a nonlinear filtering problem. This paper presents the formulation of such nonlinear filtering problem and derives a filtering method for estimating flow model parameters. We analyze observability for the derived filters and demonstrate that the resulting method improves navigation accuracy for mobile platforms.

The impact of observing characteristics on the ability to predict ozone under varying polluted photochemical regimes

Abstract. We conduct analyses to assess how characteristics of observations of ozone and its precursors affect air quality forecasting and research. To carry out this investigation, we use a photochemical box model and its adjoint integrated with a Lagrangian 4D-variational data assimilation system. Using this framework in conjunction with pseudo-observations, we perform an ozone precursor source inversion and estimate surface emissions. We then assess the resulting improvement in ozone air quality prediction. We use an analytical model to conduct uncertainty analyses. Using this analytical tool, we address some key questions regarding how the characteristics of observations affect ozone precursor emission inversion and in turn ozone prediction. These questions include what the effect is of choosing which species to observe, of varying amounts of observation noise, of changing the observing frequency and the observation time during the diurnal cycle, and of how these different scenarios interact with different photochemical regimes. In our investigation we use three observed species scenarios: CO and NO2; ozone, CO, and NO2; and HCHO, CO and NO2. The photochemical model was set up to simulate a range of summertime polluted environments spanning NOx-(NO and NO2)-limited to volatile organic compound (VOC)-limited conditions. We find that as the photochemical regime changes, here is a variation in the relative importance of trace gas observations to be able to constrain emission estimates and to improve the subsequent ozone forecasts. For example, adding ozone observations to an NO2 and CO observing system is found to decrease ozone prediction error under NOx- and VOC-limited regimes, and complementing the NO2 and CO system with HCHO observations would improve ozone prediction in the transitional regime and under VOC-limited conditions. We found that scenarios observing ozone and HCHO with a relative observing noise of lower than 33 % were able to achieve ozone prediction errors of lower than 5 ppbv (parts per billion by volume). Further, only observing intervals of 3 h or shorter were able to consistently achieve ozone prediction errors of 5 ppbv or lower across all photochemical regimes. Making observations closer to the prediction period and either in the morning or afternoon rush hour periods made greater improvements for ozone prediction: 0.2–0.3 ppbv for the morning rush hour and from 0.3 to 0.8 ppbv for the afternoon compared to only 0–0.1 ppbv for other times of the day. Finally, we made two complementary analyses that show that our conclusions are insensitive to the assumed diurnal emission cycle and to the choice of which VOC species emission to estimate using our framework. These questions will address how different types of observing platform, e.g. geostationary satellites or ground monitoring networks, could support future air quality research and forecasting.

A Hybrid Particle–Ensemble Kalman Filter for Lagrangian Data Assimilation

Abstract Lagrangian measurements from passive ocean instruments provide a useful source of data for estimating and forecasting the ocean’s state (velocity field, salinity field, etc.). However, trajectories from these instruments are often highly nonlinear, leading to difficulties with widely used data assimilation algorithms such as the ensemble Kalman filter (EnKF). Additionally, the velocity field is often modeled as a high-dimensional variable, which precludes the use of more accurate methods such as the particle filter (PF). Here, a hybrid particle–ensemble Kalman filter is developed that applies the EnKF update to the potentially high-dimensional velocity variables, and the PF update to the relatively low-dimensional, highly nonlinear drifter position variable. This algorithm is tested with twin experiments on the linear shallow water equations. In experiments with infrequent observations, the hybrid filter consistently outperformed the EnKF, both by better capturing the Bayesian posterior and by better tracking the truth.