Traditional Data Assimilation Research Articles

Real-time data assimilation for the thermodynamic modeling of cryogenic storage tanks

The thermal management of cryogenic storage tanks requires advanced control strategies to minimize the boil-off losses produced by heat leakages and sloshing-enhanced heat and mass transfer. This work presents a data-assimilation approach to calibrate a 0D thermodynamic model for cryogenic fuel tanks from data collected in real time from multiple tanks. The model combines energy and mass balance between three control volumes (the ullage vapor, the liquid, and the solid tank) with an Artificial Neural Network (ANN) for predicting the heat transfer coefficients from the current tank state.The proposed approach combines ideas from traditional data assimilation and multi-environment reinforcement learning, where an agent’s training (model assimilation) is carried out simultaneously on multiple environments (systems). The real-time assimilation uses a mini-batch version of the Limited-memory Broyden–Fletcher–Goldfarb–Shanno with bounds (L-BFGS-B) and adjoint-based gradient computation for solving the underlying optimization problem. The approach is tested on synthetic datasets simulating multiple tanks undergoing different operation phases (pressurization, hold, long-term storage, and sloshing). The results show that the assimilation is robust against measurement noise and uses it to explore the parameter space further. Moreover, the work shows that simultaneously sampling from multiple environments accelerates the assimilation.

History matching for geological carbon storage using data-space inversion with spatio-temporal data parameterization

History matching based on monitoring data will enable uncertainty reduction, and thus improved aquifer management, in industrial-scale carbon storage operations. In traditional model-based data assimilation, geomodel parameters are modified to force agreement between flow simulation results and observations. In data-space inversion (DSI), history-matched quantities of interest, e.g., posterior pressure and saturation fields conditioned to observations, are inferred directly, without constructing posterior geomodels. This is accomplished efficiently using a set of O(1000) prior simulation results, data parameterization, and posterior sampling within a Bayesian setting. In this study, we develop and implement (in DSI) a deep-learning-based parameterization to represent spatio-temporal pressure and CO2 saturation fields at a set of time steps. The new parameterization uses an adversarial autoencoder (AAE) for dimension reduction and a convolutional long short-term memory (convLSTM) network to represent the spatial distribution and temporal evolution of the pressure and saturation fields. This parameterization is used with an ensemble smoother with multiple data assimilation (ESMDA) in the DSI framework to enable posterior predictions. A realistic 3D system characterized by prior geological realizations drawn from a range of geological scenarios is considered. A local grid refinement procedure is introduced to estimate the error covariance term that appears in the history matching formulation. Extensive history matching results are presented for various quantities, for multiple synthetic true models, using the new DSI framework. Substantial uncertainty reduction in posterior pressure and saturation fields is achieved in all cases. The framework is also applied to efficiently provide posterior predictions for a range of error covariance specifications. Such an assessment would be very expensive using a traditional model-based approach.

A Four‐Dimensional Variational Constrained Neural Network‐Based Data Assimilation Method

AbstractAdvances in data assimilation (DA) methods and the increasing amount of observations have continuously improved the accuracy of initial fields in numerical weather prediction during the last decades. Meanwhile, in order to effectively utilize the rapidly increasing data, Earth scientists must further improve DA methods. Recent studies have introduced machine learning (ML) methods to assist the DA process. In this paper, we explore the potential of a four‐dimensional variational (4DVar) constrained neural network (NN) method for accurate DA. Our NN is trained to approximate the solution of the variational problem, thereby avoiding the need for expensive online optimization when generating the initial fields. In the context that the full‐field system truths are unavailable, our approach embeds the system's kinetic features described by a series of analysis fields into the NN through a 4DVar‐form loss function. Numerical experiments on the Lorenz96 physical model demonstrate that our method can generate better initial fields than most traditional DA methods at a low computational cost, and is robust when assimilating observations with higher error outside of the distributions where it is trained. Furthermore, our NN‐based DA model is effective against Lorenz96 physical models with larger variable numbers. Our approach exemplifies how ML methods can be leveraged to improve both the efficiency and accuracy of DA techniques.

A Machine Learning Augmented Data Assimilation Method for High‐Resolution Observations

AbstractThe accuracy of initial conditions is an important driver of the forecast skill of numerical weather prediction models. Increases in the quantity of available measurements, particularly high‐resolution remote sensing observational data products from satellites, are valuable inputs for improving those initial condition estimates. However, the traditional data assimilation methods for integrating observations into forecast models are computationally expensive. This makes incorporating dense observations into operational forecast systems challenging, and it is often prohibitively time‐consuming. Additionally, high‐resolution observations often have correlated observation errors which are difficult to estimate and create problems for assimilation systems. As a result, large quantities of data are discarded and not used for state initialization. Using the Lorenz‐96 system for testing, we demonstrate that a simple machine learning method can be trained to assimilate high‐resolution data. Using it to do so improves both initial conditions and forecast accuracy. Compared to using the Ensemble Kalman Filter with high‐resolution observations ignored, our augmented method has an average root‐mean‐squared error reduced by 37%. Ensemble forecasts using initial conditions generated by the augmented method are more accurate and reliable at up to 10 days of forecast lead time.

Ensemble optimal interpolation for adjoint-free biogeochemical data assimilation.

Advanced marine ecosystem models can contain more than 100 biogeochemical variables, making data assimilation for these models a challenging prospect. Traditional variational data assimilation techniques like 4dVar rely on tangent linear and adjoint code, which can be difficult to create for complex ecosystem models with more than a few dozen variables. More recent hybrid ensemble-variational data assimilation techniques use ensembles of model forecasts to produce model statistics and can thus avoid the need for tangent linear or adjoint code. We present a new implementation of a four-dimensional ensemble optimal interpolation (4dEnOI) technique for use with coupled physical-ecosystem models. Our 4dEnOI implementation uses a small ensemble, and spatial and variable covariance localization to create reliable flow-dependent statistics. The technique is easy to implement, requires no tangent linear or adjoint code, and is computationally suitable for advanced ecosystem models. We test the 4dEnOI implementation in comparison to a 4dVar technique for a simple marine ecosystem model with 4 biogeochemical variables, coupled to a physical circulation model for the California Current System. In these tests, our 4dEnOI reference implementation performs similarly well to the 4dVar benchmark in lowering the model observation misfit. We show that the 4dEnOI results depend heavily on covariance localization generally, and benefit from variable localization in particular, when it is applied to reduce the coupling strength between the physical and biogeochemical model and the biogeochemical variables. The 4dEnOI results can be further improved by small modifications to the algorithm, such as multiple 4dEnOI iterations, albeit at additional computational cost.

Reconstructing tephra fall deposits via ensemble-based data assimilation techniques

Abstract. In recent years, there has been a growing interest in ensemble approaches for modelling the atmospheric transport of volcanic aerosol, ash, and lapilli (tephra). The development of such techniques enables the exploration of novel methods for incorporating real observations into tephra dispersal models. However, traditional data assimilation algorithms, including ensemble Kalman filter (EnKF) methods, can yield suboptimal state estimates for positive-definite variables such as those related to volcanic aerosols and tephra deposits. This study proposes two new ensemble-based data assimilation techniques for semi-positive-definite variables with highly skewed uncertainty distributions, including aerosol concentrations and tephra deposit mass loading: the Gaussian with non-negative constraints (GNC) and gamma inverse-gamma (GIG) methods. The proposed methods are applied to reconstruct the tephra fallout deposit resulting from the 2015 Calbuco eruption using an ensemble of 256 runs performed with the FALL3D dispersal model. An assessment of the methodologies is conducted considering two independent datasets of deposit thickness measurements: an assimilation dataset and a validation dataset. Different evaluation metrics (e.g. RMSE, MBE, and SMAPE) are computed for the validation dataset, and the results are compared to two references: the ensemble prior mean and the EnKF analysis. Results show that the assimilation leads to a significant improvement over the first-guess results obtained from the simple ensemble forecast. The evidence from this study suggests that the GNC method was the most skilful approach and represents a promising alternative for assimilation of volcanic fallout data. The spatial distributions of the tephra fallout deposit thickness and volume according to the GNC analysis are in good agreement with estimations based on field measurements and isopach maps reported in previous studies. On the other hand, although it is an interesting approach, the GIG method failed to improve the EnKF analysis.

A hybrid data assimilation system based on machine learning

In the earth sciences, numerical weather prediction (NWP) is the primary method of predicting future weather conditions, and its accuracy is affected by the initial conditions. Data assimilation (DA) can provide high-precision initial conditions for NWP. The hybrid 4DVar-EnKF is currently an advanced DA method used by many operational NWP centres. However, it has two major shortcomings: The complex development and maintenance of the tangent linear and adjoint models and the empirical combination of the results of 4DVar and EnKF. In this paper, a new hybrid DA method based on machine learning (HDA-ML) is presented to overcome these drawbacks. In the new method, the tangent linear and adjoint models in the 4DVar part of the hybrid algorithm can be easily obtained by using a bilinear neural network to replace the forecast model, and a CNN model is adopted to fuse the analysis of 4DVar and EnKF to adaptively obtain the optimal coefficient of combination rather than the empirical coefficient as in the traditional hybrid DA method. The hybrid DA methods are compared with the Lorenz-96 model using the true values as labels. The experimental results show that HDA-ML improves the assimilation performance and significantly reduces the time cost. Furthermore, using observations instead of the true values as labels in the training system is more realistic. The results show comparable assimilation performance to that in the experiments with the true values used as the labels. The experimental results show that the new method has great potential for application to operational NWP systems.

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R2-score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(104) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(105) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈O(105)−O(107) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

A comparative analysis for a deep learning model (hyDL-CO v1.0) and Kalman filter to predict CO concentrations in China

Abstract. The applications of novel deep learning (DL) techniques in atmospheric science are rising quickly. Here we build a hybrid DL model (hyDL-CO), based on convolutional neural networks (CNNs) and long short-term memory (LSTM) neural networks, to provide a comparative analysis between DL and Kalman filter (KF) to predict carbon monoxide (CO) concentrations in China in 2015–2020. We find the performance of DL model is better than KF in the training period (2015–2018): the mean bias and correlation coefficients are 9.6 ppb and 0.98 over eastern China and are −12.5 ppb and 0.96 over grids with independent observations (i.e., grids with CO observations that are not used in DL training and KF assimilation). By contrast, the assimilated CO concentrations by KF exhibit comparable correlation coefficients but larger negative biases. Furthermore, the DL model demonstrates good temporal extensibility in the test period (2019–2020): the mean bias and correlation coefficients are 95.7 ppb and 0.93 over eastern China and 81.0 ppb and 0.91 over grids with independent observations, while CO observations are not fed into the DL model as an input variable. Despite these advantages, we find a weaker prediction capability of the DL model than KF in the test period, and a noticeable underestimation of CO concentrations at extreme pollution events in the DL model. This work demonstrates the advantages and disadvantages of DL models to predict atmospheric compositions with respect to traditional data assimilation, which is helpful for better applications of this novel technique in future studies.

Efficient high-dimensional variational data assimilation with machine-learned reduced-order models

Abstract. Data assimilation (DA) in geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by 4 orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA). Our overall formulation is denoted AIEADA (Artificial Intelligence Emulator-Assisted Data Assimilation).

An improved framework for the dynamic likelihood filtering approach to data assimilation.

We propose improvements to the Dynamic Likelihood Filter (DLF), a Bayesian data assimilation filtering approach, specifically tailored to wave problems. The DLF approach was developed to address the common challenge in the application of data assimilation to hyperbolic problems in the geosciences and in engineering, where observation systems are sparse in space and time. When these observations have low uncertainties, as compared to model uncertainties, the DLF exploits the inherent nature of information and uncertainties to propagate along characteristics to produce estimates that are phase aware as well as amplitude aware, as would be the case in the traditional data assimilation approach. Along characteristics, the stochastic partial differential equations underlying the linear or nonlinear stochastic dynamics are differential equations. This study focuses on developing the explicit challenges of relating dynamics and uncertainties in the Eulerian and Lagrangian frames via dynamic Gaussian processes. It also implements the approach using the ensemble Kalman filter (EnKF) and compares the DLF approach to the conventional one with respect to wave amplitude and phase estimates in linear and nonlinear wave problems. Numerical comparisons show that the DLF/EnKF outperforms the EnKF estimates, when applied to linear and nonlinear wave problems. This advantage is particularly noticeable when sparse, low uncertainty observations are used.

An Initial Field Intelligent Correcting Algorithm for Numerical Forecasting Based on Artificial Neural Networks under the Conditions of Limited Observations: Part I—Focusing on Ocean Temperature

For the numerical forecasting of ocean temperature, the effective fusion of observations and the initial field under the conditions of limited observations has always been a significant problem. Traditional data assimilation methods cannot make full use of limited observations to correct the initial field. In order to obtain an optimal initial field with limited observations, this study proposed an intelligent correcting (IC) algorithm based on artificial neural networks (ANNs). The IC algorithm can fully mine the correlation laws between the grid points using historical data, and this process essentially replaces the estimation of background error covariance in traditional data assimilation methods. Experimental results show that the IC algorithm can lead to superior forecasting accuracy, with a lower root mean square error (around 0.7 °C) and higher coefficient of determination (0.9934) relative to the optimal interpolation method. Through the IC algorithm, the largest reduction in mean forecasting error can reach around −0.5 °C and the maximum percentage decline in mean forecasting error can reach 30% compared with the original numerical forecasting results. Therefore, the experiments validate that the IC algorithm can effectively correct the initial field under the conditions of limited observations.

A Hybrid Method for Fine-Scale Wind Field Retrieval Based on Machine Learning and Data Assimilation

To better describe the main features of the complex airflow in the atmospheric boundary layer, a hybrid wind field retrieval method based on machine learning (ML) and data assimilation (DA) is proposed. Based on the joint measurement of lidar and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in situ</i> measurements, a 3-D variational data assimilation (3DVAR) method is used to retrieve the fine-scale wind field. To address the iterative interpolation problem in the traditional DA methods, this article isolates the interpolation from the optimization and uses the regression methods in ML to estimate the interpolated observations on analysis grids. More specifically, the supervised regression and semisupervised regression are, respectively, used for lidar and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in situ</i> observations according to their heterogeneity. Simulation and field measurement results indicate that, compared with the traditional DA methods, the proposed method can better estimate both 2-D and 3-D velocities, by an improvement of more than 42.3% on average.

A new data assimilation method coupled with real-time detection of sensitive parameters: An example of phytoplankton modeling in lakes

数据同化是提升复杂机理过程模型精度的关键技术之一，而湖泊藻类模型的敏感参数具有随时间动态变化的特征，导致数据同化过程中无法精准更新某一时段的敏感参数，影响数据同化的模型精度提升效果.针对上述问题，本研究耦合了参数敏感性分析与集合卡尔曼滤波，研发了一种能够实时识别模型敏感参数的新型数据同化算法；为验证研发算法的效率，依托巢湖的高频水质自动监测数据，测试算法对藻类动态模型的精度提升效果.测试结果表明：研发算法能够精准跟踪模型敏感参数的动态变化，并根据监测数据实时更新模型敏感参数，实现了水质高频自动监测数据与藻类动态模型的深度融合，藻类生物量模拟精度提升了55%，即纳什系数（NSE）从0.49提升到0.76，模拟精度提升效果也显著优于传统数据同化算法（NSE=0.63）.研发算法可应用于其它水生态环境模型的数据同化，为水生态环境相关要素的精准模拟预测提供关键技术支撑.;Data assimilation is a critical method to improve the performance of complex process-based models. However, the sensitive parameters for lake models are generally changing over time. Therefore, it is challenging to accurately update the sensitive parameters for a specific period, which affects the performance of data assimilation. To address the problem, this study developed a new data assimilation method by coupling the methods of parameter sensitivity analysis and the Ensemble Kalman Filter. The new method aimed to identify the model's sensitive parameters in real time. To evaluate its performance, we collected the high-frequency water quality automatic monitoring data of Lake Chaohu, and investigated the performance improvement of a phytoplankton dynamic model using a new data assimilation method. Our investigation results showed that the developed method was able to identify the sensitive parameters of the model in each simulation period, and updated them based on the measured data to achieve better performance. The simulation accuracy of phytoplankton biomass increased by 55%, i.e., the Nash-Sutcliffe Efficiency (NSE) increased from 0.49 to 0.76. This performance is better than that of the traditional data assimilation method (NSE=0.63). The method can be applied to the data assimilation of other ecological and environmental models, technically supporting an accurate prediction of environmental and ecological factors.

The assessment of potential observability for joint chemical states and emissions in atmospheric modelings

In predictive geophysical model systems, uncertain initial values and model parameters jointly influence the temporal evolution of the system. This renders initial-value-only optimization by traditional data assimilation methods as insufficient. However, blindly extending the optimization parameter set jeopardizes the validity of the resulting analysis because of the increase of the ill-posedness of the inversion task. Hence, it becomes important to assess the potential observability of measurement networks for model state and parameters in atmospheric modelings in advance of the optimization. In this paper, we novelly establish the dynamic model of emission rates and extend the transport-diffusion model extended by emission rates. Considering the Kalman smoother as underlying assimilation technique, we develop a quantitative assessment method to evaluate the potential observability and the sensitivity of observation networks to initial values and emission rates jointly. This benefits us to determine the optimizable parameters to observation configurations before the data assimilation procedure and make the optimization more efficiently. For high-dimensional models in practical applications, we derive an ensemble based version of the approach and give several elementary experiments for illustrations.

Integrating deep learning-based data assimilation and hydrogeophysical data for improved monitoring of DNAPL source zones during remediation

Data assimilation techniques allow the integration of multi-source data with physical modeling of dense nonaqueous phase liquid (DNAPL) source zones to monitor their evolution in time and thereby improve remediation efforts. The DNAPL source zone is governed by multiphase physics and its architecture is usually highly irregular and nonstationary. Traditional data assimilation methods use stationary models to describe these nonstationary DNAPL saturation fields, leading to significant errors in the estimation of unknown source zone architectures (SZAs). In this study, we propose a geostatistical data assimilation method to improve performance monitoring of DNAPL remediation. We estimate the DNAPL SZA after parameterizing the DNAPL saturation field using a deep learning method instead of a stationary statistical model. First, we train a convolutional variational autoencoder (CVAE) network to capture the physics of DNAPL infiltration and depletion. Then, we incorporate the CVAE network within the Ensemble Kalman filter (EnKF) to resolve the evolving DNAPL SZA by assimilating time-lapse hydrogeophysical measurements (i.e., oscillatory hydraulic tomography, downgradient dissolved DNAPL concentration, and electrical resistivity tomography). To assess the proposed CVAE-EnKF framework's performance, we conducted numerical experiments with evolving DNAPL SZAs in a 2D heterogeneous aquifer. Results show that the proposed framework significantly improved DNAPL saturation estimates throughout the source zone over time, compared to the standard EnKF method. CVAE-EnKF reduced the estimation error of DNAPL mass remediated by 51% from EnKF, along with better predictions of source zone longevity. It also better captured the morphology and center/spread of mass of the evolving remediated portions of the source zone and provided better estimates of pooled-DNAPL mass. Furthermore, the CVAE-EnKF method is more stable and converges faster than EnKF alone. Overall, the proposed data assimilation framework can improve real-time monitoring of DNAPL remediation and risk analysis.

Hybrid data assimilation based on multilayer perceptron

Data assimilation is widely used in weather forecasting, ocean forecasting, remote sensing observation and other fields. The result of data assimilation directly affects the accuracy of forecasting. Data assimilation algorithms have been studied extensively. The classical algorithms include 3D-Var, 4D-Var, Kalman filter and ensemble Kalman filter. New assimilation algorithms also emerge one after another. In recent years, people have tried to use a hybrid of various data assimilation methods. Deep learning is a hot research field in recent years. Deep learning is able to mine the hidden features of data for analysis and prediction. In this paper, we attempt to use the classical deep learning model-multilayer perceptron to extract the characteristics of traditional data assimilation algorithms to perform hybrid data assimilation. The experiments show that the results of this method are significantly improved compared with the traditional methods, and reveal the value of multilayer perceptron in the direction of hybrid data assimilation.

Data assimilation model based on machine learning

Data assimilation(DA) is a method mainly absorbs the observation data into the simulation model, integrates the errors of observation and simulation, and provides a more accurate state so as to reduce the forecast error. Data assimilation has been widely used in the fields of atmosphere and ocean. However, traditional data assimilation methods require a lot of computing resources and consume a long time. Machine learning is a data analysis method with strong learning ability and rapid prediction ability. Long Short-Term Memory network (LSTM) is a widely used machine learning model, which has good effect on time series prediction. In this paper, we use the historical data of data assimilation to train LSTM model and get the prediction model. The experimental results show that the LSTM model can learn the latent law from the historical data, the results of the model fit the real data well, and the calculation speed is greatly improved compared with the original data assimilation algorithm.

Fast data assimilation (FDA): Data assimilation by machine learning for faster optimize model state

Data assimilation (DA) can provide the more accurate initial state for numerical forecasting models. But traditional DA algorithms has the problem of long calculation time. This paper proposes fast data assimilation (FDA) based on machine learning. For training model, FDA uses 4DVAR, iForest, MLP, and also includes a modified model that does not require observations. This paper applies FDA in the Lorenz63 dynamical system. The experimental results show that the single analysis time of FDA is almost 524 times faster than 4DVAR. FDA greatly reduces the time of the DA process.

A Data-Driven Method for Hybrid Data Assimilation with Multilayer Perceptron

Abstract Accurate and timely weather prediction is of significance for autonomous vehicles, such as designing more appropriate sensors or other configurations and developing safer driving strategies. Generally, as the mainstream weather prediction method, numerical weather prediction (NWP) relies on high-quality spatio-temporal observations. However, the precise state of the real world is not measurable. Thus, how to obtain a proper initial condition estimation based on big geospatial-temporal data is a crucial procedure for NWP. Data assimilation (DA) has been a traditional solution to the problem, for the better performance of which various mathematical-physics models have been used. However, the computational effectiveness and efficiency are still largely compromised by the complicated and nonparallel integration process in existing DA methods. In this paper, we propose a novel data-driven method named HDA-MLP to address the DA problem. We first constructed a customized MLP by introducing the temporal peculiarities of the state variables to simulate and optimize pure 3DVar and EnKF. Then we blended the optimized analysis fields directly by implicitly updating the background error covariance matrix through another neural network model to alleviate the dependence on traditional DA methods. We conducted extensive experiments to investigate the effectiveness and efficiency of the proposal by utilizing two classical nonlinear dynamic models. Results reveal that our approach has better robustness and enhanced capability to capture the variation of state variables. Notably, the analysis quality and computational efficiency are significantly improved.