Sparse Observations Research Articles

A local particle filter and its Gaussian mixture extension implemented with minor modifications to the LETKF

Abstract. A particle filter (PF) is an ensemble data assimilation method that does not assume Gaussian error distributions. Recent studies proposed local PFs (LPFs), which use localization, as in the ensemble Kalman filter, to apply the PF efficiently for high-dimensional dynamics. Among others, Penny and Miyoshi (2016) developed an LPF in the form of the ensemble transform matrix of the local ensemble transform Kalman filter (LETKF). The LETKF has been widely accepted for various geophysical systems, including numerical weather prediction (NWP) models. Therefore, implementing the LPF consistently with an existing LETKF code is useful. This study develops a software platform for the LPF and its Gaussian mixture extension (LPFGM) by making slight modifications to the LETKF code with a simplified global climate model known as Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY). A series of idealized twin experiments were accomplished under the ideal-model assumption. With large inflation by the relaxation to prior spread, the LPF showed stable filter performance with dense observations but became unstable with sparse observations. The LPFGM showed a more accurate and stable performance than the LPF with both dense and sparse observations. In addition to the relaxation parameter, regulating the resampling frequency and the amplitude of Gaussian kernels was important for the LPFGM. With a spatially inhomogeneous observing network, the LPFGM was superior to the LETKF in sparsely observed regions, where the background ensemble spread and non-Gaussianity were larger. The SPEEDY-based LETKF, LPF, and LPFGM systems are available as open-source software on GitHub (https://github.com/skotsuki/speedy-lpf, last access: 16 November 2022) and can be adapted to various models relatively easily, as in the case of the LETKF.

Seismic wavefield reconstruction based on compressed sensing using data-driven reduced-order model

SUMMARYReconstruction of the distribution of ground motion due to an earthquake is one of the key technologies for the prediction of seismic damage to infrastructure. Particularly, the immediate reconstruction of the spatially continuous wavefield is valuable for decision-making of disaster response decisions in the initial phase. For a fast and accurate reconstruction, utilization of prior information is essential. In fluid mechanics, full-state recovery, which recovers the full state from sparse observation using a data-driven model reduced-order model, is actively used. In this study, the framework developed in the field of fluid mechanics is applied to seismic wavefield reconstruction. A seismic wavefield reconstruction framework based on compressed sensing using the data-driven reduced-order model (ROM) is proposed and its characteristics are investigated through numerical experiments. The data-driven ROM is generated from the data set of the wavefield using the singular value decomposition. The spatially continuous seismic wavefield is reconstructed from the sparse and discrete observation and the data-driven ROM. The observation sites used for reconstruction are effectively selected by the sensor optimization method for linear inverse problems based on a greedy algorithm. The proposed framework was applied to simulation data of theoretical waveform with the subsurface structure of the horizontally stratified three layers. The validity of the proposed method was confirmed by the reconstruction based on the noise-free observation. Since the ROM of the wavefield is used as prior information, the reconstruction error is reduced to an approximately lower error bound of the present framework, even though the number of sensors used for reconstruction is limited and randomly selected. In addition, the reconstruction error obtained by the proposed framework is much smaller than that obtained by the Gaussian process regression. For the numerical experiment with noise-contaminated observation, the reconstructed wavefield is degraded due to the observation noise, but the reconstruction error obtained by the present framework with all available observation sites is close to a lower error bound, even though the reconstructed wavefield using the Gaussian process regression is fully collapsed. Although the reconstruction error is larger than that obtained using all observation sites, the number of observation sites used for reconstruction can be reduced while minimizing the deterioration and scatter of the reconstructed data by combining it with the sensor optimization method. Hence, a better and more stable reconstruction of the wavefield than randomly selected observation sites can be realized, even if the reconstruction is carried out with a smaller number of observations with observation noise, by combining it with the sensor optimization method.

Coupling Techniques for Nonlinear Ensemble Filtering

We consider filtering in high-dimensional non-Gaussian state-space models with intractable transition kernels, nonlinear and possibly chaotic dynamics, and sparse observations in space and time. We propose a novel filtering methodology that harnesses transportation of measures, convex optimization, and ideas from probabilistic graphical models to yield robust ensemble approximations of the filtering distribution in high dimensions. Our approach can be understood as the natural generalization of the ensemble Kalman filter (EnKF) to nonlinear updates, using stochastic or deterministic couplings. The use of nonlinear updates can reduce the intrinsic bias of the EnKF at a marginal increase in computational cost. We avoid any form of importance sampling and introduce non-Gaussian localization approaches for dimension scalability. Our framework achieves state-of-the-art tracking performance on challenging configurations of the Lorenz-96 model in the chaotic regime.

Research Spotlights

The first Research Spotlights article in this issue is concerned with filtering, a task of paramount importance in a great many applications such as numerical weather prediction and geophysical data assimilation. Authors Alessio Spantini, Ricardo Baptista, and Youssef M. Marzouk, in their article “Coupling Techniques for Nonlinear Ensemble Filtering,” describe discrete-time filtering as the act of characterizing the sequence of conditional distributions of the latent field at observation times, given all currently available measurements. Despite the existing literature on filtering, issues such as high-dimensional state spaces and sparse (in both space and time) observations still prove formidable in practice. The traditional approach of ensemble-based data assimilation is the ensemble Kalman filter (EnKF), involving a prediction (forecasting) step followed by an analysis step. However, the authors note an intrinsic bias of EnKF due to the linearity of the transformation, estimated under Gaussian assumptions, that is used in the analysis step, which limits its accuracy. To overcome this, they propose two non-Gaussian generalizations of the EnKF---the so-called stochastic and deterministic map filters---using nonlinear transformations derived from couplings between the forecast distribution and the filtering distribution. What is crucial is that the transformations “can be estimated efficiently...perhaps using only convex optimization,” that they “are easy to `localize' in high dimensions,” and that their computation “should not become increasingly challenging as the variance of the observation noise decreases.” Following a comprehensive description of their new approaches, the authors demonstrate numerically the superiority of their stochastic map filter approach over traditional EnKF. The subsequent discussion offers the reader several jumping off points for future research. Recovery of a sparse solution to a large-scale optimization problem is another ubiquitous problem arising in many applications such as image reconstruction, signal processing, and machine learning. The cost functional typically includes a regularization term in the form of an $\ell_1$ norm term on the solution and/or regularized solution to enforce sparsity. Designing suitable algorithms for such recovery problems is the subject of our second Research Spotlights article. In “Sparse Approximations with Interior Point Methods,” authors Valentina De Simone, Daniela di Serafino, Jacek Gondzio, Spyridon Pougkakiotis, and Marco Viola set out to correct the misconception that first-order methods are to be preferred over second-order methods out of hand. Through case studies, they offer evidence that interior point methods (IPMs) which are constructed to “exploit special features of the problems in the linear algebra of IPMs” and which are designed “to take advantage of the expected sparsity of the optimal solution” can in fact be the method of choice for solving this class of optimization problems. The key to their approach is a reformulation of the original sparse approximation problem to one which is seemingly larger but which has properties upon which one can capitalize for computational gain. For each of four representative applications, the authors show how to take computational advantage of the problem-specific structure of the underlying linear systems involved at each iteration. These efforts are complemented by leveraging the expected sparsity: employing heuristics to drop near zero variables, thereby replacing very large, ill-conditioned intermediate systems by better conditioned, smaller systems. Their conclusion is that time invested in tailoring solvers to structure admitted by the reformulated variant, and in taking advantage of expected sparsity, may be well spent, since their demonstrations have shown it is possible for IPMs to have a “noticeable advantage” over state-of-the-art first-order methods for sparse approximation problems.

Advancing AI-based pan-European groundwater monitoring

The main challenge of pan-European groundwater (GW) monitoring is the sparsity of collated water table depth (wtd) observations. The wtd anomaly (wtda ) is a measure of the increased wtd due to droughts. Combining long short-term memory (LSTM) networks and transfer learning (TL), we propose an AI-based methodology LSTM-TL to produce reliable wtda estimates at the European scale in the absence of consistent wtd observational data sets. The core idea of LSTM-TL is to transfer the modeled relationship between wtda and input hydrometeorological forcings to the observation-based estimation, in order to provide reliable wtda estimates for regions with no or sparse wtd observations. With substantially reduced computational cost compared to physically-based numerical models, LSTM-TL obtained wtda estimates in good agreement with in-situ wtda measurements from 2569 European GW monitoring wells, showing r ⩾ 0.5, root-mean-square error ⩽1.0 and Kling-Gupta efficiency ⩾0.3 at about or more than half of the pixels. Based on the reconstructed long-term European monthly wtda data from the early 1980s to the near present, we provide the first estimate of seasonal wtda trends in different European regions, that is, significant drying trends in central and eastern Europe, which facilitates the understanding of historical GW dynamics in Europe. The success of LSTM-TL in estimating wtda also highlights the advantage of combining AI techniques with knowledge contained in physically-based numerical models in hydrological studies.

Equation‐Free Surrogate Modeling of Geophysical Flows at the Intersection of Machine Learning and Data Assimilation

AbstractThere is a growing interest in developing data‐driven reduced‐order models for atmospheric and oceanic flows that are trained on data obtained either from high‐resolution simulations or satellite observations. The data‐driven models are non‐intrusive in nature and offer significant computational savings compared to large‐scale numerical models. These low‐dimensional models can be utilized to reduce the computational burden of generating forecasts and estimating model uncertainty without losing the key information needed for data assimilation (DA) to produce accurate state estimates. This paper aims at exploring an equation‐free surrogate modeling approach at the intersection of machine learning and DA in Earth system modeling. With this objective, we introduce an end‐to‐end non‐intrusive reduced‐order modeling (NIROM) framework equipped with contributions in modal decomposition, time series prediction, optimal sensor placement, and sequential DA. Specifically, we use proper orthogonal decomposition (POD) to identify the dominant structures of the flow, and a long short‐term memory network to model the dynamics of the POD modes. The NIROM is integrated within the deterministic ensemble Kalman filter (DEnKF) to incorporate sparse and noisy observations at optimal sensor locations obtained through QR pivoting. The feasibility and the benefit of the proposed framework are demonstrated for the NOAA Optimum Interpolation Sea Surface Temperature (SST) V2 data set. Our results indicate that the NIROM is stable for long‐term forecasting and can model dynamics of SST with a reasonable level of accuracy. Furthermore, the prediction accuracy of the NIROM gets improved by almost one order of magnitude by the DEnKF algorithm.

Learning Perceptual Locomotion on Uneven Terrains Using Sparse Visual Observations

To proactively navigate and traverse various terrains, active use of visual perception becomes indispensable. We aim to investigate the feasibility and performance of using sparse visual observations to achieve perceptual locomotion over a range of common terrains (steps, ramps, gaps, and stairs) in human-centered environments. We formulate a selection of sparse visual inputs suitable for locomotion over the terrains of interest, and propose a learning framework to integrate exteroceptive and proprioceptive states. We design state observations and a training curriculum to learn feedback control policies effectively over a range of different terrains. We extensively validate and benchmark the learned policy in various tasks: omnidirectional walking on flat ground, and forward locomotion over various obstacles, showing high success rate of traversability. Furthermore, we study exteroceptive ablations and evaluate policy generalization by adding various levels of noise and testing on new unseen terrains. We demonstrate the capabilities of autonomous perceptual locomotion that can be achieved by <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only</i> using sparse visual observations from direct depth measurements, which are easily available from a Lidar or RGB-D sensor, showing robust ascent and descent over high stairs of 20 cm height, i.e., 50% leg length, and robustness against noise and unseen terrains.

High frequency stochastic acoustic wavefront propagation and joint ocean-acoustic inference: The GMM-DO wavefront

In marine applications, the value of accurate modeling and learning for stochastic acoustic propagation in uncertain ocean environments cannot be overstated. In this work, we derive stochastic theory and schemes for (i) modeling of high frequency acoustic propagation in uncertain ocean environments and (ii) joint Bayesian assimilation of ocean-acoustic measurements to infer fields, parameters, and uncertain model functions. We first obtain the Dynamically Orthogonal (DO) wavefront equations to solve for the stochastic extension of the Liouville Equation that governs the dynamics of acoustic wavefront in an augmented phase space. These DO wavefront equations provide the prior for the Gaussian Mixture Model—DO (GMM-DO) filter that completes joint physics-acoustics Bayesian inference using sparse observations. Specifically, given a set of receivers, the Eulerian nature of the DO wavefront equations allows for the efficient extraction of arrival time prior probability distributions. The GMM-DO Waverfront filter then combines these joint priors with arrival time measurements using Bayes rule, jointly inferring environmental properties (e.g., unknown source location and/or sound speed field), the acoustic wavefront distribution, and the arrival time distribution itself. We evaluate results using high-frequency applications, illustrating the estimation of mean fields and properties, but also of probability density distributions and model parameterizations.

Dynamic mode decomposition for underwater acoustics on autonomous platforms

A sufficiently accurate representation of the ocean physics is needed for reliable underwater acoustics forecasting. Ocean physics are often simulated by high-dimensional data-assimilative numerical ocean simulations. Due to operational constraints such as computing, memory, communication-bandwidth, etc., it is infeasible to run these numerical simulations onboard the platforms. In this work, we employ the onboard data-assimilative incremental, Low-Rank Dynamic Mode Decomposition (iLRDMD) for forecast transmission, onboard computation, and data assimilation. The iLRDMD is an adaptive reduced-order model (ROM) that uses proper orthogonal decomposition (POD) and DMD for compression and transmission of high-fidelity ocean forecast to communication-disadvantaged platforms. It provides reduced-order onboard forecasting and also enables efficient updates of the DMD model itself from inputs from remote centers. Finally, when sparse observations are made by the platform, the iLRDM uses Bayesian data assimilation to correct both the forecasts and the DMD model. We demonstrate the use of deterministic and data-assimilative iLRDMD forecasted ocean fields for underwater acoustics computations in real ocean examples. We further explore the use of our adaptive ROM for joint reduction of the ocean physics and acoustics fields.

Joint ocean physics-acoustics inversion and mutual-information based experiment design in realistic ocean fields

Reliable acoustic exploration and navigation in the ocean requires precise knowledge of the environmental fields (e.g. ocean physics, bathymetry, seabed) and acoustic parameters (e.g. source location and frequencies). However, such knowledge is typically incomplete due to the sparse and heterogeneous observations, and to the complex high-dimensional multiscale dynamics. In this work, we use our stochastic Dynamically Orthogonal Parabolic Equation (DO-ParEq) framework to: (i) model the ocean environment uncertainties computed from a large realistic ocean ensemble forecast, and (ii) predict the resulting stochastic acoustic fields and their probability distributions. We then employ new nonlinear Bayesian learning algorithms that use Gaussian Mixture Models to assimilate ocean-acoustic measurements and jointly infer the unknown properties of the ocean environment, the bathymetry, the acoustic field and parameters, and even the model parameterizations themselves. We showcase the developed techniques in realistic data-assimilative sea experiments in the New York Bight Region and the Mediterranean Sea. Finally, we demonstrate the impact of selecting measurement locations using a mutual-information-based algorithm that maximizes the information content in the learned fields based on the measurement constraints.

Sources and Pathways of Intraseasonal Meridional Kinetic Energy in the Equatorial Atlantic Ocean

Abstract In the equatorial Atlantic Ocean, meridional velocity variability exhibits a pronounced peak on intraseasonal time scales whereas zonal velocity dominantly varies on seasonal to interannual time scales. We focus on the intraseasonal meridional velocity variability away from the near-surface layer, its source regions, and its pathways into the deep ocean. This deep intraseasonal velocity variability plays a key role in equatorial dynamics as it is an important energy source for the deep equatorial circulation. The results are based on the output of a high-resolution ocean model revealing intraseasonal energy levels along the equator at all depths that are in good agreement with shipboard and moored velocity data. Spectral analyses reveal a pronounced signal of intraseasonal Yanai waves with westward phase velocities and zonal wavelengths longer than 450 km. Different sources and characteristics of these Yanai waves are identified: near the surface between 40° and 10°W, low-baroclinic-mode Yanai waves with periods of around 30 days are excited. These waves have a strong seasonal cycle with a maximum in August. High-frequency Yanai waves (10–20-day period) are excited at the surface east of 10°W. In the region between the North Brazil Current and the Equatorial Undercurrent, high-baroclinic-mode Yanai waves with periods between 30 and 40 days are generated. Yanai waves with longer periods (40–80 days) are shed from the deep western boundary current. The Yanai wave energy is carried along beams eastward and downward, thus explaining differences in strength, structure, and periodicity of the meridional intraseasonal variability in the equatorial Atlantic Ocean. Significance Statement Past studies show that intraseasonal meridional kinetic energy is important for the deep equatorial circulation (DEC). However, numerical studies use intraseasonal variability with varying characteristics to investigate the formation and maintenance of the DEC. This is partly because of sparse observations at depth that are limited to single locations. This study investigates intraseasonal meridional kinetic energy in the equatorial Atlantic in a high-resolution ocean model that is tested against available shipboard and moored observations. We analyze the spatial and temporal distribution and the baroclinic structure of intraseasonal variability. Using the model, we identify different sources and pathways of intraseasonal energy in the deep equatorial Atlantic. We offer groundwork for further studies on the formation and maintenance of the DEC.

A detection of the sea level fingerprint of Greenland Ice Sheet melt.

Rapid melting of ice sheets and glaciers drives a unique geometry, or fingerprint, of sea level change. However, the detection of individual fingerprints has been challenging because of sparse observations at high latitudes and the difficulty of disentangling ocean dynamic variability from the signal. We predict the fingerprint of Greenland Ice Sheet (GrIS) melt using recent ice mass loss estimates from radar altimetry data and model reconstructions of nearby glaciers and compare this prediction to an independent, altimetry-derived sea surface height trend corrected for ocean dynamic variability in the region adjacent to the ice sheet. A statistically significant correlation between the two fields (P < 0.001) provides an unambiguous observational detection of the near-field sea level fingerprint of recent GrIS melting in our warming world.

Using Ice Cores to Evaluate CMIP6 Aerosol Concentrations Over the Historical Era

AbstractThe radiative effect of anthropogenic aerosols is one of the largest uncertainties in Earth's energy budget over the industrial period. This uncertainty is in part due to sparse observations of aerosol concentrations in the pre‐satellite era. To address this lack of measurements, ice cores can be used, which contain the aerosol concentration record. To date, these observations have been under‐utilized for comparison to aerosol concentrations found in state‐of‐the‐art Earth system models (ESMs). Here we compare long term trends in concentrations of sulfate and black carbon (BC) between 15 ice cores and 11 ESMs over nine regions around the world during the period 1850–2000. We find that for sulfate concentration trends model results generally agree with ice core records, whereas for BC concentration the model trends differ from the records. Absolute concentrations of both investigated species are overestimated by the models, probably in part due to representation errors. However, we assume that biases in relative trends are not altered by these errors. Ice cores in the European Alps and Greenland record a maximum BC concentration before 1950, while most ESMs used in this study agree on a post‐1950 maximum. We source this bias to an error in BC emission inventories in Europe. Emission perturbation experiments using NorESM2‐LM support the observed finding that BC concentrations in Northern Greenland ice cores are recording European emissions. Errors in BC emission inventories have implications for all future and past studies where Coupled Model Intercomparison Project Phase 6 historical simulations are compared to observations relevant to aerosol forcing.

Optimal control methods for nonlinear parameter estimation in biophysical neuron models.

Functional forms of biophysically-realistic neuron models are constrained by neurobiological and anatomical considerations, such as cell morphologies and the presence of known ion channels. Despite these constraints, neuron models still contain unknown static parameters which must be inferred from experiment. This inference task is most readily cast into the framework of state-space models, which systematically takes into account partial observability and measurement noise. Inferring only dynamical state variables such as membrane voltages is a well-studied problem, and has been approached with a wide range of techniques beginning with the well-known Kalman filter. Inferring both states and fixed parameters, on the other hand, is less straightforward. Here, we develop a method for joint parameter and state inference that combines traditional state space modeling with chaotic synchronization and optimal control. Our methods are tailored particularly to situations with considerable measurement noise, sparse observability, very nonlinear or chaotic dynamics, and highly uninformed priors. We illustrate our approach both in a canonical chaotic model and in a phenomenological neuron model, showing that many unknown parameters can be uncovered reliably and accurately from short and noisy observed time traces. Our method holds promise for estimation in larger-scale systems, given ongoing improvements in calcium reporters and genetically-encoded voltage indicators.

SAR Image Reconstruction of Vehicle Targets Based on Tensor Decomposition

Due to the imaging mechanism of Synthetic Aperture Radars (SARs), the target shape on an SAR image is sensitive to the radar incidence angle and target azimuth, but there is strong correlation and redundancy between adjacent azimuth images of SAR targets. This paper studies multi-angle SAR image reconstruction based on non-negative Tucker decomposition using adjacent azimuth images reconstructed to form a sparse tensor. Sparse tensors are used to perform non-negative Tucker decomposition, resulting in non-negative core tensors and factor matrices. The reconstruction tensor is obtained by calculating the n-mode product of the core tensor and the factor matrix, and then image reconstruction is realized. The similarity between the original image and the reconstructed image is calculated by using the structural similarity index and the cosine of the angle between the feature vectors. The reconstruction results of three target images of MSTAR show that the reconstructed image has a similarity higher than 95% with the original image in most cases, which can support target recognition under sparse observation to a certain extent.

Drivers and distribution of global ocean heat uptake over the last half century

Since the 1970s, the ocean has absorbed almost all of the additional energy in the Earth system due to greenhouse warming. However, sparse observations limit our knowledge of where ocean heat uptake (OHU) has occurred and where this heat is stored today. Here, we equilibrate a reanalysis-forced ocean-sea ice model, using a spin-up that improves on earlier approaches, to investigate recent OHU trends basin-by-basin and associated separately with surface wind trends, thermodynamic properties (temperature, humidity and radiation) or both. Wind and thermodynamic changes each explain ~ 50% of global OHU, while Southern Ocean forcing trends can account for almost all of the global OHU. This OHU is enabled by cool sea surface temperatures and sensible heat gain when atmospheric thermodynamic properties are held fixed, while downward longwave radiation dominates when winds are fixed. These results address long-standing limitations in multidecadal ocean-sea ice model simulations to reconcile estimates of OHU, transport and storage.

The reliability of categorical triple collocation for evaluating soil freeze/thaw datasets

Seasonal soil freeze/thaw (FT) state transition plays a critical role in the range of ecosystem, hydrological and biogeochemical processes. A thorough and large-scale validation of remote-sensed or model-based FT products is therefore quite important. Previous validation studies have applied categorical triple collocation (CTC) as a cross-validation method to estimate the relative performance ranking of various FT datasets, including in situ observations. While CTC has proven useful for qualitatively evaluating FT datasets, quantitative estimates of classification accuracy, which has not yet been assessed against direct validation results, would be even more valuable. To fill this gap, we compare CTC estimated performance rankings and quantitative classification accuracies with those obtained from dense soil temperature and sparse surface temperature observations from April 2015 through December 2019. CTC estimated classification accuracies are found to be strongly correlated (r > 0.927) with dense ground observations, along with very low bias (< 0.038) and RMSE (< 0.086). However, the bias and RMSE of CTC-estimated freeze accuracies are significantly inflated when sparse surface temperatures are used instead as the benchmark. Small errors are found with low absolute values (<0.317) of CTC-estimated class imbalance and a sample size of at least 365. CTC can generally provide the correct performance ranking for each product within a triplet - with low risk of incorrectly ranking all three products. In addition, a sample size of 10–160 is adequate for CTC to provide the correct ranking for the highest- or lowest-ranked product. This improves our knowledge and understanding of the reliability of the CTC method.

Sparse SAR Imaging Method for Ground Moving Target via GMTSI-Net

Ground moving targets (GMT), due to the existence of velocity in range and azimuth direction, will lead to the deviation from their true position and defocus in the azimuth direction during the synthetic aperture radar (SAR) imaging process. To address this problem and compress the amount of echo data, a sparse SAR imaging method for ground moving targets is proposed. Specifically, we first constructed a two-dimensional sparse observation model of the GMT based on matched filter operators. Then, the observation model was solved by a deep network, GMT sparse imaging network (GMTSI-Net), which was mainly obtained by unfolding an iterative soft threshold algorithm (ISTA)-based iterative solution. Furthermore, we designed an adaptive unfolding module in the imaging network to improve the adaptability of the network to the input of echo data with different sampling ratios. The proposed imaging network can achieve faster and more accurate SAR images of ground moving targets under a low sampling ratio and signal-to-noise ratio (SNR). Simulated and measured data experiments were conducted to demonstrate the performance of imaging quality of the proposed method.

A comparison of multiple methods for mapping groundwater levels in the Mu Us Sandy Land, China

Study regionMu Us Sandy Land, China. Study focusThe groundwater table in unconfined aquifers is often a subdued replica of the topography or land surface. In hilly terrain, spatial interpolation methods are prone to errors such as higher interpolation water level than the ground surface due to sparse observation wells. Meanwhile, in ridge areas, groundwater depth can easily be overestimated with limited groundwater observations. To address these issues, this paper proposes a framework to learn groundwater depth using related groundwater variables based on the extreme gradient boosting (XGB) machine learning method. Moreover, the groundwater level map is drawn based on the predicted groundwater depths and elevation data.New Hydrological Insights for the RegionThe results obtained by the XGB algorithm are compared with those of common interpolation methods (including inverse distance weighting, ordinary kriging, and collaborative ordinary kriging) and machine learning methods (including multiple linear regression, geographically weighted regression, support vector regression, and random forest). The results showed that the XGB using the Tweedie loss function performs best in learning groundwater depth compared with interpolation methods and other machine learning methods, and almost completely overcomes the effects of imbalanced data. The topographic factors such as elevation, topographic position index, and topographic roughness index are key factors for groundwater depth. This study provides a reference for improving regional GL mapping, especially in areas of complex terrain.

Airborne observations during KORUS-AQ show that aerosol optical depths are more spatially self-consistent than aerosol intensive properties

Abstract. Aerosol particles can be emitted, transported, removed, or transformed, leading to aerosol variability at scales impacting the climate (days to years and over hundreds of kilometers) or the air quality (hours to days and from meters to hundreds of kilometers). We present the temporal and spatial scales of changes in AOD (aerosol optical depth) and aerosol size (using Ångström exponent – AE; fine-mode fraction – FMF) over Korea during the 2016 KORUS-AQ (KORea–US Air Quality) atmospheric experiment. We use measurements and retrievals of aerosol optical properties from airborne instruments for remote sensing (4STAR; Spectrometers for Sky-Scanning Sun-Tracking Atmospheric Research) and in situ (LARGE; NASA Langley Aerosol Research Group Experiment) on board the NASA DC-8 and geostationary satellites (GOCI; Geostationary Ocean Color Imager; Yonsei aerosol retrieval – YAER, version 2) as well as from reanalysis (MERRA-2; Modern-Era Retrospective Analysis for Research and Applications, version 2). Measurements from 4STAR when flying below 1000 m show an average AOD at 501 nm of 0.36 and an average AE of 1.11 with large standard deviation (0.12 and 0.15 for AOD and AE, respectively), likely due to mixing of different aerosol types (fine and coarse mode). The majority of AOD due to fine-mode aerosol is observed at altitudes lower than 2 km. Even though there are large variations, for 18 out of the 20 flight days, the column AOD measurements by 4STAR along the NASA DC-8 flight trajectories match the South Korean regional average derived from GOCI. GOCI-derived FMF, which was found to be slightly low compared to AErosol RObotic NETwork (AERONET) sites (Choi et al., 2018), is lower than 4STAR's observations during KORUS-AQ. Understanding the variability of aerosols helps reduce uncertainties in the aerosol direct radiative effect by quantifying the errors due to interpolating between sparse aerosol observation sites or modeled pixels, potentially reducing uncertainties in the upcoming observational capabilities. We observed that, contrary to the prevalent understanding, AE and FMF are more spatially variable than AOD during KORUS-AQ, even when accounting for potential sampling biases by using Monte Carlo resampling. Averaging between measurements and models for the entire KORUS-AQ period, the reduction in correlation by 15 % is 65.0 km for AOD and shorter at 22.7 km for AE. While there are observational and model differences, the predominant factor influencing spatial–temporal homogeneity is the meteorological period. High spatiotemporal variability occurs during the dynamic period (25–31 May), and low spatiotemporal variability occurs during the blocking pattern (1–7 June). While AOD and FMF / AE are interrelated, the spatial variability and relative variability of these parameters in this study indicate that microphysical processes vary at scales shorter than aerosol concentration processes at which microphysical processes such as aerosol particle formation, growth, and coagulation mostly impact the dominant aerosol size (characterized by, e.g., FMF / AE) and to some degree AOD. In addition to impacting aerosol size, aerosol concentration processes such as aerosol emission, transport, and removal mostly impact the AOD.