Unobserved Regions Research Articles

Constrained Bayesian optimization for engineering bridge design

Designing a bridge is a complex endeavour, involving multiple variables, limitations and requirements. The design process often includes high-fidelity analyses that are computationally expensive, and the internal working tools of the analysis software are often unknown. This limits the applicability of conventional numerical optimization, especially due to time constraints. As a mean to reduce the computational burden, surrogate modelling may be applied. Surrogate models are constructed on the basis of observed results from the computationally expensive high-fidelity analyses, and serves as a fast approximation of unobserved regions in the design space. If probabilistic surrogates are applied, the probabilistic element may be exploited in the optimization phase, resulting in a scheme known as Bayesian optimization. In this article, it is described how to derive a constrained Bayesian optimization scheme in the process of bridge design, where both the goal and constraints are approximated using probabilistic surrogates. The article also presents a case study where constrained Bayesian optimization is applied to a three-span post-tensioned concrete girder and the results are compared to conventional surrogate-based optimization. The results from the case study show that the Bayesian optimization schemes converge after about six iterations, significantly faster than the conventional surrogate optimization scheme, with a consistently higher relative improvement—providing a faster and more confident process for surrogate-based optimization.

Integration of partially observed multimodal and multiscale neural signals for estimating a neural circuit using dynamic causal modeling.

Integrating multiscale, multimodal neuroimaging data is essential for a comprehensive understanding of neural circuits. However, this is challenging due to the inherent trade-offs between spatial coverage and resolution in each modality, necessitating a computational strategy that combines modality-specific information effectively. This study introduces a dynamic causal modeling (DCM) framework designed to address the challenge of combining partially observed, multiscale signals across a larger-scale neural circuit by employing a shared neural state model with modality-specific observation models. The proposed method achieves robust circuit inference by iteratively integrating parameter estimates from local microscale and global meso- or macroscale circuits, derived from signals across various scales and modalities. Parameters estimated from high-resolution data within specific regions inform global circuit estimation by constraining neural properties in unobserved regions, while large-scale circuit data help elucidate detailed local circuitry. Using a virtual ground truth system, we validated the method across diverse experimental settings, combining calcium imaging (CaI), voltage-sensitive dye imaging (VSDI), and blood-oxygen-level-dependent (BOLD) signals-each with distinct coverage and resolution. Our reciprocal and iterative parameter estimation approach markedly improves the accuracy of neural property and connectivity estimates compared to traditional one-step estimation methods. This iterative integration of local and global parameters presents a reliable approach to inferring extensive, complex neural circuits from partially observed, multimodal, and multiscale data, showcasing how information from different scales reciprocally enhances entire circuit parameter estimation.

Generalization of neural network models for complex network dynamics

Differential equations are a ubiquitous tool to study dynamics, ranging from physical systems to complex systems, where a large number of agents interact through a graph. Data-driven approximations of differential equations present a promising alternative to traditional methods for uncovering a model of dynamical systems, especially in complex systems that lack explicit first principles. A recently employed machine learning tool for studying dynamics is neural networks, which can be used for solution finding or discovery of differential equations. However, deploying deep learning models in unfamiliar settings-such as predicting dynamics in unobserved state space regions or on novel graphs-can lead to spurious results. Focusing on complex systems whose dynamics are described with a system of first-order differential equations coupled through a graph, we study generalization of neural network predictions in settings where statistical properties of test data and training data are different. We find that neural networks can accurately predict dynamics beyond the immediate training setting within the domain of the training data. To identify when a model is unable to generalize to novel settings, we propose a statistical significance test.

CURTAINs flows for flows: Constructing unobserved regions with maximum likelihood estimation

Model independent techniques for constructing background data templates using generative models have shown great promise for use in searches for new physics processes at the LHC. We introduce , a major improvement to the CURTAINs method by training the conditional normalizing flow between two side-band regions using maximum likelihood estimation instead of an optimal transport loss. The new training objective improves the robustness and fidelity of the transformed data and is much faster and easier to train. We compare the performance against the previous approach and the current state of the art using the LHC Olympics anomaly detection dataset, where we see a significant improvement in sensitivity over the original CURTAINs method. Furthermore, CURTAINsF4F requires substantially less computational resources to cover a large number of signal regions than other fully data driven approaches. When using an efficient configuration, an order of magnitude more models can be trained in the same time required for ten signal regions, without a significant drop in performance.

Impact of Data Assimilation in Sensitive Features on the Predictability of the 2012 Great Arctic Cyclone

AbstractThe Great Arctic Cyclone 2012 (AC12) is used to understand the role of initial condition errors in the predictability at 2–3‐day forecast range of a high‐impact summer Arctic Cyclone (AC). Ensemble sensitivity analysis (ESA) is first performed to identify potentially sensitive regions of the cyclone evolution using an ensemble baseline forecast with conventional in situ observations assimilated. A pseudo‐observation method is then introduced to investigate impacts of hypothetical observations in these sensitive but unobserved regions. In the baseline experiments with in situ observations assimilated, the forecasted AC12 reaches its peak intensity 18 hr earlier than in the verifying Global Forecast System Analysis (GFS‐ANL) and the cyclone track is biased toward the southwest. Using ESA, the time of peak intensity and the cyclone track error are identified to be sensitive to the upstream trough, downstream ridge, and the tropopause polar vortex (TPV) to the northeast (NE TPV) of the AC12. These features were not observed by the in situ observation networks. To examine the impact of the observation gaps, pseudo‐observations drawn from GFS‐ANL are assimilated. Pseudo‐observations sample the three features separately to study the impact of the initial condition error on the predictability of AC12. The cyclone peak intensity timing error and track error are greatly reduced when the initial condition error is reduced near the NE TPV. A southward expansion of the NE TPV and the corresponding southward shifting low‐level front lead the forecasted AC12 to progress to the east, which better agrees with the verifying GFS‐ANL.

Uncertainty Quantification of Neural Reflectance Fields for Underwater Scenes

Neural radiance fields and neural reflectance fields are novel deep learning methods for generating novel views of 3D scenes from 2D images. To extend the neural scene representation techniques to complex underwater environments, beyond neural reflectance fields underwater (BNU) was proposed, which considers the relighting conditions of on-aboard light sources by using neural reflectance fields, and approximates the attenuation and backscatter effects of water with an additional constant. Because the quality of the neural representation of underwater scenes is critical to downstream tasks such as marine surveying and mapping, the model reliability should be considered and evaluated. However, current neural reflectance models lack the ability of quantifying the uncertainty of underwater scenes that are not directly observed during training, which hinders their widespread use in the field of underwater unmanned autonomous navigation. To address this issue, we introduce an ensemble strategy to BNU that quantifies cognitive uncertainty in color space and unobserved regions with the expectation and variance of RGB values and termination probabilities along the ray. We also employ a regularization method to smooth the density of the underwater neural reflectance model. The effectiveness of the present method is demonstrated in numerical experiments.

Ensemble flow reconstruction in the atmospheric boundary layer from spatially limited measurements through latent diffusion models

Due to costs and practical constraints, field campaigns in the atmospheric boundary layer typically only measure a fraction of the atmospheric volume of interest. Machine learning techniques have previously successfully reconstructed unobserved regions of flow in canonical fluid mechanics problems and two-dimensional geophysical flows, but these techniques have not yet been demonstrated in the three-dimensional atmospheric boundary layer. Here, we conduct a numerical analogue of a field campaign with spatially limited measurements using large-eddy simulation. We pose flow reconstruction as an inpainting problem, and reconstruct realistic samples of turbulent, three-dimensional flow with the use of a latent diffusion model. The diffusion model generates physically plausible turbulent structures on larger spatial scales, even when input observations cover less than 1% of the volume. Through a combination of qualitative visualization and quantitative assessment, we demonstrate that the diffusion model generates meaningfully diverse samples when conditioned on just one observation. These samples successfully serve as initial conditions for a large-eddy simulation code. We find that diffusion models show promise and potential for other applications for other turbulent flow reconstruction problems.

Towards an astronomical foundation model for stars with a transformer-based model

ABSTRACT Rapid strides are currently being made in the field of artificial intelligence using transformer-based models like Large Language Models (LLMs). The potential of these methods for creating a single, large, versatile model in astronomy has not yet been explored. In this work, we propose a framework for data-driven astronomy that uses the same core techniques and architecture as used by LLMs. Using a variety of observations and labels of stars as an example, we build a transformer-based model and train it in a self-supervised manner with cross-survey data sets to perform a variety of inference tasks. In particular, we demonstrate that a single model can perform both discriminative and generative tasks even if the model was not trained or fine-tuned to do any specific task. For example, on the discriminative task of deriving stellar parameters from Gaia XP spectra, we achieve an accuracy of 47 K in Teff, 0.11 dex in log g, and 0.07 dex in [M/H], outperforming an expert XGBoost model in the same setting. But the same model can also generate XP spectra from stellar parameters, inpaint unobserved spectral regions, extract empirical stellar loci, and even determine the interstellar extinction curve. Our framework demonstrates that building and training a single foundation model without fine-tuning using data and parameters from multiple surveys to predict unmeasured observations and parameters is well within reach. Such ‘Large Astronomy Models’ trained on large quantities of observational data will play a large role in the analysis of current and future large surveys.

Piezoresistive Effect of Conductive and Non-Conductive Fillers in Bi-Layer Hybrid CNT Composites under Extreme Strain.

Polymers mixed with conductive fillers hold significant potential for use in stretchable and wearable sensor devices. Enhancing the piezoresistive effect and mechanical stability is critical for these devices. To explore the changes in the electrical resistance under high strains, typically unachievable in single-layer composites, bi-layer structures were fabricated from carbon nanotubes (CNTs) and EcoFlex composites to see unobservable strain regions. Spherical types of non-conductive fillers composed of polystyrene and conductive filler, coated with Ni and Au on non-conductive fillers, were used as secondary fillers to improve the piezoresistive sensitivity of composites, and their respective impact on the conductive network was compared. The electrical and mechanical properties were examined in the static state to understand the impact of these secondary fillers. The changes in the electrical resistance under 100% and 300% tensile strain, and their dependence on the inherent electrical properties of the secondary fillers, were also investigated. Single-layer CNT composites proved incapable of withstanding 300% strain, whereas the bi-layer structures proved resilient. By implementing cyclic stretching tests, contrary to non-conductive fillers, reduced piezoresistive influence of the conductive secondary filler under extreme strain conditions could be observed.

Simulation of the expanding universe in hyperbolic metamaterials.

The particle horizon represents the boundary between observable and unobservable regions of the universe, which changes as the universe expands. Based on transformation optics, hyperbolic electromagnetic metamaterials can be utilized to simulate metrics with different signs due to their unique anisotropic properties. In this paper, we use hyperbolic metamaterials to visually depict the variation of the particle horizon under three models of an expanding universe (open, flat, and closed) by substituting one-dimensional time with one-dimensional space. The good agreement between theory and simulation confirms that hyperbolic metamaterials are excellent for simulating space-times, suggesting their potential as a new platform for cosmological analogies.

CURTAINs for your sliding window: Constructing unobserved regions by transforming adjacent intervals.

We propose a new model independent technique for constructing background data templates for use in searches for new physics processes at the LHC. This method, called Curtains, uses invertible neural networks to parameterise the distribution of side band data as a function of the resonant observable. The network learns a transformation to map any data point from its value of the resonant observable to another chosen value. Using Curtains, a template for the background data in the signal window is constructed by mapping the data from the side-bands into the signal region. We perform anomaly detection using the Curtains background template to enhance the sensitivity to new physics in a bump hunt. We demonstrate its performance in a sliding window search across a wide range of mass values. Using the LHC Olympics dataset, we demonstrate that Curtains matches the performance of other leading approaches which aim to improve the sensitivity of bump hunts, can be trained on a much smaller range of the invariant mass, and is fully data driven.

Deep learning models of the discrete component of the Galactic interstellar γ -ray emission

A significant point-like component from the small scale (or discrete) structure in the H2 interstellar gas might be present in the Fermi-LAT data, but modeling this emission relies on observations of rare gas tracers only available in limited regions of the sky. Identifying this contribution is important to discriminate gamma-ray point sources from interstellar gas, and to better characterize extended gamma-ray sources. We design and train convolutional neural networks to predict this emission where observations of these rare tracers do not exist and discuss the impact of this component on the analysis of the Fermi-LAT data. In particular, we evaluate prospects to exploit this methodology in the characterization of the Fermi-LAT Galactic center excess through accurate modeling of point-like structures in the data to help distinguish between a point-like or smooth nature for the excess. We show that deep learning may be effectively employed to model the gamma-ray emission traced by these rare H2 proxies within statistical significance in data-rich regions, supporting prospects to employ these methods in yet unobserved regions.

Parking space inventory from above: Detection on aerial images and estimation for unobserved regions

AbstractParking is a vital component of today's transportation system and descriptive data are therefore of great importance for urban planning and traffic management. However, data quality is often low: managed parking places may only be partially inventoried, or parking at the curbside and on private ground may be missing. This paper presents a processing chain in which remote sensing data and statistical methods are combined to provide parking area estimates. First, parking spaces and other traffic areas are detected from aerial imagery using a convolutional neural network. Individual image segmentations are fused to increase completeness. Next, a Gamma hurdle model is estimated using the detected parking areas and OpenStreetMap and land use data to predict the parking area adjacent to streets. A systematic relationship is found between the road length and type and the parking area obtained. It is suggested that these results are informative to those needing information on parking in structurally similar regions.

A Generative Model for Quasar Spectra

We build a multi-output generative model for quasar spectra and the properties of their black hole engines, based on a Gaussian process latent-variable model. This model treats every quasar as a vector of latent properties such that the spectrum and all physical properties of the quasar are associated with non-linear functions of those latent parameters; the Gaussian process kernel functions define priors on the function space. Our generative model is trained with a justifiable likelihood function that allows us to treat heteroscedastic noise and missing data correctly, which is crucial for all astrophysical applications. It can simultaneously predict unobserved spectral regions and the physical properties of quasars in held-out test data. We apply the model to rest-frame ultraviolet and optical quasar spectra for which precise black hole masses (based on reverberation-mapping measurements) are available. Unlike reverberation-mapping studies that require multi-epoch data, our model predicts black hole masses from single-epoch spectra—even with limited spectral coverage. We demonstrate the capabilities of the model by predicting black hole masses and unobserved spectral regions. We find that we predict black hole masses at close to the best possible accuracy.

Deep Learning Augmented Data Assimilation: Reconstructing Missing Information with Convolutional Autoencoders

Abstract Remote sensing data play a critical role in improving numerical weather prediction (NWP). However, the physical principles of radiation dictate that data voids frequently exist in physical space (e.g., subcloud area for satellite infrared radiance or no-precipitation region for radar reflectivity). Such data gaps impair the accuracy of initial conditions derived from data assimilation (DA), which has a negative impact on NWP. We use the barotropic vorticity equation to demonstrate the potential of deep learning augmented data assimilation (DDA), which involves reconstructing spatially complete pseudo-observation fields from incomplete observations and using them for DA. By training a convolutional autoencoder (CAE) with a long simulation at a coarse “forecast” resolution (T63), we obtained a deep learning approximation of the “reconstruction operator,” which maps spatially incomplete observations to a model state with full spatial coverage and resolution. The CAE was applied to an incomplete streamfunction observation (∼30% missing) from a high-resolution benchmark simulation and demonstrated satisfactory reconstruction performance, even when only very sparse (1/16 of T63 grid density) observations were used as input. When only spatially incomplete observations are used, the analysis fields obtained from ensemble square root filter (EnSRF) assimilation exhibit significant error. However, in DDA, when EnSRF takes in the combined data from the incomplete observations and CAE reconstruction, analysis error reduces significantly. Such gains are more pronounced with sparse observation and small ensemble size because the DDA analysis is much less sensitive to observation density and ensemble size than the conventional DA analysis, which is based solely on incomplete observations. Significance Statement Data assimilation plays a critical role in improving the skills of modern numerical weather prediction by establishing accurate initial conditions. However, unobservable regions are common in observation data, particularly those derived from remote sensing. The nonlinear relationship between data from observable regions and the physical state of unobservable regions may impede DA efficiency. As a result, we propose that deep learning be used to improve data assimilation in such cases by reconstructing a spatially complete first guess of the physical state with deep learning and then applying data assimilation to the reconstructed field. Such deep learning augmentation is found effective in improving the accuracy of data assimilation, especially for sparse observation and small ensemble size.

Data-driven method to infer the seizure propagation patterns in an epileptic brain from intracranial electroencephalography.

Surgical interventions in epileptic patients aimed at the removal of the epileptogenic zone have success rates at only 60-70%. This failure can be partly attributed to the insufficient spatial sampling by the implanted intracranial electrodes during the clinical evaluation, leading to an incomplete picture of spatio-temporal seizure organization in the regions that are not directly observed. Utilizing the partial observations of the seizure spreading through the brain network, complemented by the assumption that the epileptic seizures spread along the structural connections, we infer if and when are the unobserved regions recruited in the seizure. To this end we introduce a data-driven model of seizure recruitment and propagation across a weighted network, which we invert using the Bayesian inference framework. Using a leave-one-out cross-validation scheme on a cohort of 45 patients we demonstrate that the method can improve the predictions of the states of the unobserved regions compared to an empirical estimate that does not use the structural information, yet it is on the same level as the estimate that takes the structure into account. Furthermore, a comparison with the performed surgical resection and the surgery outcome indicates a link between the inferred excitable regions and the actual epileptogenic zone. The results emphasize the importance of the structural connectome in the large-scale spatio-temporal organization of epileptic seizures and introduce a novel way to integrate the patient-specific connectome and intracranial seizure recordings in a whole-brain computational model of seizure spread.

Clustering analysis of the distribution characteristics of unobserved economic regions in China——Based on multi-indicator panel data

The existence of unobserved economy is one of the important factors affecting GDP calculation. This paper uses the provincial panel data from 2010 to 2019 in China, and adopts the method of principal component feature extraction to carry out cluster analysis on the multi-indicator panel data. This method preserves the dynamic characteristics of the panel data, calculates the comprehensive score of each eigenvalue, and gives weight to the eigenvalue by using the entropy method, so as to optimize the clustering results representing the eight indicators of the unobserved economy. Through the analysis, it is found that the regional development of China’s unobserved economy is obviously different, and each type has different influencing factors. This result has important practical significance for different regions in China to formulate differentiated unobserved economic governance policies. This also helps to make better use of resources and develop an energy-saving economy.

Modeling Watershed-Scale Historic Change in the Alpine Treeline Ecotone Using Random Forest

Historic changes in Alpine Treeline Ecotone were modeled using 21 topographic, climatic, geologic, and disturbance variables in a random forest model. Airborne LiDAR and oblique historic repeat photography were used to identify changes in canopy cover in the West Castle Watershed (WCW), Alberta, Canada (49.3° N, 114.4° W). A Random Forest model was trained on ∼30% of the watershed which was observable in oblique imagery, then used for a spatial extension to predict change classes in the unobserved regions of the watershed. Overall accuracy of the model was 77.3% and kappa showed moderate agreement at 0.56. The relative strength of each prediction variable was compared using permutation importance. Fire exposure, annual temperature, and annual solar radiation were the highest-ranking variables; canopy cover decreases on warm, fire-exposed aspects at high elevations, and increases on cool, non-fire-exposed aspects.

Bridging the gap: Machine learning to resolve improperly modeled dynamics

We present a data-driven modeling strategy to overcome improperly modeled dynamics for systems exhibiting complex spatio-temporal behaviors. We propose a Deep Learning framework to resolve the differences between the true dynamics of the system and the dynamics given by a model of the system that is either inaccurately or inadequately described. Our machine learning strategy leverages data generated from the improper system model and observational data from the actual system to create a neural network to model the dynamics of the actual system. We evaluate the proposed framework using numerical solutions obtained from three increasingly complex dynamical systems. Our results show that our system is capable of learning a data-driven model that provides accurate estimates of the system states both in previously unobserved regions as well as for future states. Our results show the power of state-of-the-art machine learning frameworks in estimating an accurate prior of the system’s true dynamics that can be used for prediction up to a finite horizon.

Efficient Sampling for Rapid Estimation of 3-D Stiffness Distribution via Active Tactile Exploration

This article proposes a novel efficient sampling strategy to rapidly estimate the distribution of stiffness over an inhomogeneous object with a highly limited number of sample points taken from the object surface. The stiffness on the object surface is modeled as a mass-spring system, and its distribution is estimated via tactile exploration using Gaussian process regression. The main objective of this article is to improve the efficiency of the estimation process while producing an accurate estimate for both the overall distribution and some particular areas (i.e., high/low stiff areas). Specifically, the mutual information is employed to quantify the amount of information on the whole space of interest provided by each sample point. The estimated stiffness distribution is also taken into account to locate the extreme stiffness areas. An objective function that consists of these two criteria is proposed to optimally balance between the exploration of the unobserved regions and exploitation of certain local areas that have high/low stiffness. Physical experiments on a variety of inhomogeneous objects demonstrate the advantage of the proposed algorithm in comparison to a popular existing algorithm in terms of accuracy and estimation speed.