Laplace Approximation Research Articles

Hospital admission risks and excess costs for neurological symptoms attributable to long-term exposure to fine particulate matter in New York State, USA

BackgroundAlthough emerging evidence suggests that PM2.5 is linked to neurological symptoms (NSs) via neuroinflammation, relevant studies are scarce. This study aimed to investigate the risks and excess costs of hospital admission for five NSs—fatigue, headache, dizziness, convulsion, and paralysis—attributable to long-term exposure to PM2.5 in New York State, USA. MethodsWe analyzed the New York Statewide Planning and Research Cooperative System (SPARCS) from 2010 to 2016. A Bayesian hierarchical model with integrated nested Laplace approximations was performed to estimate the risks and excess costs of hospital admission for NSs due to long-term exposure to PM2.5 at the county level. ResultsA 1 μg/m3 increase in lag 0–1 years PM2.5 was associated with an increased risk of headache and convulsion by 1.06 (95% CI: 1.01, 1.11) and 1.04 (95% CI: 1.01, 1.06), respectively. The excess hospital admission cost for five NSs attributable to lag 0–1 years PM2.5 above the new World Health Organization guideline (annual standard: 5 μg/m3) was $200.24 (95% CI: 6.00, 376.96) million during 2011–2016, recording the highest for convulsion ($153.73 [95% CI: 63.61, 244.19] million). ConclusionsThis study provides quantitative estimates of risks and excess costs for NSs attributable to long-term PM2.5 and suggests that policies that reduce long-term PM2.5 concentration in accordance with the new WHO air quality guidelines can yield substantial health and economic benefits related to NSs in the New York State population.

Fast iterative solver for 3D finite-element modeling of controlled-source electromagnetic responses with vector and scalar potentials

The controlled-source electromagnetic (CSEM) forward modeling with vector and scalar potentials could provide insights into the inductive and galvanic effects. We have developed a finite-element (FE) forward modeling algorithm for the CSEM with vector and scalar potentials. The first-order vector and nodal shape functions are used for the vector and scalar potentials, respectively. To mitigate the null space and thus the nonuniqueness of the potentials caused by the curl operator, we introduce a new preconditioner constructed from an incomplete Cholesky decomposition with zero fill-ins of the Laplacian approximation to the resulted matrix after the FE discretization. We implemented a preconditioned quasiminimal residual method to iteratively solve the resulting linear system with the new preconditioner. We first verified the accuracy of the algorithm through comparison against the analytic solutions obtained for a three-layered earth model. The new iterative modeling algorithm converges much faster compared with the modeling algorithm with the preconditioner constructed from the original stiffness matrix. The efficiency and accuracy are further illustrated by comparison with the modeling scheme based on coupled potentials with explicit enforcement of the Coulomb gauge condition for the vector potential by modeling three 3D models. In addition, the accuracy of our algorithm is demonstrated via a comparison of our numerical solution for a 3D model with the numerical solution obtained from the integral equation method. We also present the inductive and galvanic components of the horizontal secondary electric field for numerical solutions obtained with direct and iterative solvers. The numerical test demonstrated that, aside from the efficiency improved with the new iterative solver, the unstable and nonuniqueness problem for the potentials is eliminated by the new preconditioner with the Laplacian approximation involved.

Shape reconstructions by using plasmon resonances with enhanced sensitivity

This paper investigates the shape reconstructions of sub-wavelength objects from near-field measurements in transverse electromagnetic scattering. This geometric inverse problem is notoriously ill-posed and challenging. We develop a novel reconstruction scheme of using plasmon resonances with significantly enhanced sensitivity and resolution. First, by spectral analysis we establish a sharp quantitative relationship between the sensitivity of the reconstruction and the plasmon resonance. It shows that the sensitivity functional blows up when plasmon resonance occurs. Hence, the signal to noise ratio is significantly improved and the robustness and effectiveness of the reconstruction are ensured. Second, a variational regularization method is proposed to overcome the ill-posedness, and an alternating iteration method is introduced to automatically select the regularization parameters. Third, we use Laplace approximation method to capture the statistical information of the target scattering object. Both rigorous theoretical analysis and extensive numerical experiments are conducted to validate the promising features of our method.

Impact of Two-level Spatial Structure Model in Spatio-temporal Hospitalization of Female Breast Cancer Patients in Nigeria

This paper investigated the impact of two-level spatial structure model in spatio-temporal hospitalization of female breast cancer in Nigerian Health-care areas and States. The objective is to measure the relative contribution of each of the model parameters. The model has been fitted using full Laplace approximation strategy to an observational hospital-based female breast cancer mortality data from 2009 to 2016. The study explained risk variations across health-care areas and State-levels. The results showed that the State-level captured higher risk than the health-care area-level (24.18% against 17.60%). Furthermore, assessing each health-care area performance, six (6) health-care areas with large population States showed higher risk. The study recommended o further research on the risk factors responsible for the significant high risk in these health-care areas.

Spatial and seasonal determinants of Lyme borreliosis incidence in France, 2016 to 2021.

BackgroundLyme borreliosis (LB) is the most widespread hard tick-borne zoonosis in the northern hemisphere. Existing studies in Europe have focused mainly on acarological risk assessment, with few investigations exploring human LB occurrence.AimWe explored the determinants of spatial and seasonal LB variations in France from 2016 to 2021 by integrating environmental, animal, meteorological and anthropogenic factors, and then mapped seasonal LB risk predictions.MethodsWe fitted 2016-19 LB national surveillance data to a two-part spatio-temporal statistical model. Spatial and temporal random effects were specified using a Besag-York-Mollie model and a seasonal model, respectively. Coefficients were estimated in a Bayesian framework using integrated nested Laplace approximation. Data from 2020-21 were used for model validation.ResultsA high vegetation index (≥ 0.6) was positively associated with seasonal LB presence, while the index of deer presence (> 60%), mild soil temperature (15-22 °C), moderate air saturation deficit (1.5-5 mmHg) and higher tick bite frequency were associated with increased incidence. Prediction maps show a higher risk of LB in spring and summer (April-September), with higher incidence in parts of eastern, midwestern and south-western France.ConclusionWe present a national level spatial assessment of seasonal LB occurrence in Europe, disentangling factors associated with the presence and increased incidence of LB. Our findings yield quantitative evidence for national public health agencies to plan targeted prevention campaigns to reduce LB burden, enhance surveillance and identify further data needs. This approach can be tested in other LB endemic areas.

Spatio-temporal dependence modelling of extreme rainfall in South Africa: A Bayesian integrated nested Laplace approximation technique

The spatial and spatio-temporal dependence modeling to extreme value distributions have been used to analyze the extremes of daily maximum rainfall data across selected weather stations in South Africa combining generalized Pareto distribution (GPD) with the flexible Bayesian Latent Gaussian Model (LGM). The paper demonstrated the spatio-temporal GPD model for applications in extreme rainfall data that capture systematic variation through spatial and spatio-temporal modeling framework, in which the temporal constitutes the week and month as random separately. The paper uses the Bayesian integrated Nested Laplace approximation (INLA) algorithm to estimate marginal posterior means of the parameters and hyper-parameters for Bayesian spatio-temporal models. The Bayesian inferences using INLA technique were applied to obtain prediction of the return levels at each station, which incorporate uncertainty due to model estimation, as well as the randomness that is inherent in the processes.

A polynomial chaos efficient global optimization approach for Bayesian optimal experimental design

This paper proposes a global optimization framework to address the high computational cost and non convexity of Optimal Experimental Design (OED) problems. To reduce the computational burden and the presence of noise in the evaluation of the Shannon expected information gain (SEIG), this framework proposes the coupling of Laplace approximation and polynomial chaos expansions (PCE). The advantage of this procedure is that PCE allows large samples to be employed for the SEIG estimation, practically vanishing the noisy introduced by the sampling procedure. Consequently, the resulting optimization problem may be treated as deterministic. Then, an optimization approach based on Kriging surrogates is employed as the optimization engine to search for the global solution with limited computational budget. Four numerical examples are investigated and their results are compared to state-of-the-art stochastic gradient descent algorithms. The proposed approach obtained better results than the stochastic gradient algorithms in all situations, indicating its efficiency and robustness in the solution of OED problems.

Data fusion in a two-stage spatio-temporal model using the INLA-SPDE approach

This paper proposes a two-stage estimation approach for a spatial misalignment scenario that is motivated by the epidemiological problem of linking pollutant exposures and health outcomes. We use the integrated nested Laplace approximation method to estimate the parameters of a two-stage spatio-temporal model — the first stage models the exposures using data fusion while the second stage links the health outcomes to exposures. The first stage is based on the Bayesian melding model, which assumes a common latent field for the different data sources for the pollutants. The second stage fits a generalized linear mixed model using the spatial averages of the estimated latent field, and additional spatial and temporal random effects. Uncertainty from the first stage is accounted for by simulating repeatedly from the posterior predictive distribution of the latent field. A simulation study was carried out to assess the impact of the sparsity of the data on the monitors, number of time points, and the specification of the priors in terms of the biases, RMSEs, and coverage probabilities of the parameters and the block-level exposure estimates. The results show that the parameters are generally estimated correctly but there is difficulty in estimating the Matèrn field parameters. The effect of exposures on the health outcomes is the primary parameter of interest for spatial epidemiologists and health policy makers, and our results show that the proposed method estimates these very well. The proposed method is applied to measurements of NO2 concentration and respiratory hospitalizations for year 2007 in England. The results show that an increase in NO2 levels is significantly associated with an increase in the relative risks of the health outcome. Also, there is a strong spatial structure in the risks, a strong temporal autocorrelation, and a significant spatio-temporal interaction effect.

Impacts of climate change on vicennial spatial behaviors of Trichiurus japonicus in the East China Sea

Understanding fish's spatial behaviors are important for conservation biology under global warming, as one of the direct responses of fish to climate change is the alterations in distribution. We chose Trichiurus japonicus, which is an ecologically and economically important species in the East China Sea (ECS), to study the vicennial responses of demersal fish to climate change. Eight environmental variables (sea bottom temperature, sea bottom salinity, chlorophyll-a concentration, sea surface height, bathymetry, distance to the coast, slope data, and Terrain ruggedness index) combined with the annual survey data were fitted using integrated nested Laplace approximations and stochastic partial differential equations approach under the Bayesian framework to investigate its distribution shifts in spring during 1999–2019. Based on the distribution model, three spatial behavior indicators (center of gravity, Inertia, and Isotropy) were calculated and correlated with six regional climate indicators (Kuroshio index, Taiwan Warm Current Water index, meridional wind, zonal wind, Pacific Decadal Oscillation, and El Niño/Southern Oscillation) to study the potential cause of its spatial behaviors. During the study periods, the biomass of T. japonicus declined gradually, with a brief increase in 2003, 2007–2008, and 2015–2017. T. japonicus preferred high salinity, warm and deep-water area. The relatively persistent hotspots of T. japonicus in spring were in the offshore waters of eastern Zhejiang Province. The change in the center of gravity during the study periods was not obvious, which implied that the distribution shifts were not necessarily vulnerable for T. japonicus in response to climate change and it has strong adaptability to the environment. We also found that strong Kuroshio Current and Taiwan Warm Current Water as well as cold phased of El Niño/Southern Oscillation were accompanied by weakened spatial aggregation of T. japonicus. Overall, for these types of fish with relatively persistent hotspots, the establishment of seasonally protected areas will contribute to the sustainable utilization of the species.

Longitudinal Modeling of Age-Dependent Latent Traits with Generalized Additive Latent and Mixed Models

We present generalized additive latent and mixed models (GALAMMs) for analysis of clustered data with responses and latent variables depending smoothly on observed variables. A scalable maximum likelihood estimation algorithm is proposed, utilizing the Laplace approximation, sparse matrix computation, and automatic differentiation. Mixed response types, heteroscedasticity, and crossed random effects are naturally incorporated into the framework. The models developed were motivated by applications in cognitive neuroscience, and two case studies are presented. First, we show how GALAMMs can jointly model the complex lifespan trajectories of episodic memory, working memory, and speed/executive function, measured by the California Verbal Learning Test (CVLT), digit span tests, and Stroop tests, respectively. Next, we study the effect of socioeconomic status on brain structure, using data on education and income together with hippocampal volumes estimated by magnetic resonance imaging. By combining semiparametric estimation with latent variable modeling, GALAMMs allow a more realistic representation of how brain and cognition vary across the lifespan, while simultaneously estimating latent traits from measured items. Simulation experiments suggest that model estimates are accurate even with moderate sample sizes.

Spatial modelling of agro-ecologically significant grassland species using the INLA-SPDE approach

The use of spatially referenced data in agricultural systems modelling has grown in recent decades, however, the use of spatial modelling techniques in agricultural science is limited. In this paper, we test an effective and efficient technique for spatially modelling and analysing agricultural data using Bayesian hierarchical spatial models (BHSM). These models utilise analytical approximations and numerical integration called Integrated Nested Laplace Approximations (INLA). We critically analyse and compare the performance of the INLA and INLA-SPDE (Integrated Nested Laplace Approximation with Stochastic Partial Differential Equation) approaches against the more commonly used generalised linear model (glm), by modelling binary geostatistical species presence/absence data for several agro-ecologically significant Australian grassland species. The INLA-SPDE approach showed excellent predictive performance (ROCAUC 0.9271–0.9623) for all species. Further, the glm approach not accounting for spatial autocorrelation had inconsistent parameter estimates (switching between significantly positive and negative) when the dataset was subsetted and modelled at different scales. In contrast, the INLA-SPDE approach which accounted for spatial autocorrelation had stable parameter estimates. Using approaches which explicitly account for spatial autocorrelation, such as INLA-SPDE, improves model predictive performance and may provide a significant advantage for researchers by reducing the potential for Type I or false-positive errors in inferences about the significance of predictors.

Bayesian modeling of the temporal evolution of seismicity using the ETAS.inlabru package

The epidemic type aftershock sequence (ETAS) model is widely used to model seismic sequences and underpins operational earthquake forecasting (OEF). However, it remains challenging to assess the reliability of inverted ETAS parameters for numerous reasons. For example, the most common algorithms just return point estimates with little quantification of uncertainty. At the same time, Bayesian Markov chain Monte Carlo implementations remain slow to run and do not scale well, and few have been extended to include spatial structure. This makes it difficult to explore the effects of stochastic uncertainty. Here, we present a new approach to ETAS modeling using an alternative Bayesian method, the integrated nested Laplace approximation (INLA). We have implemented this model in a new R-Package called ETAS.inlabru, which is built on the R packages R-INLA and inlabru. Our study has included extending these packages, which provided tools for modeling log-Gaussian Cox processes, to include the self-exciting Hawkes process that ETAS is a special case of. While we just present the temporal component here, the model scales to a spatio-temporal model and may include a variety of spatial covariates. This is a fast method that returns joint posteriors on the ETAS background and triggering parameters. Using a series of synthetic case studies, we explore the robustness of ETAS inversions using this method of inversion. We also included runnable notebooks to reproduce the figures in this article as part of the package's GitHub repository. We demonstrate that reliable estimates of the model parameters require that the catalog data contain periods of relative quiescence, as well as triggered sequences. We explore the robustness of the method under stochastic uncertainty in the training data and show that the method is robust to a wide range of starting conditions. We show how the inclusion of historic earthquakes prior to the modeled time window affects the quality of the inversion. Finally, we show that rate-dependent incompleteness of earthquake catalogs after large earthquakes have a significant and detrimental effect on the ETAS posteriors. We believe that the speed of the inlabru inversion, which includes a rigorous estimation of uncertainty, will enable a deeper exploration of how to use ETAS robustly for seismicity modeling and operational earthquake forecasting.

On the tightness of the Laplace approximation for statistical inference

Laplace’s method is used to approximate intractable integrals in a statistical problems. The relative error rate of the approximation is not worse than Op(n−1). We provide the first statistical lower bounds showing that the n−1 rate is tight.

Generative models and Bayesian inversion using Laplace approximation

The Bayesian approach to solving inverse problems relies on the choice of a prior. This critical ingredient allows expert knowledge or physical constraints to be formulated in a probabilistic fashion and plays an important role for the success of the inference. Recently, Bayesian inverse problems were solved using generative models as highly informative priors. Generative models are a popular tool in machine learning to generate data whose properties closely resemble those of a given database. Typically, the generated distribution of data is embedded in a low-dimensional manifold. For the inverse problem, a generative model is trained on a database that reflects the properties of the sought solution, such as typical structures of the tissue in the human brain in magnetic resonance imaging. The inference is carried out in the low-dimensional manifold determined by the generative model that strongly reduces the dimensionality of the inverse problem. However, this procedure produces a posterior that does not admit a Lebesgue density in the actual variables and the accuracy attained can strongly depend on the quality of the generative model. For linear Gaussian models, we explore an alternative Bayesian inference based on probabilistic generative models; this inference is carried out in the original high-dimensional space. A Laplace approximation is employed to analytically derive the prior probability density function required, which is induced by the generative model. Properties of the resulting inference are investigated. Specifically, we show that derived Bayes estimates are consistent, in contrast to the approach in which the low-dimensional manifold of the generative model is employed. The MNIST data set is used to design numerical experiments that confirm our theoretical findings. It is shown that the approach proposed can be advantageous when the information contained in the data is high and a simple heuristic is considered for the detection of this case. Finally, the pros and cons of both approaches are discussed.

Veteran suicide and associated community characteristics in Oregon

Developing a public health approach to suicide prevention among United States (US) military veterans requires additional data and guidance on where, when, for whom, and what prevention resources should be deployed. This study examines veteran suicide mortality across one US state (Oregon) to identify county-level “hotspots” for veteran suicide, identify community characteristics associated with increased suicide among veterans, and examine excess spatial risk after accounting for space, time, and community characteristics. We linked Oregon mortality data with VA databases to identify veterans who had resided in Oregon and died by suicide between January 1, 2009 and December 31, 2018 (n = 1727). Community characteristic data were gathered at the county level from publicly available datasets on social determinants of health known to be associated with poor health outcomes, including suicide risk. We estimated spatial generalized linear mixed models for the full 10-year period and for each 5-year period using integrated nested Laplace approximation with county as the higher hierarchy. Smoothed standardized mortality ratios were used to identify counties with higher risk of veteran suicide. We found a small clustering of counties in the southwestern corner of Oregon that held the highest risk for veteran suicide across the ten years studied. In multivariable models, higher prevalence of unmarried persons was the only community measure significantly associated with increased veteran suicide risk. However, social contextual factors as a group, along with geographic space, explained most risk for suicide among veterans at the population level.

A Bayesian Implementation of the Multiscale Geographically Weighted Regression Model with INLA

The multiscale geographically weighted regression (MGWR) model is an important extension of the classical geographically weighted regression (GWR) model that can be used to explore the spatial nonstationarity of the regression relationship in spatial analysis, but also allows for different scales on conditional relationships between response and different predictors. A Bayesian version of the MGWR model is proposed to obtain estimates of the spatially varying coefficients and the bandwidths simultaneously. The hierarchical form of the Bayesian MGWR model has attractive features, including obtaining posterior estimates of the bandwidths and local parameters simultaneously, and their uncertainty can be easily measured. For Bayesian posterior inference, an efficient algorithm based on integrated nested Laplace approximation is introduced to provide a great alternative of the classical Markov chain Monte Carlo algorithm under the Bayesian framework. The performance of the proposed method is evaluated through simulation study, and it is shown that the proposed approach can correctly identify the differences between scales of parameter surfaces and also obtain precise posterior estimates. Finally, for illustration, this approach is used to analyze monthly housing cost data in the state of Georgia.

Joint models of multivariate longitudinal outcomes and discrete survival data with INLA: An application to credit repayment behaviour

Survival models with time-varying covariates (TVCs) are widely used in the literature on credit risk prediction. However, when these covariates are endogenous, the inclusion procedure has been limited to practices such as lagging these variables or treating them as exogenous. That leads to possible biased estimators (depending on the strength of the exogeneity assumption) and a lack of prediction framework that consolidates the joint evolution of the survival process and the endogenous TVCs. The use of joint models is a suitable approach for handling endogeneity, however, it comes at a high computational cost. We propose a joint model for bivariate endogenous TVCs and discrete survival data using integrated nested Laplace approximation (INLA). We illustrate the implementation via simulations and build a model for full-prepayment consumer loans. We also propose a methodology for individual survival prediction using the Laplace method that leads to more accurate approximations than comparable approaches. We evidence the superiority of joint models over the traditional survival approach for an out-of-sample and out-of-time analysis.

Laplace’s rule of succession: a simple and efficient way to compare metaheuristics

Laplace’s rule of succession: a simple and efficient way to compare metaheuristics

Multivariate generalized linear mixed models for underdispersed count data

Researchers are often interested in understanding the relationship between a set of covariates and a set of response variables. To achieve this goal, the use of regression analysis, either linear or generalized linear models, is largely applied. However, such models only allow users to model one response variable at a time. Moreover, it is not possible to directly calculate from the regression model a correlation measure between the response variables. In this article, we employed the Multivariate Generalized Linear Mixed Models framework, which allows the specification of a set of response variables and calculates the correlation between them through a random effect structure that follows a multivariate normal distribution. We used the maximum likelihood estimation framework to estimate all model parameters using Laplace approximation to integrate out the random effects. The derivatives are provided by automatic differentiation. The outer maximization was made using a general-purpose algorithm such as PORT and Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS). We delimited this problem by studying count response variables with the following distributions: Poisson, negative binomial, Conway-Maxwell-Poisson (COM-Poisson), and double Poisson. While the first distribution can model only equidispersed data, the second models equi and overdispersed, and the third and fourth models all types of dispersion (i.e. including underdispersion). The models were implemented on software R with package TMB, based on C++ templates. Besides the full specification, models with simpler structures in the covariance matrix were considered (fixed and common variance, and ρ set to 0) and fixed dispersion. These models were applied to a dataset from the National Health and Nutrition Examination Survey, where two response variables are underdispersed and one can be considered equidispersed that were measured at 1281 subjects. The double Poisson full model specification overcame the other three competitors considering three goodness-of-fit measures: Akaike Information Criteria (AIC), Bayesian Information Criteria (BIC), and maximized log-likelihood. Consequently, it estimated parameters with smaller standard error and a greater number of significant correlation coefficients. Therefore, the proposed model can deal with multivariate count responses and measures the correlation between them taking into account the effects of the covariates.

Bayesian Multiple Linear Regression Model for GDP in Nepal

Gross Domestic Product (GDP) known as the pulse of economy for any country depends on multiple factors like export-import, inflation rate and unemployment rate etc. Statistical assessment of GDP demands fresh concepts to explain GDP through its covariates in order to improve and strengthen estimation process. Descriptive analysis for the considered data set from world bank for GDP and covariates is presented through Heatmaps. Identification and relevance of possible set of covariates is done by Ordinary Least Square (OLS) regression followed by Step-wise regression. We propose an alternative statistical algorithm implemented as Bayesian Inference through Integrated Nested Laplace Approximation (INLA) which bridges the gap of accuracy in estimates as opposed to frequentist OLS regression for explaining GDP of country Nepal. Effect of changing prior parameters is assessed through Deviance Information Criterion (DIC). Different scenarios for prior distribution for regression parameters were analyzed to identify most suitable choice of parameter for normal distribution. The comparison of Bayesian and frequentist modelling results is done using several criteria such as Mean Square Error (MSE), Mean Absolute Deviation (MAD) and Mean Absolute Percent Error (MAPE). Bayesian estimation approach is a more efficient method for parametric estimation as compared to OLS classical method. GDP of Nepal was found to have strongest relationship with unemployment rate of Nepal as evident from both classical and Bayesian model.