Variational Approximation Research Articles

Numerical approximation of boundary value problems for curvature flow and elastic flow in Riemannian manifolds

We present variational approximations of boundary value problems for curvature flow (curve shortening flow) and elastic flow (curve straightening flow) in two-dimensional Riemannian manifolds that are conformally flat. For the evolving open curves we propose natural boundary conditions that respect the appropriate gradient flow structure. Based on suitable weak formulations we introduce finite element approximations using piecewise linear elements. For some of the schemes a stability result can be shown. The derived schemes can be employed in very different contexts. For example, we apply the schemes to the Angenent metric in order to numerically compute rotationally symmetric self-shrinkers for the mean curvature flow. Furthermore, we utilise the schemes to compute geodesics that are relevant for optimal interface profiles in multi-component phase field models.

Physiological Predictors for the Audio-Motor Synchronization Ability

The meshless local Petrov–Galerkin (MLPG) method is employed for anisotropic transient thermoelasticity analysis of 2D decagonal quasicrystals (QCs) subjected to transient thermal and mechanical shock loadings. The wave type model and the elasto-hydrodynamic model are applied to derive the phonon and phason governing equations, respectively. The temperature affects only the phonon field. To find the temperature distributions on the assumed 2D domain, the anisotropic heat conduction problem is solved using the MLPG method. Also, the MLPG method is successfully employed to obtain the transient behaviors of both phonon and phason displacements by solving the governing equations in local integral equations (LIEs) forms. Making use a unit step function as the test functions in the local weak-form of governing equations, we derived the local integral equations (LIEs) considered on small subdomains identical with support domains of test functions around each node. The radial basis functions are used for approximation of the spatial variation of field variables. The Laplace-transform technique is utilized to discretize the time variations.

Optical Solitons and Vortices in Fractional Media: A Mini-Review of Recent Results

The article produces a brief review of some recent results which predict stable propagation of solitons and solitary vortices in models based on the nonlinear Schrödinger equation (NLSE) including fractional one-dimensional or two-dimensional diffraction and cubic or cubic-quintic nonlinear terms, as well as linear potentials. The fractional diffraction is represented by fractional-order spatial derivatives of the Riesz type, defined in terms of the direct and inverse Fourier transform. In this form, it can be realized by spatial-domain light propagation in optical setups with a specially devised combination of mirrors, lenses, and phase masks. The results presented in the article were chiefly obtained in a numerical form. Some analytical findings are included too, in particular, for fast moving solitons and the results produced by the variational approximation. Moreover, dissipative solitons are briefly considered, which are governed by the fractional complex Ginzburg–Landau equation.

Optical solitons generated by a symbiotic organism search algorithm

We introduce a robust technique that allows generating optical solitons beams through a numerical approach of a variational approximation by using for the first time, to the best of our knowledge, the symbiotic organism search (SOS). This bio-inspired algorithm is able to optimize the design of beam parameters on-demand, such as amplitude and width, for several complex structured light beams in a plethora of nonlinear media. We report some SOS optimized optical structures such as fundamental, vortices and azimuthons optical beams for the case of Kerr, saturation and competing nonlinearities. We propagate the optimized optical beams obtained by SOS and we report that the beams show a self-trapped dynamics during their evolution. Furthermore, the nature of this algorithm allows a straightforward scalability for optimization of multiple parameters. Remarkably, this optimization can be used for different structured light beams at the same time, contrary to the most of numerical methods known.

Dissipative time-domain one-dimensional model for viscothermal acoustic propagation in wind instruments.

Accurate modeling of the acoustic propagation in tubes of varying cross section in musical acoustics must include the effects of the viscous and thermal boundary layers. Models of viscothermal losses are classically written in the frequency domain. An approximate time-domain model is proposed in which all of the physical parameters of the instrument, the bore shape or the wave celerity, are explicit coefficients. The model depends on absolute tabulated constants, which only reflect that the pipe is axisymmetric. It can be understood as a telegrapher's equations augmented by an adjustable number of auxiliary unknowns. A global energy is dissipated. A time discretization based on variational approximation is proposed along with numerical experiments and comparisons with other models.

Infinite-dimensional gradient-based descent for alpha-divergence minimisation

This paper introduces the $(\alpha, \Gamma)$-descent, an iterative algorithm which operates on measures and performs $\alpha$-divergence minimisation in a Bayesian framework. This gradient-based procedure extends the commonly-used variational approximation by adding a prior on the variational parameters in the form of a measure. We prove that for a rich family of functions $\Gamma$, this algorithm leads at each step to a systematic decrease in the $\alpha$-divergence and derive convergence results. Our framework recovers the Entropic Mirror Descent algorithm and provides an alternative algorithm that we call the Power Descent. Moreover, in its stochastic formulation, the $(\alpha, \Gamma)$-descent allows to optimise the mixture weights of any given mixture model without any information on the underlying distribution of the variational parameters. This renders our method compatible with many choices of parameters updates and applicable to a wide range of Machine Learning tasks. We demonstrate empirically on both toy and real-world examples the benefit of using the Power descent and going beyond the Entropic Mirror Descent framework, which fails as the dimension grows.

Accelerating Bayesian Estimation for Network Poisson Models Using Frequentist Variational Estimates

Abstract This work is motivated by the analysis of ecological interaction networks. Poisson stochastic block models are widely used in this field to decipher the structure that underlies a weighted network, while accounting for covariate effects. Efficient algorithms based on variational approximations exist for frequentist inference, but without statistical guaranties as for the resulting estimates. In the absence of variational Bayes estimates, we show that a good proxy of the posterior distribution can be straightforwardly derived from the frequentist variational estimation procedure, using a Laplace approximation. We use this proxy to sample from the true posterior distribution via a sequential Monte Carlo algorithm. As shown in the simulation study, the efficiency of the posterior sampling is greatly improved by the accuracy of the approximate posterior distribution. The proposed procedure can be easily extended to other latent variable models. We use this methodology to assess the influence of available covariates on the organization of several ecological networks, as well as the existence of a residual interaction structure.

Variational progressive-iterative approximation for RBF-based surface reconstruction

Variational progressive-iterative approximation for RBF-based surface reconstruction

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation.

Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets.

Multihump thermo-reorientational solitary waves in nematic liquid crystals: Modulation theory solutions

The propagation of light-induced thermo-reorientational solitary waves in nematic liquid crystals is studied using numerical solutions of the full governing equations and variational approximations. These thermo-reorientational solitary waves form as the nonlocal refractive index response to extraordinarily polarized light beams is both self-focusing via the induced rotation of the constituent molecules and self-defocusing owing to the temperature increase through optical absorption. These competing nonlinearities can lead to the formation of one- and two-dimensional multihumped solitary and ring-shaped waves at high enough optical powers with a volcano profile on the plane transverse to propagation. The variational solutions for these self-localized structured beams are in remarkably good agreement with full numerical solutions of the governing equations.

Phase-Field Approximation of Functionals Defined on Piecewise-Rigid Maps

We provide a variational approximation of Ambrosio–Tortorelli type for brittle fracture energies of piecewise-rigid solids. Our result covers both the case of geometrically nonlinear elasticity and that of linearised elasticity.

Modeling the absorption and refraction spectra of multiple quantum well structures

An algorithm for solving exciton Schrodinger equation in an arbitrary 2D multiple quantum well structure under applied electric field is proposed, allowing one to model both the absorption and refraction spectra consistently and efficiently. It gives much better quantitative results than variational approximation, while being faster than more advanced methods. The algorithm is implemented using R language and tested by comparing to other published results. Possible improvements are discussed.

Geometric Variational Inference.

Efficiently accessing the information contained in non-linear and high dimensional probability distributions remains a core challenge in modern statistics. Traditionally, estimators that go beyond point estimates are either categorized as Variational Inference (VI) or Markov-Chain Monte-Carlo (MCMC) techniques. While MCMC methods that utilize the geometric properties of continuous probability distributions to increase their efficiency have been proposed, VI methods rarely use the geometry. This work aims to fill this gap and proposes geometric Variational Inference (geoVI), a method based on Riemannian geometry and the Fisher information metric. It is used to construct a coordinate transformation that relates the Riemannian manifold associated with the metric to Euclidean space. The distribution, expressed in the coordinate system induced by the transformation, takes a particularly simple form that allows for an accurate variational approximation by a normal distribution. Furthermore, the algorithmic structure allows for an efficient implementation of geoVI which is demonstrated on multiple examples, ranging from low-dimensional illustrative ones to non-linear, hierarchical Bayesian inverse problems in thousands of dimensions.

Vector Soliton in Coupled Nonlinear Schrödinger Equation

Researchers are currently interested in studying the dynamics of the wave field in a nonlinear and dispersive medium. The Nonlinear Schrödinger Equation (NLSE), which is the fundamental equation that explains the phenomenon, has paved the way for research in a variety of fields, including soliton scattering. However, if the fields have a large number of components, the Coupled NLSE should be considered. We used orthogonally polarised and equal-amplitude vector solitons with two polarization directions to model the interactions. The effect of vector soliton scattering by external Delta potential in Coupled NLSE was studied in this paper. The scattering process is primarily determined by the initial velocity, amplitude of the soliton and potential strength. The variational approximation and direct numerical methods of Coupled NLSE were used to investigate the scattering process. The variational approximation (VA) method was used to analyse the dynamics of soliton’s width and center of mass position. The soliton may thus be reflected, transmitted or trapped within the potential. Uncoupled solitons may initially create a coupled state if their kinetic energy is less than the attractive interaction potential between solitons, but once their velocity surpasses the critical velocity, the soliton will easily pass through each other. To validate the approximation, a direct numerical simulation of CNLSE was performed. The results of the VA method and direct numerical simulation of Coupled NLSE are in good agreement when the parameters for both solutions are set to the same value. The initial velocity, potential strength and soliton amplitude play a role in the scattering of the vector soliton with Delta potential.

Interactions of Soliton in Weakly Nonlocal Nonlinear Media

Solitary waves or solitons is a nonlinear phenomenon which has been studied intensively due to its application in solid-state matter such as Bose-Einstein condensates state, plasma physics, optical fibers and nematic liquid crystal. In particular, the study of nonlinear phenomena occurs in the structure of waves gained interest of scholars since their discovery by John Russell in 1844. The Nonlinear Schrödinger Equation (NLSE) is the theoretical framework for the investigation of nonlinear pulse propagation in optical fibers. Nonlocality can be found in an underlying transport mechanisms or long-range forces like electrostatic interactions in liquid crystals and many-body interactions with matter waves in Bose-Einstein condensate or plasma waves. The length of optical beam width and length of response function are used to classify nonlocality in optical materials. The nonlocality can be categorized as weak nonlocal if the width of the optical beam broader than the length of response function and if the width of the optical beam is narrower than the length of response function, it is considered as highly nonlocal. This work investigates the interactions of solitons in a weakly nonlocal Cubic NLSE with Gaussian external potential. The variational approximation (VA) method was employed to solve non integrable NLSE to ordinary differential equation (ODE). The soliton parameters and the computational program are used to simulate the propagation of the soliton width and its center-of-mass position. In the presence of Gaussian external potential, the soliton may be transmitted, reflected or trapped based on the critical velocity and potential strength. Direct numerical simulation of Cubic NLSE is programmed to verify the results of approximation method. Good agreement is achieved between the direct numerical solution and VA method results.

Variational Bayes approximation of factor stochastic volatility models

Estimation and prediction in high dimensional multivariate factor stochastic volatility models is an important and active research area, because such models allow a parsimonious representation of multivariate stochastic volatility. Bayesian inference for factor stochastic volatility models is usually done by Markov chain Monte Carlo methods (often by particle Markov chain Monte Carlo methods), which are usually slow for high dimensional or long time series because of the large number of parameters and latent states involved. Our article makes two contributions. The first is to propose a fast and accurate variational Bayes methods to approximate the posterior distribution of the states and parameters in factor stochastic volatility models. The second is to extend this batch methodology to develop fast sequential variational updates for prediction as new observations arrive. The methods are applied to simulated and real datasets, and shown to produce good approximate inference and prediction compared to the latest particle Markov chain Monte Carlo approaches, but are much faster.

Fast and accurate variational inference for models with many latent variables

Models with a large number of latent variables are often used to utilize the information in big or complex data, but can be difficult to estimate. Variational inference methods provide an attractive solution. These methods use an approximation to the posterior density, yet for large latent variable models existing choices can be inaccurate or slow to calibrate. Here, we propose a family of tractable variational approximations that are more accurate and faster to calibrate for this case. It combines a parsimonious approximation for the parameter posterior with the exact conditional posterior of the latent variables. We derive a simplified expression for the re-parameterization gradient of the variational lower bound, which is the main ingredient of optimization algorithms used for calibration. Implementation only requires exact or approximate generation from the conditional posterior of the latent variables, rather than computation of their density. In effect, our method provides a new way to employ Markov chain Monte Carlo (MCMC) within variational inference. We illustrate using two complex contemporary econometric examples. The first is a nonlinear multivariate state space model for U.S. macroeconomic variables. The second is a random coefficients tobit model applied to two million sales by 20,000 individuals in a consumer panel. In both cases, our approximating family is considerably more accurate than mean field or structured Gaussian approximations, and faster than MCMC. Last, we show how to implement data sub-sampling in variational inference for our approximation, further reducing computation time. MATLAB code implementing the method is provided.

Predicting Conversion from MCI to AD Combining Multi-Modality Data and Based on Molecular Subtype.

Alzheimer’s disease (AD) is a neurodegenerative brain disease in the elderly. Identifying patients with mild cognitive impairment (MCI) who are more likely to progress to AD is a key step in AD prevention. Recent studies have shown that AD is a heterogeneous disease. In this study, we propose a subtyping-based prediction strategy to predict the conversion from MCI to AD in three years according to MCI patient subtypes. Structural magnetic resonance imaging (sMRI) data and multi-omics data, including genotype data and gene expression profiling derived from peripheral blood samples, from 125 MCI patients were used in the Alzheimer’s Disease Neuroimaging Initiative (ADNI)-1 dataset and from 98 MCI patients in the ADNI-GO/2 dataset. A variational Bayes approximation model based on the multiple kernel learning method was constructed to predict whether an MCI patient will progress to AD within three years. In internal fivefold cross-validation within ADNI-1, we achieved an overall AUC of 0.83 (79.20% accuracy, 81.25% sensitivity, 77.92% specificity) compared to the model without subtyping, which achieved an AUC of 0.78 (76.00% accuracy, 77.08% sensitivity, 75.32% specificity). In external validation using ADNI-1 as a training set and ADNI-GO/2 as an independent test set, we attained an AUC of 0.78 (74.49% accuracy, 74.19% sensitivity, 74.63% specificity). Identifying MCI patient subtypes with omics data would improve the accuracy of predicting the conversion from MCI to AD. In addition to evaluating statistics, obtaining the significant sMRI, single nucleotide polymorphism (SNP) and mRNA expression data from peripheral blood of MCI patients is noninvasive and cost-effective for predicting conversion from MCI to AD.

Rényi free energy and variational approximations to thermal states

We propose the construction of thermodynamic ensembles that minimize the R\'enyi free energy, as an alternative to Gibbs states. For large systems, the local properties of these R\'enyi ensembles coincide with those of thermal equilibrium, and they can be used as approximations to thermal states. We provide algorithms to find tensor network approximations to the 2-R\'enyi ensemble. In particular, a matrix-product-state representation can be found by using gradient-based optimization on Riemannian manifolds, or via a non-linear evolution which yields the desired state as a fixed point. We analyze the performance of the algorithms and the properties of the ensembles on one-dimensional spin chains.

Bayesian estimation of maximum entropy model for individualized energy landscape analysis of brain state dynamics.

The pairwise maximum entropy model (MEM) for resting state functional MRI (rsfMRI) has been used to generate energy landscape of brain states and to explore nonlinear brain state dynamics. Researches using MEM, however, has mostly been restricted to fixed‐effect group‐level analyses, using concatenated time series across individuals, due to the need for large samples in the parameter estimation of MEM. To mitigate the small sample problem in analyzing energy landscapes for individuals, we propose a Bayesian estimation of individual MEM using variational Bayes approximation (BMEM). We evaluated the performances of BMEM with respect to sample sizes and prior information using simulation. BMEM showed advantages over conventional maximum likelihood estimation in reliably estimating model parameters for individuals with small sample data, particularly utilizing the empirical priors derived from group data. We then analyzed individual rsfMRI of the Human Connectome Project to show the usefulness of MEM in differentiating individuals and in exploring neural correlates for human behavior. MEM and its energy landscape properties showed high subject specificity comparable to that of functional connectivity. Canonical correlation analysis identified canonical variables for MEM highly associated with cognitive scores. Inter‐individual variations of cognitive scores were also reflected in energy landscape properties such as energies, occupation times, and basin sizes at local minima. We conclude that BMEM provides an efficient method to characterize dynamic properties of individuals using energy landscape analysis of individual brain states.