High-dimensional Density Research Articles

Generative machine learning for detector response modeling with a conditional normalizing flow

In this paper, we explore the potential of generative machine learning models as an alternative to the computationally expensive Monte Carlo (MC) simulations commonly used by the Large Hadron Collider (LHC) experiments. Our objective is to develop a generative model capable of efficiently simulating detector responses for specific particle observables, focusing on the correlations between detector responses of different particles in the same event and accommodating asymmetric detector responses. We present a conditional normalizing flow model (𝒞𝒩ℱ) based on a chain of Masked Autoregressive Flows, which effectively incorporates conditional variables and models high-dimensional density distributions. We assess the performance of the 𝒞𝒩ℱ model using a simulated sample of Higgs boson decaying to diphoton events at the LHC. We create reconstruction-level observables using a smearing technique. We show that conditional normalizing flows can accurately model complex detector responses and their correlation. This method can potentially reduce the computational burden associated with generating large numbers of simulated events while ensuring that the generated events meet the requirements for data analyses. We make our code available at https://github.com/allixu/normalizing_flow_for_detector_response.

High-dimensional density estimation with tensorizing flow

High-dimensional density estimation with tensorizing flow

Adaptive deep density approximation for Fokker-Planck equations

In this paper we present an adaptive deep density approximation strategy based on KRnet (ADDA-KR) for solving the steady-state Fokker-Planck (F-P) equations. F-P equations are usually high-dimensional and defined on an unbounded domain, which limits the application of traditional grid based numerical methods. With the Knothe-Rosenblatt rearrangement, our newly proposed flow-based generative model, called KRnet, provides a family of probability density functions to serve as effective solution candidates for the Fokker-Planck equations, which has a weaker dependence on dimensionality than traditional computational approaches and can efficiently estimate general high-dimensional density functions. To obtain effective stochastic collocation points for the approximation of the F-P equation, we develop an adaptive sampling procedure, where samples are generated iteratively using the approximate density function at each iteration. We present a general framework of ADDA-KR, validate its accuracy and demonstrate its efficiency with numerical experiments.

Comparison and intelligent analysis of NTRU and ETRU signature algorithms for public key digital signature

Three American mathematicians made the NTRU public-key cryptosystem in 1996, it has a fast speed, small footprint, and also it is easy to produce key advantages. The NTRU signature algorithm is based on an integer base, the performance of the signature algorithm will change when the integer base becomes other bases. Based on the definition of “high-dimensional density” of lattice signatures, this paper chooses the ETRU signature algorithm formed by replacing the integer base with the Eisenstein integer base as a representative, and analyzes and compares the performance, security of NTRU and ETRU signature algorithms, SVP and CVP and other difficult issues, the speed of signature and verification, and the consumption of resources occupied by the algorithm.

Direct Tomography of High-Dimensional Density Matrices for General Quantum States of Photons.

Quantum-state tomography is the conventional method used to characterize density matrices for general quantum states. However, the data acquisition time generally scales linearly with the dimension of the Hilbert space, hindering the possibility of dynamic monitoring of a high-dimensional quantum system. Here, we demonstrate a direct tomography protocol to measure density matrices of photons in the position basis through the use of a polarization-resolving camera, where the dimension of density matrices can be as large as 580×580 in our experiment. The use of the polarization-resolving camera enables parallel measurements in the position and polarization basis and as a result, the data acquisition time of our protocol does not increase with the dimension of the Hilbert space and is solely determined by the camera exposure time (on the order of 10ms). Our method is potentially useful for the real-time monitoring of the dynamics of quantum states and paves the way for the development of high-dimensional, time-efficient quantum metrology techniques.

OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport

A normalizing flow is an invertible mapping between an arbitrary probability distribution and a standard normal distribution; it can be used for density estimation and statistical inference. Computing the flow follows the change of variables formula and thus requires invertibility of the mapping and an efficient way to compute the determinant of its Jacobian. To satisfy these requirements, normalizing flows typically consist of carefully chosen components. Continuous normalizing flows (CNFs) are mappings obtained by solving a neural ordinary differential equation (ODE). The neural ODE's dynamics can be chosen almost arbitrarily while ensuring invertibility. Moreover, the log-determinant of the flow's Jacobian can be obtained by integrating the trace of the dynamics' Jacobian along the flow. Our proposed OT-Flow approach tackles two critical computational challenges that limit a more widespread use of CNFs. First, OT-Flow leverages optimal transport (OT) theory to regularize the CNF and enforce straight trajectories that are easier to integrate. Second, OT-Flow features exact trace computation with time complexity equal to trace estimators used in existing CNFs. On five high-dimensional density estimation and generative modeling tasks, OT-Flow performs competitively to state-of-the-art CNFs while on average requiring one-fourth of the number of weights with an 8x speedup in training time and 24x speedup in inference.

Generation of Bures-Hall mixed states using coupled kicked tops

We simulate a system of coupled kicked tops to generate random density matrices distributed according to the Bures-Hall measure, which has an important role in quantum information theory. We study the effects of stochasticity and coupling parameters of the coupled-kicked-top system on the behavior of the associated bipartite pure state, eigenvalues and eigenvectors of the reduced state, and average entropies. For strongly chaotic phase space and adequate coupling between the constituting tops (subsystems), we find that the results of simulation agree with analytical results of random matrix theory. We also examine local fluctuation properties of the Bures-Hall eigenvalues using the distribution of nearest-neighbor spacing ratios. In the limit of high-dimensional density matrices, the empirical ratio distribution is found to approach the Wigner-surmise-like result for the Gaussian unitary ensemble. Additionally, we present some closed-form results for the Bures-Hall spectral density and corresponding moments using Meijer $G$ functions.

Vine Regression with Bayes Nets: A Critical Comparison with Traditional Approaches Based on a Case Study on the Effects of Breastfeeding on IQ.

Regular vines (R‐vines) copulas build high dimensional joint densities from arbitrary one‐dimensional margins and (conditional) bivariate copula densities. Vine densities enable the computation of all conditional distributions, though the calculations can be numerically intensive. Saturated continuous nonparametric Bayes nets (CNPBN) are regular vines. Computing regression functions from the vine copula density is termed vine regression. The epicycles of regression–including/excluding covariates, interactions, higher order terms, multicollinearity, model fit, transformations, heteroscedasticity, bias–are dispelled. One simply computes the regressions from the vine copula density. Only the question of finding an adequate vine copula remains. Vine regression is applied to a data set from the National Longitudinal Study of Youth relating breastfeeding to IQ. The expected effects of breastfeeding on IQ depend on IQ, on the baseline level of breastfeeding, on the duration of additional breastfeeding and on the values of other covariates. A child given two weeks breastfeeding can expect to increase his/her IQ by 1.5–2 IQ points by adding 10 weeks of breastfeeding, depending on values of other covariates. A child given two years breastfeeding can expect to gain from 0.48–0.65 IQ points from 10 additional weeks. Adding 10 weeks breastfeeding to each of the 3,179 children in this data set has a net present value $50,700,000 according to the Bayes net, compared to $29,000,000 according to the linear regression.

Unsupervised Domain Adaptation Method Based on Discriminant Sample Selection

In order to solve the problem that low classification accuracy caused by the different distribution of training set and test set, an unsupervised domain adaptation method based on discriminant sample selection (DSS) is proposed. DSS projects the samples of different domains onto a same subspace to reduce the distribution discrepancy between the source domain and the target domain, and weights the source domain instances to make the samples more discriminant. Different from the previous method based on the probability density estimation of samples, DSS tries to obtain the sample weights by solving a quadratic programming problem, which avoids the distribution estimation of samples and can be applied to any fields without suffering from the dimensional trouble caused by high-dimensional density estimation. Finally, DSS congregates the same classes by minimizing the intra-class distance. Experimental results show that the proposed method improves the classification accuracy and robustness.

Convergence rate analysis of Gaussian belief propagation for Markov networks

Gaussian belief propagation algorithm ( GaBP ) is one of the most important distributed algorithms in signal processing and statistical learning involving Markov networks. It is well known that the algorithm correctly computes marginal density functions from a high dimensional joint density function over a Markov network in a finite number of iterations when the underlying Gaussian graph is acyclic. It is also known more recently that the algorithm produces correct marginal means asymptotically for cyclic Gaussian graphs under the condition of walk summability ( or generalised diagonal dominance ) . This paper extends this convergence result further by showing that the convergence is exponential under the generalised diagonal dominance condition, and provides a simple bound for the convergence rate. Our results are derived by combining the known walk summability approach for asymptotic convergence analysis with the control systems approach for stability analysis.

An adversarial non-volume preserving flow model with Boltzmann priors

Flow-based generative models (flow models) are conceptually attractive due to tractability of the exact log-likelihood and the exact latent-variable inference. In order to generate sharper images and extend the Gaussian prior of Flow models to other discrete forms, we propose an adversarial non-volume preserving flow model with Boltzmann priors (ANVP) for modeling complex high-dimensional densities. In order to generate sharper images, an ANVP model introduces an adversarial regularizer into the loss function to penalize the condition that it places a high probability in regions where the training data distribution has a low density. Moreover, we show that the Gaussian prior can be extended to other forms such as the Boltzmann prior in the proposed ANVP model, and we use multi-scale transformations and Boltzmann priors to model the data distribution. The experiments show that proposed model is effective in image generation task.

Approximation and sampling of multivariate probability distributions in the tensor train decomposition

General multivariate distributions are notoriously expensive to sample from, particularly the high-dimensional posterior distributions in PDE-constrained inverse problems. This paper develops a sampler for arbitrary continuous multivariate distributions that is based on low-rank surrogates in the tensor train format, a methodology that has been exploited for many years for scalable, high-dimensional density function approximation in quantum physics and chemistry. We build upon recent developments of the cross approximation algorithms in linear algebra to construct a tensor train approximation to the target probability density function using a small number of function evaluations. For sufficiently smooth distributions, the storage required for accurate tensor train approximations is moderate, scaling linearly with dimension. In turn, the structure of the tensor train surrogate allows sampling by an efficient conditional distribution method since marginal distributions are computable with linear complexity in dimension. Expected values of non-smooth quantities of interest, with respect to the surrogate distribution, can be estimated using transformed independent uniformly-random seeds that provide Monte Carlo quadrature or transformed points from a quasi-Monte Carlo lattice to give more efficient quasi-Monte Carlo quadrature. Unbiased estimates may be calculated by correcting the transformed random seeds using a Metropolis–Hastings accept/reject step, while the quasi-Monte Carlo quadrature may be corrected either by a control-variate strategy or by importance weighting. We show that the error in the tensor train approximation propagates linearly into the Metropolis–Hastings rejection rate and the integrated autocorrelation time of the resulting Markov chain; thus, the integrated autocorrelation time may be made arbitrarily close to 1, implying that, asymptotic in sample size, the cost per effectively independent sample is one target density evaluation plus the cheap tensor train surrogate proposal that has linear cost with dimension. These methods are demonstrated in three computed examples: fitting failure time of shock absorbers; a PDE-constrained inverse diffusion problem; and sampling from the Rosenbrock distribution. The delayed rejection adaptive Metropolis (DRAM) algorithm is used as a benchmark. In all computed examples, the importance weight-corrected quasi-Monte Carlo quadrature performs best and is more efficient than DRAM by orders of magnitude across a wide range of approximation accuracies and sample sizes. Indeed, all the methods developed here significantly outperform DRAM in all computed examples.

Variational Autoencoder with Implicit Optimal Priors

The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the standard Gaussian distribution is usually used for the prior, this simple prior incurs over-regularization. As a sophisticated prior, the aggregated posterior has been introduced, which is the expectation of the posterior over the data distribution. This prior is optimal for the VAE in terms of maximizing the training objective function. However, KL divergence with the aggregated posterior cannot be calculated in a closed form, which prevents us from using this optimal prior. With the proposed method, we introduce the density ratio trick to estimate this KL divergence without modeling the aggregated posterior explicitly. Since the density ratio trick does not work well in high dimensions, we rewrite this KL divergence that contains the high-dimensional density ratio into the sum of the analytically calculable term and the lowdimensional density ratio term, to which the density ratio trick is applied. Experiments on various datasets show that the VAE with this implicit optimal prior achieves high density estimation performance.

Solving Bayesian inverse problems from the perspective of deep generative networks

Deep generative networks have achieved great success in high dimensional density approximation, especially for applications in natural images and language. In this paper, we investigate their approximation capability in capturing the posterior distribution in Bayesian inverse problems by learning a transport map. Because only the unnormalized density of the posterior is available, training methods that learn from posterior samples, such as variational autoencoders and generative adversarial networks, are not applicable in our setting. We propose a class of network training methods that can be combined with sample-based Bayesian inference algorithms, such as various MCMC algorithms, ensemble Kalman filter and Stein variational gradient descent. Our experiment results show the pros and cons of deep generative networks in Bayesian inverse problems. They also reveal the potential of our proposed methodology in capturing high dimensional probability distributions.

CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks

Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, which are prohibitively computationally expensive. The application of deep learning techniques to generative modeling is renewing interest in using high dimensional density estimators as computationally inexpensive emulators of fully-fledged simulations. These generative models have the potential to make a dramatic shift in the field of scientific simulations, but for that shift to happen we need to study the performance of such generators in the precision regime needed for science applications. To this end, in this work we apply Generative Adversarial Networks to the problem of generating weak lensing convergence maps. We show that our generator network produces maps that are described by, with high statistical confidence, the same summary statistics as the fully simulated maps.

Inference for the Lagged Cross‐Covariance Operator Between Functional Time Series

When considering two or more time series of functional data objects, for instance those derived from densely observed intraday stock price data of several companies, the empirical cross‐covariance operator is of fundamental importance due to its role in functional lagged regression and exploratory data analysis. Despite its relevance, statistical procedures for measuring the significance of such estimators are currently undeveloped. We present methodology based on a functional central limit theorem for conducting statistical inference for the cross‐covariance operator estimated between two stationary, weakly dependent, functional time series. Specifically, we consider testing the null hypothesis that the two series possess a specified cross‐covariance structure at a given lag. Since this test assumes that the series are jointly stationary, we also develop a change‐point detection procedure to validate this assumption of independent interest. The most imposing technical hurdle in implementing the proposed tests involves estimating the spectrum of a high dimensional spectral density operator at frequency zero. We propose a simple dimension reduction procedure based on functional principal component analysis to achieve this, which is shown to perform well in a simulation study. We illustrate the proposed methodology with an application to densely observed intraday price data of stocks listed on the New York stock exchange‐20.40

Investigation on Selective Laser Melting AlSi10Mg Cellular Lattice Strut: Molten Pool Morphology, Surface Roughness and Dimensional Accuracy.

AlSi10Mg inclined struts with angle of 45° were fabricated by selective laser melting (SLM) using different scanning speed and hatch spacing to gain insight into the evolution of the molten pool morphology, surface roughness, and dimensional accuracy. The results show that the average width and depth of the molten pool, the lower surface roughness and dimensional deviation decrease with the increase of scanning speed and hatch spacing. The upper surface roughness is found to be almost constant under different processing parameters. The width and depth of the molten pool on powder-supported zone are larger than that of the molten pool on the solid-supported zone, while the width changes more significantly than that of depth. However, if the scanning speed is high enough, the width and depth of the molten pool and the lower surface roughness almost keep constant as the density is still high. Therefore, high dimensional accuracy and density as well as good surface quality can be achieved simultaneously by using high scanning speed during SLMed cellular lattice strut.

Optimal large-scale quantum state tomography with Pauli measurements

Quantum state tomography aims to determine the state of a quantum system as represented by a density matrix. It is a fundamental task in modern scientific studies involving quantum systems. In this paper, we study estimation of high-dimensional density matrices based on Pauli measurements. In particular, under appropriate notion of sparsity, we establish the minimax optimal rates of convergence for estimation of the density matrix under both the spectral and Frobenius norm losses; and show how these rates can be achieved by a common thresholding approach. Numerical performance of the proposed estimator is also investigated.

Proximal Markov chain Monte Carlo algorithms

This paper presents a new Metropolis-adjusted Langevin algorithm (MALA) that uses convex analysis to simulate efficiently from high-dimensional densities that are log-concave, a class of probability distributions that is widely used in modern high-dimensional statistics and data analysis. The method is based on a new first-order approximation for Langevin diffusions that exploits log-concavity to construct Markov chains with favourable convergence properties. This approximation is closely related to Moreau---Yoshida regularisations for convex functions and uses proximity mappings instead of gradient mappings to approximate the continuous-time process. The proposed method complements existing MALA methods in two ways. First, the method is shown to have very robust stability properties and to converge geometrically for many target densities for which other MALA are not geometric, or only if the step size is sufficiently small. Second, the method can be applied to high-dimensional target densities that are not continuously differentiable, a class of distributions that is increasingly used in image processing and machine learning and that is beyond the scope of existing MALA and HMC algorithms. To use this method it is necessary to compute or to approximate efficiently the proximity mappings of the logarithm of the target density. For several popular models, including many Bayesian models used in modern signal and image processing and machine learning, this can be achieved with convex optimisation algorithms and with approximations based on proximal splitting techniques, which can be implemented in parallel. The proposed method is demonstrated on two challenging high-dimensional and non-differentiable models related to image resolution enhancement and low-rank matrix estimation that are not well addressed by existing MCMC methodology.

Sub-sample swapping for sequential Monte Carlo approximation of high-dimensional densities in the context of complex object tracking

In this paper, we address the problem of complex object tracking using the particle filter framework, which essentially amounts to estimate high-dimensional distributions by a sequential Monte Carlo algorithm. For this purpose, we first exploit Dynamic Bayesian Networks to determine conditionally independent subspaces of the object’s state space, which allows us to independently perform the particle filter’s propagations and corrections over small spaces. Second, we propose a swapping process to transform the weighted particle set provided by the update step of the particle filter into a “new particle set” better focusing on high peaks of the posterior distribution. This new methodology, called Swapping-Based Partitioned Sampling, is proved to be mathematically sound and is successfully tested and validated on synthetic video sequences for single or multiple articulated object tracking.