High-dimensional Distribution Research Articles

Performance evaluation of high-dimensional quantum key distribution with single-photon detector based on avalanche photodiode

The current theoretical analysis model for high-dimensional quantum key distribution (QKD) was developed without carefully considering the dead time and after-pulse of the single-photon detectors. Here, we propose a new model that accounts for these two effects. With this model, we analyze the effects of various properties of the single-photon detector on the performance of the high-dimensional quantum key distribution and show numerically that the high-dimensional quantum key distribution can be achieved up to 180 km with InGaAs/InP-type single-photon detectors. This work can improve our understanding of the factors that determine the performance of high-dimensional quantum key distribution and will help to promote the application of single-photon detectors based on avalanche photodiodes in high-dimensional quantum key distribution.

A unified performance analysis of likelihood-informed subspace methods

The likelihood-informed subspace (LIS) method offers a viable route to reducing the dimensionality of high-dimensional probability distributions arising in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of marginalization of the likelihood function, in which the approximated likelihood only has support on the intrinsic low-dimensional subspace. This approximation enables the design of inference algorithms that can scale sub-linearly with the apparent dimensionality of the problem. Intuitively, the accuracy of the approximation, and hence the performance of the inference algorithms, are influenced by three factors—the dimension truncation error in identifying the subspace, Monte Carlo error in estimating the Gram matrices, and Monte Carlo error in constructing marginalizations. This work establishes a unified framework to analyze each of these three factors and their interplay. Under mild technical assumptions, we establish error bounds for a range of existing dimension reduction techniques based on the principle of LIS. Our error bounds also provide useful insights into the accuracy of these methods. In addition, we analyze the integration of LIS with sampling methods such as Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo (SMC). We also demonstrate the applicability of our analysis on a linear inverse problem with Gaussian prior, which shows that all the estimates can be dimension-independent if the prior covariance is a trace-class operator. Finally, we demonstrate various aspects of our theoretical claims on two nonlinear inverse problems.

Allocation of flood drainage rights in the middle and lower reaches of the Yellow River based on deep learning and flood resilience

Climate change has increased the intensity and frequency of storms in many world regions, calling for new flood planning and management strategies. The concept of flood drainage rights (FDR), or the legal rights of regions to drain floodwaters into river reaches, is used in watershed planning in China. Quantifying the allocation of FDR remains challenging, where some previous methods have resulted in unreasonable or impractical allocation plans due to incomplete consideration of driving factors or the use of unscientific allocation methods. This study explores the allocation plan of FDR in the middle and lower reaches of the Yellow River Watershed in China. Climatic variability and change have caused frequent flooding in portions of the basin, with significant societal and economic implications. First, we comprehensively analyzed factors driving FDR for regions in the watershed. Following the conceptual flood resilience strategy currently being advocated for the region, we considered natural, socioeconomic, governance, resilience, and resistance factors that influence the complex allocation of FDR and established a qualitative indicator system to reflect the complexity of these driving factors. Second, we quantified FDR values for flood-prone regions in the middle and lower river reaches of this major river basin. We introduced a specific deep learning method, called the variational autoencoder (VAE) model, to quantify FDR allocation, providing a robust solution to the challenge of the multi-objective, high-dimensional, nonlinear, and non-normal distribution of factors driving FDR allocation. Next, using data from 2005 to 2019, this model was applied to the study area. The allocation of FDR (summing to 100%) across five flood-prone provinces of the watershed includes Inner Mongolia (9.36%), Shaanxi (10.00%), Shanxi (10.95%), Henan (32.58%), and Shandong (37.12%). Using the harmony evaluation method based on harmony theory, we compared the new VAE allocation method with three conventional allocation methods. The VAE allocation method has the highest degree of harmony, suggesting that it is the most practical for FDR allocation and is a reasonable allocation for flood management. Such FDR can work with water conservancy engineering facilities as part of a comprehensive management system toward protecting public health, minimizing economic losses, and preserving functions of development and natural lands within floodplains of the Yellow River.

Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems

Markov chain Monte Carlo (MCMC) methods form one of the algorithmic foundations of Bayesian inverse problems. The recent development of likelihood-informed subspace (LIS) methods offers a viable route to designing efficient MCMC methods for exploring high-dimensional posterior distributions via exploiting the intrinsic low-dimensional structure of the underlying inverse problem. However, existing LIS methods and the associated performance analysis often assume that the prior distribution is Gaussian. This assumption is limited for inverse problems aiming to promote sparsity in the parameter estimation, as heavy-tailed priors, e.g., Laplace distribution or the elastic net commonly used in Bayesian LASSO, are often needed in this case. To overcome this limitation, we consider a prior normalization technique that transforms any non-Gaussian (e.g. heavy-tailed) priors into standard Gaussian distributions, which makes it possible to implement LIS methods to accelerate MCMC sampling via such transformations. We also rigorously investigate the integration of such transformations with several MCMC methods for high-dimensional problems. Finally, we demonstrate various aspects of our theoretical claims on two nonlinear inverse problems.

Scalable implementation of (d+1) mutually unbiased bases for d -dimensional quantum key distribution

A high-dimensional quantum key distribution (QKD) can improve error rate tolerance and the secret key rate. Many $d$-dimensional QKDs have used two mutually unbiased bases (MUBs), while $(d+1)$ MUBs enable a more robust QKD, especially against correlated errors. However, a scalable implementation has not been achieved because the setups have required $d$ devices even for two MUBs or a flexible convertor for a specific optical mode. Here, we propose a scalable and general implementation of $(d+1)$ MUBs using $\log_p d$ interferometers in prime power dimensions $d=p^N$. We implemented the setup for time-bin states and observed an average error rate of 3.8% for phase bases, which is lower than the 23.17% required for a secure QKD against coherent attack in $d=4$.

Adaptive variational Hamiltonian Monte Carlo sampling with application to ocean acoustics

The Hamiltonian Monte Carlo (HMC) algorithm is a variant Markov Chain Monte Carlo (MCMC) excelling at sampling high dimensional target distributions. The HMC explores a distribution as a particle trajectory in a Hamiltonian system by augmenting distribution parameters with auxiliary momentum variables. A particle trajectory is numerically integrated from initial point to some distant low-correlation point. Projecting back to parameter space yields a proposal point with high acceptance probability leading to faster exploration of the target distribution. The Leap-frog integrator is commonly used for its volume preservation (symplecticity) and reversibility traits that maintain the detailed balance condition of MCMC. Although effective, Leap-frog has sensitivity to choice of integration step-size and number of integration steps. Poor choices cause slow exploration or bias when regions of the distribution are not explored. Adaptive step methods could remove step size issues and explore regions of a distribution where Leap-frog fails. Existing adaptive methods self-adjust step-size while maintaining symplecticity and reversibility. In this work, we apply adaptive step methods to simulated target distributions and real world ocean acoustic data and compare performance. [Work supported by Office of Naval Research.]

Convergence-Guaranteed Parametric Bayesian Distributed Cooperative Localization

Belief propagation (BP) is a popular message passing algorithm for distributed cooperative localization. However, due to the nonlinearity of measurement functions, BP implementation has no closed-form expression and requires message approximations. While nonparametric BP can be used, it suffers from a high computational complexity, thus being impractical in energy-constrained networks. In this paper, a parametric Bayesian method with Gaussian BP implementation is proposed for distributed cooperative localization. With linearization of the Euclidean norm in ranging measurements, the joint posterior distribution of agents’ locations is successively approximated with a sequence of high-dimensional Gaussian distributions. At each iteration of the successive Gaussian approximation, vector-valued Gaussian BP is further adopted to compute the marginal distributions of agents’ locations in a distributed way. It is proved by the principle of majorization-minimization that the proposed successive Gaussian approximation is guaranteed to converge, and the sequence of the estimated agents’ locations converges to a stationary point of the objective function of the maximum a posteriori estimation. Furthermore, although cooperative localization involves loopy network topologies, in which convergence property of Gaussian BP is generally unknown, it is proved in this paper that vector-valued Gaussian BP converges, making the proposed parametric BP-based method being the first one achieving convergence guarantee. Compared to the nonparametric BP counterpart, the proposed method has a much lower computational complexity and communication overhead. Simulation results demonstrate that the proposed method achieves a superior performance in localization accuracy compared to existing cooperative localization methods.

Adaptive random neighbourhood informed Markov chain Monte Carlo for high-dimensional Bayesian variable selection

We introduce a framework for efficient Markov chain Monte Carlo algorithms targeting discrete-valued high-dimensional distributions, such as posterior distributions in Bayesian variable selection problems. We show that many recently introduced algorithms, such as the locally informed sampler of Zanella (J Am Stat Assoc 115(530):852–865, 2020), the locally informed with thresholded proposal of Zhou et al. (Dimension-free mixing for high-dimensional Bayesian variable selection, 2021) and the adaptively scaled individual adaptation sampler of Griffin et al. (Biometrika 108(1):53–69, 2021), can be viewed as particular cases within the framework. We then describe a novel algorithm, the adaptive random neighbourhood informed sampler, which combines ideas from these existing approaches. We show using several examples of both real and simulated data-sets that a computationally efficient point-wise implementation (PARNI) provides more reliable inferences on a range of variable selection problems, particularly in the very large p setting.

Security analysis for a mutually partially unbiased bases–based protocol

The recently proposed mutually partially unbiased bases (MPUB)–based protocol, which encodes with Laguerre–Gaussian modes and Hermite–Gaussian modes of the same mode order, can close the security loophole caused by state-dependent diffraction. However, its pessimistic security proof limits the performance, and some practical issues, such as finite-key size and imperfect sources, have not been considered. Here, we improve the key rates of the MPUB-based protocol by accurately estimating the phase error rate. Moreover, the effect of finite-key size and its performance when combined with the decoy state method are demonstrated. Our work broadens the application scope of the MPUB-based protocol, and thereby advances the development of high-dimensional quantum key distribution using spatial modes.

Normalizing flows for likelihood-free inference with fusion simulations

Fluid-based scrape-off layer transport codes, such as UEDGE, are heavily utilized in tokamak analysis and design, but typically require user-specified anomalous transport coefficients to match experiments. Determining the uniqueness of these parameters and the uncertainties in them to match experiments can provide valuable insights to fusion scientists. We leverage recent work in the area of likelihood-free inference (‘simulation-based inference’) to train a neural network, which enables accurate statistical inference of the anomalous transport coefficients given experimental plasma profile input. UEDGE is treated as a black-box simulator and runs multiple times with anomalous transport coefficients sampled from priors, and the neural network is trained on these simulations to emulate the posterior. The neural network is trained as a normalizing flow model for density estimation, allowing it to accurately represent complicated, high-dimensional distribution functions. With a fixed simulation budget, we compare a single-round procedure to a multi-round approach that guides the training simulations toward a specific target observation. We discuss the future possibilities for use of amortized models, which train on a wide range of simulations and enable fast statistical inference for results during experiments.

A Generative Approach to Materials Discovery, Design, and Optimization.

Despite its potential to transform society, materials research suffers from a major drawback: its long research timeline. Recently, machine-learning techniques have emerged as a viable solution to this drawback and have shown accuracies comparable to other computational techniques like density functional theory (DFT) at a fraction of the computational time. One particular class of machine-learning models, known as “generative models”, is of particular interest owing to its ability to approximate high-dimensional probability distribution functions, which in turn can be used to generate novel data such as molecular structures by sampling these approximated probability distribution functions. This review article aims to provide an in-depth understanding of the underlying mathematical principles of popular generative models such as recurrent neural networks, variational autoencoders, and generative adversarial networks and discuss their state-of-the-art applications in the domains of biomaterials and organic drug-like materials, energy materials, and structural materials. Here, we discuss a broad range of applications of these models spanning from the discovery of drugs that treat cancer to finding the first room-temperature superconductor and from the discovery and optimization of battery and photovoltaic materials to the optimization of high-entropy alloys. We conclude by presenting a brief outlook of the major challenges that lie ahead for the mainstream usage of these models for materials research.

An efficient adaptive MCMC algorithm for Pseudo-Bayesian quantum tomography

We revisit the Pseudo-Bayesian approach to the problem of estimating density matrix in quantum state tomography in this paper. Pseudo-Bayesian inference has been shown to offer a powerful paradigm for quantum tomography with attractive theoretical and empirical results. However, the computation of (Pseudo-)Bayesian estimators, due to sampling from complex and high-dimensional distribution, pose significant challenges that hamper their usages in practical settings. To overcome this problem, we present an efficient adaptive MCMC sampling method for the Pseudo-Bayesian estimator by exploring an adaptive proposal scheme together with subsampling method. We show in simulations that our approach is substantially computationally faster than the previous implementation by at least two orders of magnitude which is significant for practical quantum tomography.

Hidden Markov Pólya Trees for High-Dimensional Distributions

The Pólya tree (PT) process is a general-purpose Bayesian nonparametric model that has found wide application in a range of inference problems. It has a simple analytic form and the posterior computation boils down to beta-binomial conjugate updates along a partition tree over the sample space. Recent development in PT models shows that performance of these models can be substantially improved by (i) allowing the partition tree to adapt to the structure of the underlying distributions and (ii) incorporating latent state variables that characterize local features of the underlying distributions. However, important limitations of the PT remain, including (i) the sensitivity in the posterior inference with respect to the choice of the partition tree, and (ii) the lack of scalability with respect to dimensionality of the sample space. We consider a modeling strategy for PT models that incorporates a flexible prior on the partition tree along with latent states with Markov dependency. We introduce a hybrid algorithm combining sequential Monte Carlo (SMC) and recursive message passing for posterior sampling that can scale up to 100 dimensions. While our description of the algorithm assumes a single computer environment, it has the potential to be implemented on distributed systems to further enhance the scalability. Moreover, we investigate the large sample properties of the tree structures and latent states under the posterior model. We carry out extensive numerical experiments in density estimation and two-group comparison, which show that flexible partitioning can substantially improve the performance of PT models in both inference tasks. We demonstrate an application to a mass cytometry dataset with 19 dimensions and over 200,000 observations. Supplementary Materials for this article are available online.

De Novo Molecule Design Using Molecular Generative Models Constrained by Ligand-Protein Interactions.

In recent years, molecular deep generative models have attracted much attention for its application in de novo drug design. The data-driven molecular deep generative model approximates the high dimensional distribution of the chemical space through learning from a large number of molecular structural data. So far, most of the molecular generative models rely on purely 2D ligand information in structure generation. Here, we propose a novel molecular deep generative model which adopts a recurrent neural network architecture coupled with a ligand-protein interaction fingerprint as constraints. The fingerprint was constructed on ligand docking poses and represents the 3D binding mode of ligands in the protein pocket. In the current work, generative models constrained with interaction fingerprints were trained and compared with normal RNN models. It has been shown that models trained with constraints of ligand-protein interaction fingerprint have a clear tendency to generating compounds maintaining similar binding modes. Our results demonstrate the potential application of the interaction fingerprint-constrained generative model for the targeted molecule generation and guided exploration on the drug-like chemical space.

A high dimensional dissimilarity measure

A new dissimilarity measure for high-dimensional, low sample size settings to compare high dimensional probability distributions is proposed. The asymptotic behavior of the new dissimilarity index is studied theoretically. Numerical experiments from high dimensional distributions exhibit the usefulness of the method. The eigenvalues of the matrix of dissimilarities for comparing two high dimensional samples are determined and shown to be related to the asymptotic value of the dissimilarity index. A dissimilarity visualization plot that is useful for detection of outliers and change points is proposed and utilized to find the change points in S&P500 stock return data.

Optimizing Variational Representations of Divergences and Accelerating Their Statistical Estimation

Variational representations of divergences and distances between high-dimensional probability distributions offer significant theoretical insights and practical advantages in numerous research areas. Recently, they have gained popularity in machine learning as a tractable and scalable approach for training probabilistic models and for statistically differentiating between data distributions. Their advantages include: 1) They can be estimated from data as statistical averages. 2) Such representations can leverage the ability of neural networks to efficiently approximate optimal solutions in function spaces. However, a systematic and practical approach to improving the tightness of such variational formulas, and accordingly accelerate statistical learning and estimation from data, is currently lacking. Here we develop such a methodology for building new, tighter variational representations of divergences. Our approach relies on improved objective functionals constructed via an auxiliary optimization problem. Furthermore, the calculation of the functional Hessian of objective functionals unveils the local curvature differences around the common optimal variational solution; this quantifies and orders the tightness gains between different variational representations. Finally, numerical simulations utilizing neural network optimization demonstrate that tighter representations can result in significantly faster learning and more accurate estimation of divergences in both synthetic and real datasets (of more than 1000 dimensions), often accelerated by nearly an order of magnitude.

I-SEA: Importance Sampling and Expected Alignment-Based Deep Distance Metric Learning for Time Series Analysis and Embedding

Learning effective embeddings for potentially irregularly sampled time-series, evolving at different time scales, is fundamental for machine learning tasks such as classification and clustering. Task-dependent embeddings rely on similarities between data samples to learn effective geometries. However, many popular time-series similarity measures are not valid distance metrics, and as a result they do not reliably capture the intricate relationships between the multi-variate time-series data samples for learning effective embeddings. One of the primary ways to formulate an accurate distance metric is by forming distance estimates via Monte-Carlo-based expectation evaluations. However, the high-dimensionality of the underlying distribution, and the inability to sample from it, pose significant challenges. To this end, we develop an Importance Sampling based distance metric -- I-SEA -- which enjoys the properties of a metric while consistently achieving superior performance for machine learning tasks such as classification and representation learning. I-SEA leverages Importance Sampling and Non-parametric Density Estimation to adaptively estimate distances, enabling implicit estimation from the underlying high-dimensional distribution, resulting in improved accuracy and reduced variance. We theoretically establish the properties of I-SEA and demonstrate its capabilities via experimental evaluations on real-world healthcare datasets.

Data-driven optimization of reliability using buffered failure probability

Design and operation of complex engineering systems rely on reliability optimization. Such optimization requires us to account for uncertainties expressed in terms of complicated, high-dimensional probability distributions, for which only samples or data might be available. However, using data or samples often degrades the computational efficiency, particularly as the conventional failure probability is estimated using the indicator function whose gradient is not defined at zero. To address this issue, by leveraging the buffered failure probability, the paper develops the buffered optimization and reliability method (BORM) for efficient, data-driven optimization of reliability. The proposed formulations, algorithms, and strategies greatly improve the computational efficiency of the optimization and thereby address the needs of high-dimensional and nonlinear problems. In addition, an analytical formula is developed to estimate the reliability sensitivity, a subject fraught with difficulty when using the conventional failure probability. The buffered failure probability is thoroughly investigated in the context of many different distributions, leading to a novel measure of tail-heaviness called the buffered tail index. The efficiency and accuracy of the proposed optimization methodology are demonstrated by three numerical examples. Although they might show slight deviations from a target failure probability because of sampling errors and other inaccuracies, the results underline unique advantages and potentials of the buffered failure probability for data-driven reliability analysis.

Inverse design of spontaneous parametric downconversion for generation of high-dimensional qudits

Spontaneous parametric downconversion (SPDC) in quantum optics is an invaluable resource for the realization of high-dimensional qudits with spatial modes of light. One of the main open challenges is how to directly generate a desirable qudit state in the SPDC process. This problem can be addressed through advanced computational learning methods; however, due to difficulties in modeling the SPDC process by a fully differentiable algorithm, progress has been limited. Here, we overcome these limitations and introduce a physically constrained and differentiable model, validated against experimental results for shaped pump beams and structured crystals, capable of learning the relevant interaction parameters in the process. We avoid any restrictions induced by the stochastic nature of our physical model and integrate the dynamic equations governing the evolution under the SPDC Hamiltonian. We solve the inverse problem of designing a nonlinear quantum optical system that achieves the desired quantum state of downconverted photon pairs. The desired states are defined using either the second-order correlations between different spatial modes or by specifying the required density matrix. By learning nonlinear photonic crystal structures as well as different pump shapes, we successfully show how to generate maximally entangled states. Furthermore, we simulate all-optical coherent control over the generated quantum state by actively changing the profile of the pump beam. Our work can be useful for applications such as novel designs of high-dimensional quantum key distribution and quantum information processing protocols. In addition, our method can be readily applied for controlling other degrees of freedom of light in the SPDC process, such as spectral and temporal properties, and may even be used in condensed-matter systems having a similar interaction Hamiltonian.

Understanding Uncertainties in Tropical Cyclone Rainfall Hazard Modeling Using Synthetic Storms

Abstract Tropical cyclone (TC) rainfall hazard assessment is subject to the bias in TC climatology estimation from climate simulations or synthetic downscaling. In this study, we investigate the uncertainty in TC rainfall hazard assessment induced by this bias using both rain gauge and radar observations and synthetic-storm-model-coupled TC rainfall simulations. We identify the storm’s maximum intensity, impact duration, and minimal distance to the site to be the three most important storm parameters for TC rainfall hazard, and the relationship between the important storm parameters and TC rainfall can be well captured by a physics-based TC rainfall model. The uncertainty in the synthetic rainfall hazard induced by the bias in TC climatology can be largely explained by the bias in the important storm parameters simulated by the synthetic storm model. Correcting the distribution of the most biased parameter may significantly improve rainfall hazard estimation. Bias correction based on the joint distribution of the important parameters may render more accurate rainfall hazard estimations; however, the general technical difficulties in resampling from high-dimensional joint probability distributions prevent more accurate estimations in some cases. The results of the study also support future investigation of the impact of climate change on TC rainfall hazards through the lens of future changes in the identified important storm parameters.