General Multivariate Distributions Research Articles

Interval-based reasoning over continuous variables using independent component analysis and Bayesian networks

Bayesian networks (BNs) can be used for probabilistic reasoning over discrete variables. Their extension to the continuous domain, however, remains an open research issue. In this work, we address the Bayesian inference problem for continuous variables when negative evidence is available. This situation is faced when, because of unreliable or imperfect sensors, some variables are observed to be at specific intervals. We propose two solutions to this issue. The first one, dubbed dynamic discrete BN (DD-BN), is simply an adaptation of the dynamic discretization method; the second, referred to as SICA, is based on spline functions and the independent component analysis (ICA) method. SICA can represent a wide range of general multivariate distributions and possesses some integration characteristics that facilitate the inference process. Empirical trials on public datasets and simulated data yielded promising results regarding inference time and accuracy. Indeed, SICA achieved the highest accuracy at the expense of increased inference time with increasing dimensions.

Representative Points Based on Power Exponential Kernel Discrepancy

Representative points (rep-points) are a set of points that are optimally chosen for representing a big original data set or a target distribution in terms of a statistical criterion, such as mean square error and discrepancy. Most of the existing criteria can only assure the representing properties in the whole variable space. In this paper, a new kernel discrepancy, named power exponential kernel discrepancy (PEKD), is proposed to measure the representativeness of the point set with respect to the general multivariate distribution. Different from the commonly used criteria, PEKD can improve the projection properties of the point set, which is important in high-dimensional circumstances. Some theoretical results are presented for understanding the new discrepancy better and guiding the hyperparameter setting. An efficient algorithm for searching rep-points under the PEKD criterion is presented and its convergence has also been proven. Examples are given to illustrate its potential applications in the numerical integration, uncertainty propagation, and reduction of Markov Chain Monte Carlo chains.

Multivariate goodness-of-fit tests based on Wasserstein distance

Goodness-of-fit tests based on the empirical Wasserstein distance are proposed for simple and composite null hypotheses involving general multivariate distributions. For group families, the procedure is to be implemented after preliminary reduction of the data via invariance. This property allows for calculation of exact critical values and p-values at finite sample sizes. Applications include testing for location–scale families and testing for families arising from affine transformations, such as elliptical distributions with given standard radial density and unspecified location vector and scatter matrix. A novel test for multivariate normality with unspecified mean vector and covariance matrix arises as a special case. For more general parametric families, we propose a parametric bootstrap procedure to calculate critical values. The lack of asymptotic distribution theory for the empirical Wasserstein distance means that the validity of the parametric bootstrap under the null hypothesis remains a conjecture. Nevertheless, we show that the test is consistent against fixed alternatives. To this end, we prove a uniform law of large numbers for the empirical distribution in Wasserstein distance, where the uniformity is over any class of underlying distributions satisfying a uniform integrability condition but no additional moment assumptions. The calculation of test statistics boils down to solving the well-studied semi-discrete optimal transport problem. Extensive numerical experiments demonstrate the practical feasibility and the excellent performance of the proposed tests for the Wasserstein distance of order p=1 and p=2 and for dimensions at least up to d=5. The simulations also lend support to the conjecture of the asymptotic validity of the parametric bootstrap.

Multivariate Classes of GB2 Distributions with Applications

The general beta of the second kind distribution (GB2) is a flexible distribution which includes several relevant parametric families of distributions. This distribution has important applications in earnings and income distributions, finance and insurance. In this paper, several multivariate classes of the GB2 distribution are proposed. The different multivariate versions are based on two simple univariate representations of the GB2 distribution. The first type of multivariate distributions are constructed from a stochastic dependent representations defined in terms of gamma random variables. Using this representation and beginning by two particular multivariate GB2 distributions, multivariate Singh–Maddala and Dagum income distributions are presented and several properties are obtained. Then, a general multivariate GB2 distribution is introduced. The second type of multivariate distributions are based on a generalization of the distribution of the order statistics, which gives place to multivariate GB2 distribution with support above the diagonal. We discuss the role of these families in modeling bivariate income distributions. Finally, an empirical application is given, where we show that a multivariate GB2 distribution can be useful for modeling compound precipitation and wind events in the whole range.

Constructing a multivariate distribution function with a vine copula: towards multivariate luminosity and mass functions

ABSTRACTThe need for a method to construct multidimensional distribution function is increasing recently, in the era of huge multiwavelength surveys. We have proposed a systematic method to build a bivariate luminosity or mass function of galaxies by using a copula. It allows us to construct a distribution function when only its marginal distributions are known, and we have to estimate the dependence structure from data. A typical example is the situation that we have univariate luminosity functions at some wavelengths for a survey, but the joint distribution is unknown. Main limitation of the copula method is that it is not easy to extend a joint function to higher dimensions (d > 2), except some special cases like multidimensional Gaussian. Even if we find such a multivariate analytic function in some fortunate case, it would often be inflexible and impractical. In this work, we show a systematic method to extend the copula method to unlimitedly higher dimensions by a vine copula. This is based on the pair-copula decomposition of a general multivariate distribution. We show how the vine copula construction is flexible and extendable. We also present an example of the construction of a stellar mass–atomic gas–molecular gas three-dimensional mass function. We demonstrate the maximum likelihood estimation of the best functional form for this function, as well as a proper model selection via vine copula.

Approximation and sampling of multivariate probability distributions in the tensor train decomposition

General multivariate distributions are notoriously expensive to sample from, particularly the high-dimensional posterior distributions in PDE-constrained inverse problems. This paper develops a sampler for arbitrary continuous multivariate distributions that is based on low-rank surrogates in the tensor train format, a methodology that has been exploited for many years for scalable, high-dimensional density function approximation in quantum physics and chemistry. We build upon recent developments of the cross approximation algorithms in linear algebra to construct a tensor train approximation to the target probability density function using a small number of function evaluations. For sufficiently smooth distributions, the storage required for accurate tensor train approximations is moderate, scaling linearly with dimension. In turn, the structure of the tensor train surrogate allows sampling by an efficient conditional distribution method since marginal distributions are computable with linear complexity in dimension. Expected values of non-smooth quantities of interest, with respect to the surrogate distribution, can be estimated using transformed independent uniformly-random seeds that provide Monte Carlo quadrature or transformed points from a quasi-Monte Carlo lattice to give more efficient quasi-Monte Carlo quadrature. Unbiased estimates may be calculated by correcting the transformed random seeds using a Metropolis–Hastings accept/reject step, while the quasi-Monte Carlo quadrature may be corrected either by a control-variate strategy or by importance weighting. We show that the error in the tensor train approximation propagates linearly into the Metropolis–Hastings rejection rate and the integrated autocorrelation time of the resulting Markov chain; thus, the integrated autocorrelation time may be made arbitrarily close to 1, implying that, asymptotic in sample size, the cost per effectively independent sample is one target density evaluation plus the cheap tensor train surrogate proposal that has linear cost with dimension. These methods are demonstrated in three computed examples: fitting failure time of shock absorbers; a PDE-constrained inverse diffusion problem; and sampling from the Rosenbrock distribution. The delayed rejection adaptive Metropolis (DRAM) algorithm is used as a benchmark. In all computed examples, the importance weight-corrected quasi-Monte Carlo quadrature performs best and is more efficient than DRAM by orders of magnitude across a wide range of approximation accuracies and sample sizes. Indeed, all the methods developed here significantly outperform DRAM in all computed examples.

Face recognition based on texture information and geodesic distance approximations between multivariate normal distributions

Geodesic distance is a natural dissimilarity measure between probability distributions of a specific type, and can be used to discriminate texture in image-based measurements. Furthermore, since there is no known closed-form solution for the geodesic distance between general multivariate normal distributions, we propose two efficient approximations to be used as texture dissimilarity metrics in the context of face recognition. A novel face recognition approach based on texture discrimination in high-resolution color face images is proposed, unlike the typical appearance-based approach that relies on low-resolution grayscale face images. In our face recognition approach, sparse facial features are extracted using predefined landmark topologies, that identify discriminative image locations on the face images. Given this landmark topology, the dissimilarity between distinct face images are scored in terms of the dissimilarities between their corresponding face landmarks, and the texture in each one of these landmarks is represented by multivariate normal distributions, expressing the color distribution in the vicinity of each landmark location. The classification of new face image samples occurs by determining the face image sample in the training set which minimizes the dissimilarity score, using the nearest neighbor rule. The proposed face recognition method was compared to methods representative of the state-of-the-art, using color or grayscale face images, and presented higher recognition rates. Moreover, the proposed texture dissimilarity metric also is efficient in general texture discrimination (e.g. texture recognition of material images), as our experiments suggest.

The Multivariate Generalised von Mises Distribution: Inference and Applications

Circular variables arise in a multitude of data-modelling contexts ranging from robotics to the social sciences, but they have been largely overlooked by the machine learning community. This paper partially redresses this imbalance by extending some standard probabilistic modelling tools to the circular domain. First we introduce a new multivariate distribution over circular variables, called the multivariate Generalised von Mises (mGvM) distribution. This distribution can be constructed by restricting and renormalising a general multivariate Gaussian distribution to the unit hyper-torus. Previously proposed multivariate circular distributions are shown to be special cases of this construction. Second, we introduce a new probabilistic model for circular regression inspired by Gaussian Processes, and a method for probabilistic Principal Component Analysis with circular hidden variables. These models can leverage standard modelling tools (e.g. kernel functions and automatic relevance determination). Third, we show that the posterior distribution in these models is a mGvM distribution which enables development of an efficient variational free-energy scheme for performing approximate inference and approximate maximum-likelihood learning.

Valuation of projects with minimum revenue guarantees: A Gaussian copula–based simulation approach

ABSTRACTThis technical note presents a numerical simulation technique to perform valuations of infrastructure projects with minimum revenue guarantees (MRG). It is assumed that the project cash flows—in the absence of the MRG—can be described in a probabilistic fashion by means of a very general multivariate distribution function. Then, the Gaussian copula (a numerical algorithm to generate vectors according to a prespecified probabilistic characterization) is used in combination with the MRG condition to generate a set of plausible cash flow vectors. These vectors form the basis of a Monte Carlo simulation that offers two important advantages: it is easy to implement and it makes no restrictive assumptions regarding the evolution of the cash flows over time. Thus, one can estimate the distribution of a broad set of metrics (net present value, internal rate of return, payback periods, etc.). Additionally, the method does not have any of the typical limitations of real options–based approaches, namely, cash flows that follow a Brownian motion or some specific diffusion process or whose volatility needs to be constant. The usefulness of the proposed approach is demonstrated with a simple example.

Bayesian analysis of multivariate stable distributions using one-dimensional projections

In this paper we take up Bayesian inference in general multivariate stable distributions. We exploit the representation of Matsui and Takemura (2009) for univariate projections, and the representation of the distributions in terms of their spectral measure. We present efficient MCMC schemes to perform the computations when the spectral measure is approximated discretely or, as we propose, by a normal distribution. Appropriate latent variables are introduced to implement MCMC. In relation to the discrete approximation, we propose efficient computational schemes based on the characteristic function.

Multivariate Weibull and Pareto Distributions in Hilbert Space

Two general multivariate distributions in a real separable Hilbert space H are introduced in this article, one is multivariate Weibull distribution (denoted by GMWH), the other is multivariate Pareto distribution (denoted by GMPH). They are more general than the existing references. Some characterization theorems of the GMWH and GMPH via an intensively monotone operator are proved. The limiting behaviors and the interrelationship between the GMW and GMP in Euclidean space are also studied.

Optimal Reliability Path Problem on Networks with Random and Correlated Link Travel Times

This study addressed the most reliable path problem on stochastic networks with correlated link travel times from general multivariate distributions. Both evaluation and optimization of this problem are difficult because of the absence of closed-form distribution at the path level. For tractability in reliability evaluation, the path reliability objective function was substituted by the standard-score objective with the same mean and variance as the given distribution. However, this simplified proxy objective was difficult to optimize because it was nonlinear in path mean and variance. These difficulties were circumvented by reformulating the problem as a bilevel minimization model, in which the master problem corresponded to the proxy reliability objective, and the subproblem corresponded to the solution of a related minimum robust-cost path problem. The subproblem was easier to solve because it minimized a linear combination of path travel time mean and standard deviation. An effi-cient algorithm was proposed to solve this bilevel model, whose exact and monotonic convergence was established for the proxy objective function. Computational experiments on various test networks demonstrated the robustness of the algorithm under various travel time distributions. The algorithm identified the optimal path in 96% to 99% of the cases with a maximum error of less than 0.5%. The algorithm was also applied with real-world data from an intelligent transportation systems test bed in Chennai, India. The results showed that the most reliable paths varied within the day and from day to day; this result showed potential applications of the algorithm with advanced traveler information systems.

Bayesian Analysis of Multivariate Stable Distributions Using One-Dimensional Projections

In this paper we take up Bayesian inference in general multivariate stable distributions. We exploit the representation of Matsui and Takemura (2009) for univariate projections, and the representation of the distributions in terms of their spectral measure. We present efficient MCMC schemes to perform the computations when the spectral measure is approximated discretely or, as we propose, by a normal distribution. Appropriate latent variables are introduced to implement MCMC. In relation to the discrete approximation, we propose efficient computational schemes based on the characteristic function.

Central limit theorem and influence function for the MCD estimators at general multivariate distributions

We define the minimum covariance determinant functionals for multivariate location and scatter through trimming functions and establish their existence at any multivariate distribution. We provide a precise characterization including a separating ellipsoid property and prove that the functionals are continuous. Moreover, we establish asymptotic normality for both the location and covariance estimator and derive the influence function. These results are obtained in a very general multivariate setting.

Characterizations of the General Multivariate Weibull Distributions

A general multivariate Weibull distribution is introduced in this article. The GMW is constructed by a well-defined exponent Radon measure which is satisfied by a functional equation with some assumptions. There is rarely literature published regarding the study of characterization of the multivariate Weibull distribution. Yeh (2009) investigated two characterizations on homogeneous multivariate semi-Weibull distribution. Many characterization theorems of the GMW are proved in this article. All these characterizations lead to a multivariate functional equation. Finally, the limiting distribution of the normalized geometric minima of the GMW is discerned as the general multivariate Pareto distribution.

Marginal Distributions of Random Vectors Generated by Affine Transformations of Independent Two-Piece Normal Variables

Marginal probability density and cumulative distribution functions are presented for multidimensional variables defined by nonsingular affine transformations of vectors of independent two-piece normal variables, the most important subclass of Ferreira and Steel's general multivariate skewed distributions. The marginal functions are obtained by first expressing the joint density as a mixture of Arellano-Valle and Azzalini's unified skew-normal densities and then using the property of closure under marginalization of the latter class.

Characterization of multivariate heavy-tailed distribution families via copula

The multivariate regular variation (MRV) is one of the most important tools in modeling multivariate heavy-tailed phenomena. This paper characterizes the MRV distributions through the tail dependence function of the copula associated with them. Along with some existing results, our studies indicate that the existence of the lower tail dependence function of the survival copula is necessary and sufficient for a random vector with regularly varying univariate marginals to have a MRV tail. Moreover, the limit measure of the MRV tail is explicitly characterized. Our analysis is also extended to some more general multivariate heavy-tailed distributions, including the subexponential and the long-tailed distribution families.

Characterization of Multivariate Heavy-Tailed Distribution Families via Copula

The multivariate regular variation (MRV) is one of the most important tools in modelling multivariate heavy-tailed phenomena. This paper characterizes the MRV distribution through the upper tail dependence index of the copula associated with them. Along with Theorem 2.3 in Li and Sun (2009), our results imply that the existence of the upper tail dependence index of the copula is necessary and sufficient for a random vector with regularly varying univariate marginals to have a MRV tail. Moreover, the limit measure of the MRV tail can be explicitly characterized by the tail dependence index. Our analysis is also extended to two more general multivariate heavy-tailed distributions, including the Subexponential and the Long-tailed distribution families.

An analysis of a variation of hit-and-run for uniform sampling from general regions

Hit-and-run, a class of MCMC samplers that converges to general multivariate distributions, is known to be unique in its ability to mix fast for uniform distributions over convex bodies. In particular, its rate of convergence to a uniform distribution is of a low order polynomial in the dimension. However, when the body of interest is difficult to sample from, typically a hyperrectangle is introduced that encloses the original body, and a one-dimensional acceptance/rejection is performed. The fast mixing analysis of hit-and-run does not account for this one-dimensional sampling that is often needed for implementation of the algorithm. Here we show that the effect of the size of the hyperrectangle on the efficiency of the algorithm is only a linear scaling effect. We also introduce a variation of hit-and-run that accelerates the sampler and demonstrate its capability through a computational study.

Loglinear representations of multivariate Bernoulli Rasch models

In this paper, the extended Rasch model for dichotomously scored items is derived from the general multivariate Bernoulli distribution. The necessary and sufficient conditions for the multivariate Bernoulli distribution to be equal to the extended Rasch model provide a new loglinear representation of the extended Rasch model. Conditions are also given under which the extended Rasch model is equal to the random effects Rasch model, and it is shown under what conditions the extended Rasch model is equal to a random effects Rasch model in which the underlying variable has a normal distribution. In addition, alternative models for the construction of likelihood ratio tests are proposed. One of these alternative models is Haberman's extended interaction model. Furthermore, it is shown how both the SPSS and SAS programs can be used to estimate and test loglinear representations of extended Rasch models.