Multivariate Laplace Distribution Research Articles

A Robust Interacting Multi-Model Multi-Bernoulli Mixture Filter for Maneuvering Multitarget Tracking under Glint Noise.

In practical radar systems, changes in the target aspect toward the radar will result in glint noise disturbances in the measurement data. The glint noise has heavy-tailed characteristics and cannot be perfectly modeled by the Gaussian distribution widely used in conventional tracking algorithms. In this article, we investigate the challenging problem of tracking a time-varying number of maneuvering targets in the context of glint noise with unknown statistics. By assuming a set of models for the possible motion modes of each single-target hypothesis and leveraging the multivariate Laplace distribution to model measurement noise, we propose a robust interacting multi-model multi-Bernoulli mixture filter based on the variational Bayesian method. Within this filter, the unknown noise statistics are adaptively learned while filtering and the predictive likelihood is approximately calculated by means of the variational lower bound. The effectiveness and superiority of our proposed filter is verified via computer simulations.

A Bayesian Approach to Low-Thrust Maneuvering Spacecraft Tracking

Bayesian estimation with an explicit transitional prior is required for a tracking algorithm to be embedded in most multitarget tracking frameworks. This paper describes a novel approach capable of tracking maneuvering spacecraft with an explicit transitional prior and in a Bayesian framework, with fewer than two observations passes per day. The algorithm samples thrust profiles according to a multivariate Laplace distribution. It is shown that multivariate Laplace distributions are particularly suited to track maneuvering spacecraft, leading to a log probability function that is almost linear with the thrust. Principles from rare event simulation theory are used to propagate the tails of the distribution. Fast propagation is enabled by multi-fidelity methods. Because of the diffuse transitional prior, a novel k-nearest-neighbor-based ensemble Gaussian mixture filter is developed and used. The method allows Bayesian tracking of maneuvering spacecraft for several scenarios with fewer than two measurement passes per day and with a mismatch between the true and expected thrust magnitude of up to a factor of 200. The validity domain and statistical significance of the method are shown by simulation through several Monte Carlo trials in different scenarios and with different filter settings.

Multi-mode industrial soft sensor method based on mixture Laplace variational auto-encoder

The industrially collected process data usually exhibit non-Gaussian and multi-mode characteristics. Due to sensor failures, irregular disturbances, and transmission problems, there are unavoidable outliers that make the data exhibit heavy-tailed characteristics. To this end, a variational auto-encoder regression method based on the mixture Laplacian distribution (MLVAER) is proposed, by introducing a type-II multivariate Laplacian distribution in the latent variable space for robust modeling, and further extending it to the mixture form to accommodate multi-mode processes, the corresponding reparameterization trick is finally proposed for the mixture form of this distribution for neural network gradient descent training. The model based on this distribution assumption has higher degrees of freedom than the model based on the traditional multivariate Laplace distribution assumption when the network structure is the same. Numerical simulation and experiments on two industrial examples demonstrate that the proposed algorithm reduces the root mean square error by over 15% compared to other algorithms.

Alternative modified Cholesky decomposition of the precision matrix of longitudinal data

The correlation matrix might be of scientific interest for longitudinal data. However, few studies have focused on both robust estimation of the correlation matrix against model misspecification and robustness to outliers in the data, when the precision matrix possesses a typical structure. In this paper, we propose an alternative modified Cholesky decomposition (AMCD) for the precision matrix of longitudinal data, which results in robust estimation of the correlation matrix against model misspecification of the innovation variances. A joint mean-covariance model with multivariate normal distribution and AMCD is established, the quasi-Fisher scoring algorithm is developed, and the maximum likelihood estimators are proven to be consistent and asymptotically normally distributed. Furthermore, a double-robust joint modeling approach with multivariate Laplace distribution and AMCD is established, and the quasi-Newton algorithm for maximum likelihood estimation is developed. The simulation studies and real data analysis demonstrate the effectiveness of the proposed AMCD method.

A novel robust multivariate Laplace distribution-based distributed consensus information fusion

The multivariate Laplace distribution (MLD) possesses a lesser set of prior parameters compared to the Student’s t-distribution, thus simplifying the modeling of measurement data subject to heavy-tailed noise corruption. This short communication introduces a novel distributed hybrid consensus fusion algorithm utilizing the MLD modeling and variational Bayesian approach. Firstly, we formulate the measurement noise associated with individual sensor nodes and introduce the hierarchical Gaussian form of MLD to enhance the feasibility of variational inference. Secondly, we opt for the inverse-Wishart (IW) distribution to serve as the conjugate prior to the positive definite matrix, enabling adaptive parameter learning within the MLD context. As a result, a novel hierarchical Gaussian state–space model (SSM) is established for multi-sensors network. Subsequently, we derive a novel local information filter to acquire higher precision information and innovation vector–matrix pairs, outperforming the Student’s t distribution-based filter. These derivations are grounded in the new hierarchical SSM, and the vector–matrix pairs aid in information fusion across the network. Lastly, we present a distributed robust consensus filter for fusing local information using a hybrid consensus approach that devoid of a central fusion node. Simulation results demonstrate that the proposed consensus filter can obtain better estimation accuracy under the interference of heavy-tailed measurement outliers.

Tweedie-type formulae for a multivariate Laplace distribution

In this paper, a set of Tweedie-type formulae are derived for a Kotz-type multivariate Laplace distribution. These formulae can be used as a powerful regression calibration tool to construct more efficient bias-corrected estimation procedures in statistical models, when some variables are contaminated by measurement errors with the Kotz-type multivariate Laplace distribution.

A Novel Robust Nonlinear Kalman Filter Based on Multivariate Laplace Distribution

In this brief, we investigate the robust state estimation of nonlinear systems with heavy-tailed noise. Based on the fact that the multivariate Laplace (ML) distribution has heavier tails than Gaussian distribution but is only determined by its mean and covariance, we utilize the ML distribution to model the heavy-tailed measurement noise. Based on the Gaussian mixture form of ML distribution, the system state and the covariance of ML distribution are inferred by variational Bayesian method, and then a closed recursive robust nonlinear Kalman filter is obtained with the nonlinear integrations calculated by numerical integration methods. Simulation results demonstrate that the proposed algorithm is insensitive to its free parameters and can obtain better estimation accuracy than related robust algorithms.

Parameter estimates for two-way repeated measurement MANOVA based on multivariate Laplace distribution

Repeated measures data describe multiple measurements taken from the same experimental unit under the different treatment conditions. In particular, researches with repeated measures data in various fields such as health and behavioral sciences, education, and psychology have an important role in applied statistics. There are many methods used to analyze the results of research designs planned with these measurements. The most important difference between these methods is the assumptions on which the models are based. One of the most important assumptions needed by classical methods is the normality assumption. Many methods are valid under the assumption of normality. However, it is not always possible to hold this assumption in applications. For this reason, in the analysis of repeated measures data, different distributions are necessary that can provide flexibility beyond the normal distribution, especially in cases where the assumption of normality does not hold. In this study, it is proposed to use Multivariate Laplace distribution (MLD) which is an alternative distribution in cases where normality assumption does not hold by examining the multivariate variance analysis model (MANOVA). Under MLD assumption, the parameter estimates for the Two-way Repeated Measures MANOVA model are carried out with the maximum likelihood (ML) estimation and ML estimates are obtained via the EM Algorithm.

Robust Mixture Probabilistic Partial Least Squares Model for Soft Sensing With Multivariate Laplace Distribution

Data collected in modern industrial processes often exhibit complex non-Gaussian and multimodal characteristics. In order to address these problems, a robust mixture probabilistic partial least squares (RMPPLS) model-based soft sensor is developed in this article, where two different kinds of hidden variables are introduced in the formulated model structure. The multivariate Laplace distribution is employed for robust modeling, and mixture form of the probabilistic partial least squares model is adopted for multimodal description. The unknown parameters are estimated in the expectation-maximization (EM) scheme and the corresponding soft sensor is finally constructed. A numerical example and the Tennessee Eastman (TE) process case study are explored to verify the effectiveness of the proposed algorithm.

A new multivariate Laplace distribution based on the mixture of normal distributions

In this paper, we propose a new multivariate Laplace distribution from normal variance mixture models, called as Type II multivariate Laplace distribution. Unlike the original multivariate Laplace distribution whose all components must have the same value for the mixing variate, the random components in the new distribution could have different value for its own mixing variate and are correlated only through the dependence structure of the normal random vector. Thus, it contains the multiplication of independently identical distribution univariate Laplace distributions as a special case if the normal covariance matrix is diagonal. A tractable stochastic representation is used to derive the probability density function and other statistical properties. The maximum likelihood estimates of parameters via an ECM (expectation/conditional maximization) algorithm and the Bayesian methods are derived. Some simulation studies are conducted to evaluate the performance of the proposed methods. Applications in two real data sets indicate that the Type II multivariate Laplace distribution could have a better performance and is distinct from the original one.

Mixture robust L1 probabilistic principal component regression and soft sensor application

AbstractIn this paper, the multivariate Laplace distribution (also called L1 distribution) is adopted to construct a robust probabilistic principal component regression model (MRPPCR‐L1) under multiple operating modes. In the practical industrial chemistry process, outliers exist due to incorrect recording, disturbances, and process noises and might result in modelling distortion. To address this problem, Laplace distribution, instead of the Gaussian distribution in traditional methods, is introduced to reduce the negative influence of outliers. Moreover, probabilistic principal component regression is employed for dealing with the mixture modelling problem owing to its probabilistic property to determine the operating modes. The formulation of this approach is derived with the expectation maximum algorithm and the soft sensing model is also developed for prediction. Compared to the conventional method, a numerical example and the Tennessee Eastman process are used to demonstrate the robust modelling performance of the proposed method.

Probabilistic Modeling of Speech in Spectral Domain using Maximum Likelihood Estimation

The performance of many speech processing algorithms depends on modeling speech signals using appropriate probability distributions. Various distributions such as the Gamma distribution, Gaussian distribution, Generalized Gaussian distribution, Laplace distribution as well as multivariate Gaussian and Laplace distributions have been proposed in the literature to model different segment lengths of speech, typically below 200 ms in different domains. In this paper, we attempted to fit Laplace and Gaussian distributions to obtain a statistical model of speech short-time Fourier transform coefficients with high spectral resolution (segment length >500 ms) and low spectral resolution (segment length <10 ms). Distribution fitting of Laplace and Gaussian distributions was performed using maximum-likelihood estimation. It was found that speech short-time Fourier transform coefficients with high spectral resolution can be modeled using Laplace distribution. For low spectral resolution, neither the Laplace nor Gaussian distribution provided a good fit. Spectral domain modeling of speech with different depths of spectral resolution is useful in understanding the perceptual stability of hearing which is necessary for the design of digital hearing aids.

Evaluation of outlier detection method performance in symmetric multivariate distributions

Determining outliers is more complicated in multivariate data sets than it is in univariate cases. The aim of this study is to evaluate the blocked adaptive computationally efficient outlier nominators (BACON) algorithm, the fast minimum covariance determinant (FAST-MCD) method, and the robust Mahalanobis distance (RM) method in multivariate data sets. For this purpose, outlier detection methods were compared for multivariate normal, Laplace, and Cauchy distributions with different sample sizes and numbers of variables. False-negative and false-positive ratios were used to evaluate the methods’ performance. The results of this work indicate that the performance of these methods varies according to the distribution type.

Robust mixture multivariate linear regression by multivariate Laplace distribution

Assuming that the error terms follow a multivariate Laplace distribution, we propose a robust estimation procedure for mixture of multivariate linear regression models in this paper. Using the fact that the multivariate Laplace distribution is a scale mixture of the multivariate standard normal distribution, an efficient EM algorithm is designed to implement the proposed robust estimation procedure. The performance of the proposed algorithm is thoroughly evaluated by some simulation and comparison studies.

Multivariate semi-α-Laplace distributions

ABSTRACTA multivariate semi-α-Laplace distribution (denoted by Ms-αLaplace) is introduced and studied in this paper. It is more general than the multivariate Linnik and Laplace distributions proposed by Sabu and Pillai (1991) or Anderson (1992). The Ms-αLaplace distribution has univariate semi-α-Laplace (Pillai, 1985) as marginal distribution. Various characterization theorems of the Ms-αLaplace distribution based on the closure property of the normalized geometric sum are proved.

Linear mixed model with Laplace distribution (LLMM)

Linear mixed modeling (LMM) is a comprehensive technique used for clustered, panel and longitudinal data. The main assumption of classical LMM is having normally distributed random effects and error terms. However, there are several situations for that we need to use heavier tails distributions than the (multivariate) normal to handle outliers and/or heavy tailness in data. In this study, we focus on LMM using the multivariate Laplace distribution which is known as the heavy tailed alternative to the normal distribution. The parameter estimators of interest are generated with EM algorithm for the proposed model. A simulation study is provided to illustrate the performance of the Laplace distribution over the normal distribution for LMM. Also, a real data example is used to explore the behavior of the proposed estimators over the counterparts.

Gaussian Scale Mixture Models for Robust Linear Multivariate Regression with Missing Data

We present an algorithm for multivariate robust Bayesian linear regression with missing data. The iterative algorithm computes an approximative posterior for the model parameters based on the variational Bayes (VB) method. Compared to the EM algorithm, the VB method has the advantage that the variance for the model parameters is also computed directly by the algorithm. We consider three families of Gaussian scale mixture models for the measurements, which include as special cases the multivariate t distribution, the multivariate Laplace distribution, and the contaminated normal model. The observations can contain missing values, assuming that the missing data mechanism can be ignored. A Matlab/Octave implementation of the algorithm is presented and applied to solve three reference examples from the literature.

Outlier‐Robust Bayesian Multinomial Choice Modeling

SummaryA Bayesian method for outlier‐robust estimation of multinomial choice models is presented. The method can be used for both correlated as well as uncorrelated choice alternatives and guarantees robustness towards outliers in the dependent and independent variables. To account for outliers in the response direction, the fat‐tailed multivariate Laplace distribution is used. Leverage points are handled via a shrinkage procedure. A simulation study shows that estimation of the model parameters is less influenced by outliers compared to non‐robust alternatives. An analysis of margarine scanner data shows how our method can be used for better pricing decisions. Copyright © 2015 John Wiley & Sons, Ltd.

Measures of multivariate skewness and kurtosis in high-dimensional framework

Skewness and kurtosis characteristics of a multivariate p-dimensional distribution introduced by Mardia (1970) have been used in various testing procedures and demonstrated attractive asymptotic properties in large sample settings. However these characteristics are not designed for high-dimensional problems where the dimensionality, p can largely exceeds the sample size, N. Such type of high-dimensional data are commonly encountered in modern statistical applications. This the suggests that new measures of skewness and kurtosis that can accommodate high-dimensional settings must be derived and carefully studied. In this paper, we show that, by exploiting the dependence structure, new expressions for skewness and kurtosis are introduced as an extension of the corresponding Mardia’s measures, which uses the potential advantages that the block-diagonal covariance structure has to offer in high dimensions. Asymptotic properties of newly derived measures are investigated and the cumulant based characterizations are presented along with of applications to a mixture of multivariate normal distributions and multivariate Laplace distribution, for which the explicit expressions of skewness and kurtosis are obtained. Test statistics based on the new measures of skewness and kurtosis are proposed for testing a distribution shape, and their limit distributions are established in the asymptotic framework where N→∞ and p is fixed but large, including p>N. For the dependence structure learning, the gLasso based technique is explored followed by AIC step which we propose for optimization of the gLasso candidate model. Performance accuracy of the test procedures based on our estimators of skewness and kurtosis are evaluated using Monte Carlo simulations and the validity of the suggested approach is shown for a number of cases when p>N.

Bayesian doubly adaptive elastic-net Lasso for VAR shrinkage

We develop a novel Bayesian doubly adaptive elastic-net Lasso (DAELasso) approach for VAR shrinkage. DAELasso achieves variable selection and coefficient shrinkage in a data-based manner. It deals constructively with explanatory variables which tend to be highly collinear by encouraging the grouping effect. In addition, it also allows for different degrees of shrinkage for different coefficients. Rewriting the multivariate Laplace distribution as a scale mixture, we establish closed-form conditional posteriors that can be drawn from a Gibbs sampler. An empirical analysis shows that the forecast results produced by DAELasso and its variants are comparable to those from other popular Bayesian methods, which provides further evidence that the forecast performances of large and medium sized Bayesian VARs are relatively robust to prior choices, and, in practice, simple Minnesota types of priors can be more attractive than their complex and well-designed alternatives.