Multivariate Normal Distribution Research Articles

PVSPE: A pyramid vision multitask transformer network for spacecraft pose estimation

Spacecraft pose estimation (SPE) plays a vital role in the relative navigation system for on-orbit servicing and active debris removal. Current deep learning-based methods have made great achievements on object pose estimation. However, towards the challenging onboard SPE missions, most existing Convolutional Neural Network (CNN) methods failed to capture remote vision attention, leading to the reduction of accuracy and robustness. In this paper, we presented an end-to-end multi-task Pyramid Transformer SPE network (PVSPE) consisting of two novel feature extraction modules: EnhancedPVT (EnPVT) and SlimGFPN. The EnPVT module is designed to combine global spatial and channel attention, while the Slim GFPN module can fuse features more effectively. Matrix Fisher and multivariate Gaussian distributions are further employed to model the uncertainty of pose regression to increase its accuracy. Extensive experiments are carried out on challenging SPEED + and SHIRT datasets, to validate the performances on pose estimation and vision-based navigation, respectively. The results show that the proposed PVSPE model achieved high accuracy for SPE on the SPEED + dataset even under different scales and severe illumination, demonstrating its robustness and high generalization. Leveraging the insightful uncertainty model of PVSPE, the vision-based navigation pipeline, combined with Kalman filters, accurately estimated the satellite pose under challenging rendezvous scenarios on the SHIRT dataset, with degree-level attitude errors and centimeter-level translation accuracy at steady-state.

Data patterns that reliably precede US recessions

AbstractThis paper proposes a method of forecasting US recessions beginning with data displays that contain the last 12 quarters of seven US expansions. These end‐of‐expansion displays allow observers to see for themselves what is different about the last year before recessions compared with the two earlier years. Using a statistical model that treats this historical data as draws from a 12‐dimensional multivariate normal distribution, the most recent data are probabilistically inserted into these images where the recent data are most like the historical data. This is a recession forecast based not on presumed patterns but on patterns revealed by the data.

Variational Bayesian phylogenies through matrix representation of tree space.

In this article, we study the distance matrix as a representation of a phylogeny by way of hierarchical clustering. By defining a multivariate normal distribution on (a subset of) the entries in a matrix, this allows us to represent a distribution over rooted time trees. Here, we demonstrate tree distributions can be represented accurately this way for a number of published tree distributions. Though such a representation does not map to unique trees, restriction to a subspace, in particular one we call a "cube", makes the representation bijective at the cost of not being able to represent all possible trees. We introduce an algorithm "cubeVB" specifically for cubes and show through well calibrated simulation study that it is possible to recover parameters of interest like tree height and length. Although a cube cannot represent all of tree space, it is a great improvement over a single summary tree, and it opens up exciting new opportunities for scaling up Bayesian phylogenetic inference. We also demonstrate how to use a matrix representation of a tree distribution to get better summary trees than commonly used maximum clade credibility trees. An open source implementation of the cubeVB algorithm is available from https://github.com/rbouckaert/cubevb as the cubevb package for BEAST 2.

Intrinsic Information-Theoretic Models

With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N (μ, σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(μ,Σ0) with a constant variance–covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger’s equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of “intrinsic”. Similarly, as in our previous work with the estimator’s variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the “intrinsic” Cramér–Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes’ theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of “intrinsic” information-theoretic models, which are essential for developing physics.

Integration of just-in-time learning with variational autoencoder for cell culture process monitoring based on Raman spectroscopy.

Protein production in the biopharmaceutical industry necessitates the utilization of multiple analytical techniques and control methodologies to ensure both safety and consistency. To facilitate real-time monitoring and control of cell culture processes, Raman spectroscopy has emerged as a versatile analytical technology. This technique, categorized as a Process Analytical Technology, employs chemometric models to establish correlations between Raman signals and key variables of interest. One notable approach for achieving real-time monitoring is through the application of just-in-time learning (JITL), an industrial soft sensor modeling technique that utilizes Raman signals to estimate process variables promptly. The conventional Raman-based JITL method relies on the K-nearest neighbor (KNN) algorithm with Euclidean distance as the similarity measure. However, it falls short of addressing the impact of data uncertainties. To rectify this limitation, this study endeavors to integrate JITL with a variational autoencoder (VAE). This integration aims to extract dominant Raman features in a nonlinear fashion, which are expressed as multivariate Gaussian distributions. Three experimental runs using different cell lines were chosen to compare the performance of the proposed algorithm with commonly utilized methods in the literature. The findings indicate that the VAE-JITL approach consistently outperforms partial least squares, convolutional neural network, and JITL with KNN similarity measure in accurately predicting key process variables.

Multidimensional credibility: A new approach based on joint distribution function

Abstract In the traditional multidimensional credibility models developed by Jewell ((1973) Operations Research Center, pp. 73–77.), the estimation of the hypothetical mean vector involves complex matrix manipulations, which can be challenging to implement in practice. Additionally, the estimation of hyperparameters becomes even more difficult in high-dimensional risk variable scenarios. To address these issues, this paper proposes a new multidimensional credibility model based on the conditional joint distribution function for predicting future premiums. First, we develop an estimator of the joint distribution function of a vector of claims using linear combinations of indicator functions based on past observations. By minimizing the integral of the expected quadratic distance function between the proposed estimator and the true joint distribution function, we obtain the optimal linear Bayesian estimator of the joint distribution function. Using the plug-in method, we obtain an explicit formula for the multidimensional credibility estimator of the hypothetical mean vector. In contrast to the traditional multidimensional credibility approach, our newly proposed estimator does not involve a matrix as the credibility factor, but rather a scalar. This scalar is composed of both population information and sample information, and it still maintains the essential property of increasingness with respect to the sample size. Furthermore, the new estimator based on the joint distribution function can be naturally extended and applied to estimate the process covariance matrix and risk premiums under various premium principles. We further illustrate the performance of the new estimator by comparing it with the traditional multidimensional credibility model using bivariate exponential-gamma and multivariate normal distributions. Finally, we present two real examples to demonstrate the findings of our study.

Sparse subnetwork inference for neural network epistemic uncertainty estimation with improved Hessian approximation

Despite significant advances in deep neural networks across diverse domains, challenges persist in safety-critical contexts, including domain shift sensitivity and unreliable uncertainty estimation. To address these issues, this study investigates Bayesian learning for uncertainty handling in modern neural networks. However, the high-dimensional, non-convex nature of the posterior distribution poses practical limitations for epistemic uncertainty estimation. The Laplace approximation, as a cost-efficient Bayesian method, offers a practical solution by approximating the posterior as a multivariate normal distribution but faces computational bottlenecks in precise covariance matrix computation and storage. This research employs subnetwork inference, utilizing only a subset of the parameter space for Bayesian inference. In addition, a Kronecker-factored and low-rank representation is explored to reduce space complexity and computational costs. Several corrections are introduced to converge the approximated curvature to the exact Hessian matrix. Numerical results demonstrate the effectiveness and competitiveness of this method, whereas qualitative experiments highlight the impact of Hessian approximation granularity and parameter space utilization in Bayesian inference on mitigating overconfidence in predictions and obtaining high-quality uncertainty estimates.

Asymptotic Relative Efficiency Comparison for some Fit Indices in Structural Equation Modeling

There are many fit statistics used in the structural equation modeling, and new ones are consistently being developed. Because of the variety of fit statistics, it is very important to be able to decide which fit statistics are appropriate to use in studies. When comparing any two statistics, the asymptotic relative efficiency (ARE) between them is used. The ARE can use as a power of the fit indices is one of the familiar optimal criteria. It is frequently more convenient, and also more suggestive, to use a measure of relative merit called the relative efficiency. This study aimed to compare of fit indices using Fraser’s asymptotic relative efficiency. The data sets were derived from the multivariate normal distribution using the mean vector and covariance matrix. It was determined that the most efficient fit indices in terms of asymptotic relative efficiency were Z-Test of Wilson & Hilferty (W&H), Root Mean Square Error of Approximation (RMSEA), and Chi-Square indices, respectively.

Point Cloud Denoising and Feature Preservation: An Adaptive Kernel Approach Based on Local Density and Global Statistics.

Noise removal is a critical stage in the preprocessing of point clouds, exerting a significant impact on subsequent processes such as point cloud classification, segmentation, feature extraction, and 3D reconstruction. The exploration of methods capable of adapting to and effectively handling the noise in point clouds from real-world outdoor scenes remains an open and practically significant issue. Addressing this issue, this study proposes an adaptive kernel approach based on local density and global statistics (AKA-LDGS). This method constructs the overall framework for point cloud denoising using Bayesian estimation theory. It dynamically sets the prior probabilities of real and noise points according to the spatial function relationship, which varies with the distance from the points to the center of the LiDAR. The probability density function (PDF) for real points is constructed using a multivariate Gaussian distribution, while the PDF for noise points is established using a data-driven, non-parametric adaptive kernel density estimation (KDE) approach. Experimental results demonstrate that this method can effectively remove noise from point clouds in real-world outdoor scenes while maintaining the overall structural features of the point cloud.

Low and high dimensional wavelet thresholds for matrix-variate normal distribution

The matrix-variate normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables. In this paper, we introduce a wavelet shrinkage estimator based on Stein’s unbiased risk estimate (SURE) threshold for matrix-variate normal distribution. We find a new SURE threshold for soft thresholding wavelet shrinkage estimator under the reflected normal balanced loss function in low and high dimensional cases. Also, we obtain the restricted wavelet shrinkage estimator based on non-negative sub matrix of the mean matrix. Finally, we present a simulation study to test the validity of the wavelet shrinkage estimator and two real examples for low and high dimensional data sets.

Dynamic probability control limits for the adaptive multivariate EWMA chart

AbstractEffective monitoring of the control charts requires the establishment of appropriate control limits. Various methods have been proposed to determine the fixed control limits (FCLs) based on a specified in‐control average run‐length value, such as Markov chains, integral equations, and Monte Carlo simulations.With the FCLs, the conditional false alarm rate (CFAR) of a control chart varies over time in an unexpected and undesirable way, where the CFAR refers to the probability of a false alarm at a particular time given no previous false alarm. To address this issue, dynamic probability control limits (DPCLs) can be employed to keep the CFAR constant over time. In this study, we determine the DPCLs using fixed and time‐varying sample sizes for the adaptive multivariate EWMA chart when the underlying process is assumed to follow a multivariate normal distribution. The DPCLs are designed to automatically adjust to changes in the probability distribution of the sample size, while maintaining a consistent CFAR. This results in a closely matched run‐length performance for an in‐control process compared to that of the geometric distribution.

Learning likelihood ratios with neural network classifiers

The likelihood ratio is a crucial quantity for statistical inference in science that enables hypothesis testing, construction of confidence intervals, reweighting of distributions, and more. Many modern scientific applications, however, make use of data- or simulation-driven models for which computing the likelihood ratio can be very difficult or even impossible. By applying the so-called “likelihood ratio trick,” approximations of the likelihood ratio may be computed using clever parametrizations of neural network-based classifiers. A number of different neural network setups can be defined to satisfy this procedure, each with varying performance in approximating the likelihood ratio when using finite training data. We present a series of empirical studies detailing the performance of several common loss functionals and parametrizations of the classifier output in approximating the likelihood ratio of two univariate and multivariate Gaussian distributions as well as simulated high-energy particle physics datasets.

Simplified Methods for Modelling Dependent Parameters in Health Economic Evaluations: A Tutorial.

In health economic evaluations, model parameters are often dependent on other model parameters. Although methods exist to simulate multivariate normal (MVN) distribution data and estimate transition probabilities in Markov models while considering competing risks, they are technically challenging for health economic modellers to implement. This tutorial introduces easily implementable applications for handling dependent parameters in modelling. Analytical proofs and proposed simplified methods for handling dependent parameters in typical health economic modelling scenarios are provided, and implementation of these methods are illustrated in seven examples along with the SAS and R code. Methods to quantify the covariance and correlation coefficients of correlated variables based on published summary statistics and generation of MVN distribution data are demonstrated using examples of physician visits data and cost component data. The use of univariate normal distribution data instead of MVN distribution data to capture population heterogeneity is illustrated based on the results from multiple regression models with linear predictors, and two examples are provided (linear fixed-effects model and Cox proportional hazards model). A conditional probability method is introduced to handle two or more state transitions in a single Markov model cycle and applied in examples of one- and two-way state transitions. This tutorial proposes an extension of routinely used methods along with several examples. These simplified methods may be easily applied by health economic modellers with varied statistical backgrounds.

Group sequential designs for pragmatic clinical trials with early outcomes: methods and guidance for planning and implementation

BackgroundGroup sequential designs are one of the most widely used methodologies for adaptive design in randomized clinical trials. In settings where early outcomes are available, they offer large gains in efficiency compared to a fixed design. However, such designs are underused and used predominantly in therapeutic areas where there is expertise and experience in implementation. One barrier to their greater use is the requirement to undertake simulation studies at the planning stage that require considerable knowledge, coding experience and additional costs. Based on some modest assumptions about the likely patterns of recruitment and the covariance structure of the outcomes, some simple analytic expressions are presented that negate the need to undertake simulations.MethodsA model for longitudinal outcomes with an assumed approximate multivariate normal distribution and three contrasting simple recruitment models are described, based on fixed, increasing and decreasing rates. For assumed uniform and exponential correlation models, analytic expressions for the variance of the treatment effect and the effects of the early outcomes on reducing this variance at the primary outcome time-point are presented. Expressions for the minimum and maximum values show how the correlations and timing of the early outcomes affect design efficiency.ResultsSimulations showed how patterns of information accrual varied between correlation and recruitment models, and consequentially to some general guidance for planning a trial. Using a previously reported group sequential trial as an exemplar, it is shown how the analytic expressions given here could have been used as a quick and flexible planning tool, avoiding the need for extensive simulation studies based on individual participant data.ConclusionsThe analytic expressions described can be routinely used at the planning stage of a putative trial, based on some modest assumptions about the likely number of outcomes and when they might occur and the expected recruitment patterns. Numerical simulations showed that these models behaved sensibly and allowed a range of design options to be explored in a way that would have been difficult and time-consuming if the previously described method of simulating individual trial participant data had been used.

DIF Detection for Multiple Groups: Comparing Three‐Level GLMMs and Multiple‐Group IRT Models

AbstractFor assessment scales applied to different groups (e.g., students from different states; patients in different countries), multigroup differential item functioning (MG‐DIF) needs to be evaluated in order to ensure that respondents with the same trait level but from different groups have equal response probabilities on a particular item. The current study compares two approaches for DIF detection: a multiple‐group item response theory (MG‐IRT) model and a generalized linear mixed model (GLMM). In the MG‐IRT model approach, item parameters are constrained to be equal across groups and DIF is evaluated for each item in each group. In the GLMM, groups are treated as random, and item difficulties are modeled as correlated random effects with a joint multivariate normal distribution. Its nested structure allows the estimation of item difficulty variances and covariances at the group level. We use an excerpt from the PISA 2015 reading domain as an exemplary empirical investigation, and conduct a simulation study to compare the performance of the two approaches. Results from the empirical investigation show that the detection of countries with DIF is similar in both approaches. Results from the simulation study confirm this finding and indicate slight advantages of the MG‐IRT model approach.

Determination of correlations in multivariate count data with informative observation times.

We consider there are various types of recurrent events and the total number of occurrences are collected at the random observation times. It has concerned that the observation process may not be independent to the multivariate event processes, hence the total counts and observation times may be correlated and the dependence may exist among different types of the event processes as well. Many methods have developed nonparametric models to accommodate such unknown structures; however, it is difficult to assess and directly quantify their correlation relationships. A multivariate frailty model is proposed to this study, in which the event and observation processes are linked by frailty variables whose joint distribution can be implicitly specified through the multivariate normal distribution with some unknown covariance matrix. The Bayesian inference method is conducted to obtain the estimates of the regression coefficients and correlation parameters. We use a form of trigonometric functions to represent the covariance matrix, so that it meets the positive-definiteness condition efficiently during the estimation schemes. The simulation studies demonstrate the utility of the proposed models. We apply the model to a skin cancer prevention study, and aim to determine the covariate and association effects. We found treatment is significant in determining the duration of examination times; prior-counts, age and gender are significant variables on the occurrence rates of tumor counts. Using the covariance matrix to access the underlying dependent structure, the mutual correlations among them are all positive, and the basal cell counts are more related to the examination times.

Marginal Probability-Based Integer Handling for CMA-ES Tackling Single-and Multi-Objective Mixed-Integer Black-Box Optimization

This study targets the mixed-integer black-box optimization (MI-BBO) problem where continuous and integer variables should be optimized simultaneously. The CMA-ES, our focus in this study, is a population-based stochastic search method that samples solution candidates from a multivariate Gaussian distribution (MGD), which shows excellent performance in continuous BBO. The parameters of MGD, mean and (co)variance, are updated based on the evaluation value of candidate solutions in the CMA-ES. If the CMA-ES is applied to the MI-BBO with straightforward discretization, however, the variance corresponding to the integer variables becomes much smaller than the granularity of the discretization before reaching the optimal solution, which leads to the stagnation of the optimization. In particular, when binary variables are included in the problem, this stagnation more likely occurs because the granularity of the discretization becomes wider, and the existing integer handling for the CMA-ES does not address this stagnation. To overcome these limitations, we propose a simple integer handling for the CMA-ES based on lower-bounding the marginal probabilities associated with the generation of integer variables in the MGD. The numerical experiments on the MI-BBO benchmark problems demonstrate the efficiency and robustness of the proposed method. Furthermore, in order to demonstrate the generality of the idea of the proposed method, in addition to the single-objective optimization case, we incorporate it into multi-objective CMA-ES and verify its performance on bi-objective mixed-integer benchmark problems.

Natural gradient ascent in evolutionary games

We consider evolutionary games with a continuous trait space where the replicator dynamics are restricted to the manifold of multivariate Gaussian distributions. We demonstrate that replicator dynamics are gradient flows with respect to the Fisher information metric. The potential function for these gradient flows is closely related to the mean fitness. Our findings extend previous results on natural gradient ascent in evolutionary games with a finite strategy set. Throughout the paper we pursue an information-geometric point of view on evolutionary games. This sheds a new light on the replicator dynamics as a learning process, realizing the compromise between maximization of the mean fitness and preservation of the diversity.

TIME SERIES WITH MULTIPLE CHANGE POINTS AND CENSORED OBSERVATIONS

This article examines a Bayesian model for a nonstationary time series with an unknown number of change points and censored observations. Each segment is assumed to be an autoregressive process with order one. To estimate the number and locations of change points, we use the reversible jump Markov chain Monte Carlo (RJMCMC) algorithm. The censored problem is solved by imputing the censored values from a multivariate normal distribution based on the observed part. A numerical example shows that the estimates of the number of change points and their localizations have little bias. Additionally, the estimates are robust to the censoring percentage.

Multivariate Generalized Linear Mixed Models for Count Data

Univariate regression models have rich literature for counting data. However, this is not the case for multivariate count data. Therefore, we present the Multivariate Generalized Linear Mixed Models framework that deals with a multivariate set of responses, measuring the correlation between them through random effects that follows a multivariate normal distribution. This model is based on a GLMM with a random intercept and the estimation process remains the same as a standard GLMM with random effects integrated out via Laplace approximation. We efficiently implemented this model through the TMB package available in R. We used Poisson, negative binomial (NB), and COM-Poisson distributions. To assess the estimator properties, we conducted a simulation study considering four different sample sizes and three different correlation values for each distribution. We achieved unbiased and consistent estimators for Poisson and NB distributions; for COM-Poisson estimators were consistent, but biased, especially for dispersion, variance, and correlation parameter estimators. These models were applied to two datasets. The first concerns a sample from 30 different sites collected in Australia where the number of times each one of the 41 different ant species was registered; which results in an impressive 820 variance-covariance and 41 dispersion parameters are estimated simultaneously, let alone the regression parameters. The second is from the Australia Health Survey with 5 response variables and 5190 respondents. These datasets can be considered overdispersed by the generalized dispersion index. The COM-Poisson model overcame the other two competitors considering three goodness-of-fit indexes, AIC, BIC, and maximized log-likelihood values. As a result, it estimated parameters with smaller standard errors and a greater number of significant correlation coefficients. Therefore, the proposed model is capable of dealing with multivariate count data, either under- equi- or overdispersed responses, and measuring any kind of correlation between them taking into account the effects of the covariates.