Positive Definite Covariance Matrix Research Articles

Ensemble variational Bayesian approximation for the inversion and uncertainty quantification of Darcy flows in heterogeneous porous media with random parameters

The inversion and uncertainty quantification of physical parameters are important for many science and engineering problems. For nonlinear parameter inversion problems of flows in hydrocarbon reservoirs, estimating the true posterior probability distribution function for the random parameter is difficult when the dimension of the parameter is relatively high. Inversion methods such as gradient-based or heuristic optimization methods searching for the maximum a posterior are generally point-estimate, while uncertainty quantification using MCMC is computationally expensive. In the current study, a new ensemble variational Bayesian approximation method is developed to estimate the posterior probability density using an ensemble of realizations regarded as a trial sample distribution. The objective is to optimize the trial sample distribution such that the evidence lower bound is maximized indicating that the distance between the trial and true posterior distribution is minimized. A subspace is built using principle component analysis based on finite samples to obtain a diagonal positive-definite covariance matrix, and ensemble-based data assimilation is used to optimize the trial distribution efficiently. Heterogeneous 2D and 3D test cases of transient single- and two-phase Darcy flows are presented for validation.

Estimation of clutter covariance matrix in stap based on knowledge-aided and geometric methods

The insufficient number of available samples can cause inaccurate estimation of the clutter covariance matrix (CCM) in space-time adaptive processing (STAP), resulting in degraded clutter suppression performance. To tackle this problem, a CCM estimation approach based on knowledge-aided (KA) and geometric methods is proposed in this paper. A combination of environmental as well as structural (Persymmetric or Symmetric structure) knowledge information is utilized to model the covariance matrix of each sample as a knowledge-aided Hermitian positive definite (KA-HPD) covariance matrix. The estimation problem is introduced into the Riemannian manifold composed of the KA-HPD covariance matrices, and the geometric method is used for nonlinear processing. Based on the Kullback-Leibler (KL) divergence and the KL mean, the final estimated CCM is designed as a weighted combination of each KA-HPD covariance matrix. Experiment results show that the two designed structural covariance matrix estimators possess superior clutter suppression performance.

A data-driven statistical-stochastic surrogate modeling strategy for complex nonlinear non-stationary dynamics

We propose a statistical-stochastic surrogate modeling approach to predict the response of the mean and variance statistics under various initial conditions and external forcing perturbations. The proposed modeling framework extends the purely statistical modeling approach that is practically limited to the homogeneous statistical regime for high-dimensional state variables. The new closure system allows one to overcome several practical issues that emerge in the non-homogeneous statistical regimes. First, the proposed ensemble modeling that couples the mean statistics and stochastic fluctuations naturally produces positive-definite covariance matrix estimation, which is a challenging issue that hampers the purely statistical modeling approaches. Second, the proposed closure model, which embeds a non-Markovian neural-network model for the unresolved fluxes such that the variance of the dynamics is consistent, overcomes the inherent instability of the stochastic fluctuation dynamics. Effectively, the proposed framework extends the classical stochastic parametric modeling paradigm for the unresolved dynamics to a semi-parametric parameterization with a residual Long-Short-Term-Memory neural network architecture. Third, based on empirical information metric, we provide an efficient and effective training procedure by fitting a loss function that measures the differences between response statistics. Supporting numerical examples are provided with the Lorenz-96 model, a system of ODEs that admits the characteristic of chaotic dynamics with both homogeneous and inhomogeneous statistical regimes. In the latter case, we will see the effectiveness of the statistical prediction even though the resolved Fourier modes corresponding to the leading mean energy and variance spectra do not coincide.

Analyzing The Covid-19 Pandemic of Volatility Spillover Influence the Collaboration of Foreign and Indian Stock Markets

One of the most crucial variables in investment selections is volatility. Unexpected information causes an investor to trade unusually in the market, which influences market volatility. Furthermore, various market sectors are affected differently by this type of trading behaviour. This research investigates the impact of COVID-19 on stock market volatility in India using a generalised autoregressive conditional model. The research was conducted using daily closing prices of stock indices include Nifty 50 and Nifty 500, from September 8, 2019, to July 9, 2021. In this article, the TGARCH model (1,1) was utilized to evaluate the volatility of NSE listed shares. The stock market's volatility has been calculated using the NSE's closing price. To reduce the skewness in the stock price data distribution, the natural logarithm of each price data is employed in the estimations. During the pre-COVID and COVID periods, the conditional volatility of the daily return series showed signs of volatility variations. Furthermore, the study aimed to compare stock price returns in pre-COVID19 and post-COVID19 scenarios to global indexes such as the NASDAQ, Nikkei 225, and FTSE. The stock market in India suffered volatility throughout the epidemic, according to the findings. Consequently, the study recommends NSE stock exchange bond indices to explore the volatility spillover influence between foreign exchange and the stock market in India. In this work, the positive definite covariance matrix is given, therefore a multivariate GARCH with BEKK model is used to estimate the covariance correlation and identify the consequences that stock market downturns can create. SPSS and Eviews software are used to analyze the data. The Augmented Dickey-Fuller (ADF) and KPSS unit root tests have been used to determine whether a time series is stationary or nonstationary. Whereas it corrects for heteroscedasticity and autocorrelation consistency in ADF test statistics, the study employed the KPSS unit root test to estimate the right result. In addition, to investigate the impact of COVID19 on stock market volatility in terms of negative and positive shocks in financial decisions, the TGARCH model captures asymmetry. The finding that the variable has a negative and statistically significant coefficient suggests that the COVID-19 outbreak lowered stock market volatility in India. In terms of historical errors, the coefficients represent the persistence of volatility for each nation. NIFTY and NASDAQ have the largest and longest-term spillover effect. According to the findings, India is the least sensitive to external shocks.

Accelerating Geostatistical Modeling and Prediction With Mixed-Precision Computations: A High-Productivity Approach With PaRSEC

Geostatistical modeling, one of the prime motivating applications for exascale computing, is a technique for predicting desired quantities from geographically distributed data, based on statistical models and optimization of parameters. Spatial data are assumed to possess properties of stationarity or non-stationarity via a kernel fitted to a covariance matrix. A primary workhorse of stationary spatial statistics is Gaussian maximum log-likelihood estimation (MLE), whose central data structure is a dense, symmetric positive definite covariance matrix of the dimension of the number of correlated observations. Two essential operations in MLE are the application of the inverse and evaluation of the determinant of the covariance matrix. These can be rendered through the Cholesky decomposition and triangular solution. In this contribution, we reduce the precision of weakly correlated locations to single- or half- precision based on distance. We thus exploit mathematical structure to migrate MLE to a three-precision approximation that takes advantage of contemporary architectures offering BLAS3-like operations in a single instruction that are extremely fast for reduced precision. We illustrate application-expected accuracy worthy of double-precision from a majority half-precision computation, in a context where uniform single-precision is by itself insufficient. In tackling the complexity and imbalance caused by the mixing of three precisions, we deploy the <monospace>PaRSEC</monospace> runtime system. <monospace>PaRSEC</monospace> delivers on-demand casting of precisions while orchestrating tasks and data movement in a multi-GPU distributed-memory environment within a tile-based Cholesky factorization. Application-expected accuracy is maintained while achieving up to <inline-formula><tex-math notation="LaTeX">$1.59X$</tex-math></inline-formula> by mixing FP64/FP32 operations on 1536 nodes of <monospace>HAWK</monospace> or 4096 nodes of <monospace>Shaheen II</monospace> , and up to <inline-formula><tex-math notation="LaTeX">$2.64X$</tex-math></inline-formula> by mixing FP64/FP32/FP16 operations on 128 nodes of <monospace>Summit</monospace> , relative to FP64-only operations. This translates into up to 4.5, 4.7, and 9.1 (mixed) PFlop/s sustained performance, respectively, demonstrating a synergistic combination of exascale architecture, dynamic runtime software, and algorithmic adaptation applied to challenging environmental problems.

Multivariate Normal Approximation on the Wiener Space: New Bounds in the Convex Distance

AbstractWe establish explicit bounds on the convex distance between the distribution of a vector of smooth functionals of a Gaussian field and that of a normal vector with a positive-definite covariance matrix. Our bounds are commensurate to the ones obtained by Nourdin et al. (Ann Inst Henri Poincaré Probab Stat 46(1):45–58, 2010) for the (smoother) 1-Wasserstein distance, and do not involve any additional logarithmic factor. One of the main tools exploited in our work is a recursive estimate on the convex distance recently obtained by Schulte and Yukich (Electron J Probab 24(130):1–42, 2019). We illustrate our abstract results in two different situations: (i) we prove a quantitative multivariate fourth moment theorem for vectors of multiple Wiener–Itô integrals, and (ii) we characterize the rate of convergence for the finite-dimensional distributions in the functional Breuer–Major theorem.

Hybrid projection methods for large-scale inverse problems with mixed Gaussian priors

When solving ill-posed inverse problems, a good choice of the prior is critical for the computation of a reasonable solution. A common approach is to include a Gaussian prior, which is defined by a mean vector and a symmetric and positive definite covariance matrix, and to use iterative projection methods to solve the corresponding regularized problem. However, a main challenge for many of these iterative methods is that the prior covariance matrix must be known and fixed (up to a constant) before starting the solution process. In this paper, we develop hybrid projection methods for inverse problems with mixed Gaussian priors where the prior covariance matrix is a convex combination of matrices and the mixing parameter and the regularization parameter do not need to be known in advance. Such scenarios may arise when data is used to generate a sample prior covariance matrix (e.g., in data assimilation) or when different priors are needed to capture different qualities of the solution. The proposed hybrid methods are based on a mixed Golub–Kahan process, which is an extension of the generalized Golub–Kahan bidiagonalization, and a distinctive feature of the proposed approach is that both the regularization parameter and the weighting parameter for the covariance matrix can be estimated automatically during the iterative process. Furthermore, for problems where training data are available, various data-driven covariance matrices (including those based on learned covariance kernels) can be easily incorporated. Numerical examples from tomographic reconstruction demonstrate the potential for these methods.

Target Detection Through Riemannian Geometric Approach With Application to Drone Detection

Radar detection of small drones in presence of noise and clutter is considered from a differential geometry viewpoint. The drone detection problem is challenging due to low radar cross section (RCS) of drones, especially in cluttered environments and when drones fly low and slow in urban areas. This paper proposes two detection techniques, the Riemannian-Brauer matrix (RBM) and the angle-based hybrid-Brauer (ABHB), to improve the probability of drone detection under small sample size and low signal-to-clutter ratio (SCR). These techniques are based on the regularized Burg algorithm (RBA), the Brauer disc (BD) theorem, and the Riemannian mean and distance. Both techniques exploit the RBA to obtain a Toeplitz Hermitian positive definite (THPD) covariance matrix from each snapshot and apply the BD theorem to cluster the clutter-plus-noise THPD covariance matrices. The proposed Riemannian-Brauer matrix technique is based on the Riemannian distance between the Riemannian mean of clutter-plus-noise cluster and potential targets. The proposed angle-based hybrid-Brauer technique uses the Euclidean tangent space and the Riemannian geodesical distances between the Riemannian mean, the Riemannian median and the potential target point. The angle at the potential target on the manifold is computed using the law of cosines on the manifold. The proposed detection techniques show advantage over the fast Fourier transform, the Riemannian distance-based matrix and the Kullback-Leibler (KLB) divergence detectors. The validity of both proposed techniques are demonstrated with real data.

Sensitivity of the Posterior Mean on the Prior Assumptions: An Application of the Ellipsoid Bound Theorem

This study examines the sensitivity of the posterior mean to change in the prior assumptions. Three plausible choices of prior which include informative, relative-non informative and non-informative priors are considered. The paper considers information level for a prior to cause a notable change in the Bayesian posterior point estimate. The study develops a framework for evaluating a bound for a robust posterior point estimate. The Ellipsoid Bound theorem is employed to derive the Ellipsoid Bound for an independent normal gamma prior distribution. The proposed modification ellipsoid bound for the large prior was establised by varrying different variance co-variance sizes for the independent normal gamma prior. This bound represents the range for the posterior mean when is insensitive and when it’s sensitive in both location and spread. The result shows that; for a large prior parameter value (greater than the OLS estimate) with a positive definite prior variance covariance matrix, and prior parameter values interval which contains the OLS estimate then, the posterior estimate will be less than both the OLS and the prior estimates. Similarly, if the lower bound of the prior parameter values range is greater than the OLS estimate then: The posterior estimate will be greater than the OLS estimate but smaller than the prior estimate. Furthermore, it is observed that no matter the degrees of confidence in the prior values, data information is powerful enough to modify it.

A General Version of Price\u2019s Theorem

Assume that X_{Sigma } in mathbb {R}^{n} is a centered random vector following a multivariate normal distribution with positive definite covariance matrix Sigma . Let g : mathbb {R}^{n} rightarrow mathbb {C} be measurable and of moderate growth, say |g(x)| lesssim (1 + |x|)^{N}. We show that the map Sigma mapsto mathbb {E}[g(X_{Sigma })] is smooth, and we derive convenient expressions for its partial derivatives, in terms of certain expectations mathbb {E}[(partial ^{alpha }g)(X_{Sigma })] of partial (distributional) derivatives of g. As we discuss, this result can be used to derive bounds for the expectation mathbb {E}[g(X_{Sigma })] of a nonlinear function g(X_{Sigma }) of a Gaussian random vector X_{Sigma } with possibly correlated entries. For the case when gleft( xright) = g_{1}(x_{1}) cdots g_{n}(x_{n}) has tensor-product structure, the above result is known in the engineering literature as Price’s theorem, originally published in 1958. For dimension n = 2, it was generalized in 1964 by McMahon to the general case g : mathbb {R}^{2} rightarrow mathbb {C}. Our contribution is to unify these results, and to give a mathematically fully rigorous proof. Precisely, we consider a normally distributed random vector X_{Sigma } in mathbb {R}^{n} of arbitrary dimension n in mathbb {N}, and we allow the nonlinearity g to be a general tempered distribution. To this end, we replace the expectation mathbb {E}left[ g(X_{Sigma })right] by the dual pairing leftlangle g,,phi _{Sigma }rightrangle _{mathcal {S}',mathcal {S}}, where phi _{Sigma } denotes the probability density function of X_{Sigma }.

Transfer Learning Algorithm of P300-EEG Signal Based on XDAWN Spatial Filter and Riemannian Geometry Classifier

The electroencephalogram (EEG) signal in the brain–computer interface (BCI) has suffered great cross-subject variability. The BCI system needs to be retrained before each time it is used, which is a waste of resources and time. Thus, it is difficult to generalize a fixed classification method for all subjects. Therefore, the transfer learning method proposed in this article, which combines XDAWN spatial filter and Riemannian Geometry classifier (RGC), can achieve offline cross-subject transfer learning in the P300-speller paradigm. The XDAWN spatial filter is used to enhanced the P300 components in the raw signal as well as reduce its dimensions. Then, the Riemannian Geometry Mean (RGM) is used as the reference matrix to perform the affine transformation of the symmetric positive definite (SPD) covariance matrix calculated from the filtered signal, which makes the data from different subjects comparable. Finally, the RGC is used to obtain the result of transfer learning experiments. The proposed algorithm was evaluated on two datasets (Dataset I from real patients and Dataset II from the laboratory). By comparing with two state-of-the-art and classic algorithms in the current BCI field, Ensemble of Support Vector Machine (E-SVM) and Stepwise Linear Discriminant Analysis (SWLDA), the maximum averaged area under the receiver operating characteristic curve (AUC) score of our algorithm reached 0.836, proving the potential of our proposed algorithm.

On Quantifying Expert Opinion about Multinomial Models that Contain Covariates

SummaryThe paper addresses the task of forming a prior distribution to represent expert opinion about a multinomial model that contains covariates. The task has not previously been addressed. We suppose that the sampling model is a multinomial logistic regression and represent expert opinion about the regression coefficients by a multivariate normal distribution. This logistic–normal model gives a flexible prior distribution that can capture a broad variety of expert opinion. The challenge is to find meaningful assessment tasks that an expert can perform and which should yield appropriate information to determine the values of parameters in the prior distribution, and to develop theory for determining the parameter values from the assessments. A method is proposed that meets this challenge. The method is implemented in interactive easy-to-use software that is freely available. It provides a graphical interface that the expert uses to assess quartiles of sets of proportions and the method determines a mean vector and a positive definite covariance matrix to represent the expert's opinions. The assessment tasks chosen yield parameter values that satisfy the usual laws of probability without the expert being aware of the constraints that this imposes. Special attention is given to feedback that encourages the expert to consider his or her opinions from a different perspective. The method is illustrated in an example that shows its viability and usefulness.

A Framework to Perform Asset Allocation Based on Partitional Clustering

Over the past years, many approaches to perform asset allocation have been proposed in the literature. Most of them tackle this problem as an optimization task, where the goal is to maximize return, whilst minimizing the risk. However, such approaches require the inversion of a positive-definite covariance matrix, usually resulting in the concentration of allocation, instability and low performance. Some methods have been recently introduced to solve this problem by facing it as a clustering problem. This paper introduces a framework for asset allocation based on partitional clustering algorithms. The idea is to segment the assets into clusters of correlated assets, allocate resources for each cluster and then within each cluster. The framework allows the use of different partitional clustering algorithms, intragroup and intergroup allocation methods. Also, various assessment criteria are considered, and a specialized initialization method is proposed for the clustering algorithm. The framework is evaluated with the Brazilian Stock Exchange (B3) data from the period 12/2005 to 04/2020. Different initialization methods are used for the clustering algorithm together with two intergroup and two intragroup techniques, resulting in five experimental scenarios. The results are compared with the Ibovespa index, the mean-variance model of Markowitz, and the risk-parity model recently proposed by Lopez de Prado.

Non-Gaussian Random Bi-matrix Models for Bi-free Central Limit Distributions with Positive Definite Covariance Matrices

In this paper, we construct random two-faced families of matrices with non-Gaussian entries to approximate a bi-free central limit distribution with a positive definite covariance matrix. We prove that, under modest conditions weaker than independence, a family of random two-faced families of matrices with non-Gaussian entries is asymptotically bi-free from a two-faced family of constant diagonal matrices.

A Machine Learning Approach to Estimating Large Positive Definite Covariance Matrix of High Frequency Data

This paper proposes a novel covariance estimator via a machine learning approach when both the sampling frequency and covariance dimension are large. Assuming that a large covariance matrix can be decomposed into low rank and sparse components, our method simultaneously provides a consistent estimation of these two components in a one-step procedure. Moreover, in the presence of microstructure noises and asynchronous trading, the covariance estimator is guaranteed to be positive definite with the optimal rate of convergence. Taking into account the serial dependent feature of financial data, we further provide a data-driven algorithm to select the optimal tuning parameters in practice. We apply the proposed estimator to vast portfolio allocations, which enjoy significantly enhanced out-of-sample portfolio risk and Sharpe ratios. The success of our approach helps justify the role that machine learning techniques play in finance.

Error Exponents for Dimension-Matched Vector Multiple Access Channels With Additive Noise

We analyze a class of vector multiple access channels with additive noise, where the sum of the dimensions of the transmitted signals matches that of the received signal. We first focus on the case without power constraints, in the Poltyrev sense, using point process techniques. We find the Poltyrev capacity region for noise processes that are independent and identically distributed over channel uses. For each rate vector strictly in the Poltyrev capacity region, we study, for each subset of the transmitters, the exponent of the decay in block length of the smallest possible probability that decoding results in error for each transmitter in that subset. In the case of independent and identically distributed Gaussian noise, with arbitrary positive definite covariance matrix, we derive random coding exponents for each type of error event—these are lower bounds to the true error exponents. This also leads to random coding error exponents in the traditional power-constrained case, where the power constraint at each transmitter is defined by an arbitrary positive definite matrix.

Computation of the maximum likelihood estimator in low-rank factor analysis

Factor analysis is a classical multivariate dimensionality reduction technique popularly used in statistics, econometrics and data science. Estimation for factor analysis is often carried out via the maximum likelihood principle, which seeks to maximize the Gaussian likelihood under the assumption that the positive definite covariance matrix can be decomposed as the sum of a low-rank positive semidefinite matrix and a diagonal matrix with nonnegative entries. This leads to a challenging rank constrained nonconvex optimization problem, for which very few reliable computational algorithms are available. We reformulate the low-rank maximum likelihood factor analysis task as a nonlinear nonsmooth semidefinite optimization problem, study various structural properties of this reformulation; and propose fast and scalable algorithms based on difference of convex optimization. Our approach has computational guarantees, gracefully scales to large problems, is applicable to situations where the sample covariance matrix is rank deficient and adapts to variants of the maximum likelihood problem with additional constraints on the model parameters. Our numerical experiments validate the usefulness of our approach over existing state-of-the-art approaches for maximum likelihood factor analysis.

A Convex Optimization Approach for Computing Correlated Choice Probabilities With Many Alternatives

A popular discrete choice model that incorporates correlation information is the multinomial probit (MNP) model where the random utilities of the alternatives are chosen from a multivariate normal distribution. Computing the choice probabilities is challenging in the MNP model when the number of alternatives is large. Mishra et al. (IEEE Transactions on Automatic Control, 2012) have proposed a semidefinite optimization approach to compute choice probabilities for the distribution of the random utilities that maximizes expected agent utility given only the mean, variance, and covariance information. Their model is referred to as the cross moment (CMM) model. Computing the choice probabilities with many alternatives is challenging in the CMM model, since one needs to solve large-scale semidefinite programs. We develop a simpler formulation as a representative agent model by maximizing over the choice probabilities in the unit simplex where the objective function is the sum of the expected utilities and a strongly concave perturbation function. By characterizing the perturbation function for the CMM model and its gradient, we develop a simple first-order gradient method with inexact line search to compute choice probabilities. We establish local linear convergence of this algorithm under mild assumptions on the choice probabilities. An implication of our results is that inverting the choice probabilities to compute the mean utilities is straightforward given any positive-definite covariance matrix. Numerical experiments show that this method can compute choice probabilities for a large number of alternatives within a reasonable amount of time while explicitly capturing the correlation information. Comparisons with simulation methods for MNP and semidefinite programming methods for CMM indicate the efficacy of the method.

Embedding road networks and travel time into distance metrics for urban modelling

ABSTRACTUrban environments are restricted by various physical, regulatory and customary barriers such as buildings, one-way systems and pedestrian crossings. These features create challenges for predictive modelling in urban space, as most proximity-based models rely on Euclidean (straight line) distance metrics which, given restrictions within the urban landscape, do not fully capture spatial urban processes. Here, we argue that road distance and travel time provide effective alternatives, and we develop a new low-dimensional Euclidean distance metric based on these distances using an isomap approach. The purpose of this is to produce a valid covariance matrix for Kriging. Our primary methodological contribution is the derivation of two symmetric dissimilarity matrices ( and ), with which it is possible to compute low-dimensional Euclidean metrics for the production of a positive definite covariance matrix with commonly utilised kernels. This new method is implemented into a Kriging predictor to estimate house prices on 3,669 properties in Coventry, UK. We find that a metric estimating a combination of road distance and travel time, in both and , produces a superior house price predictor compared with alternative state-of-the-art methods, that is, a standard Euclidean metric in and a non-restricted road distance metric in and . F

On the relationship between conditional (CAR) and simultaneous (SAR) autoregressive models

We clarify relationships between conditional (CAR) and simultaneous (SAR) autoregressive models. We review the literature on this topic and find that it is mostly incomplete. Our main result is that a SAR model can be written as a unique CAR model, and while a CAR model can be written as a SAR model, it is not unique. In fact, we show how any multivariate Gaussian distribution on a finite set of points with a positive-definite covariance matrix can be written as either a CAR or a SAR model. We illustrate how to obtain any number of SAR covariance matrices from a single CAR covariance matrix by using Givens rotation matrices on a simulated example. We also discuss sparseness in the original CAR construction, and for the resulting SAR weights matrix. For a real example, we use crime data in 49 neighborhoods from Columbus, Ohio, and show that a geostatistical model optimizes the likelihood much better than typical first-order CAR models. We then use the implied weights from the geostatistical model to estimate CAR model parameters that provides the best overall optimization.