Second Moment Matrix Research Articles

Gradient-Enhanced Kriging for High-Dimensional Bayesian Optimization with Linear Embedding

This paper explores the application of gradient-enhanced (GE) kriging for Bayesian optimization (BO) problems with a high-dimensional parameter space. We utilize the active subspace method to embed the original parameter space in a low-dimensional subspace. The active subspace is detected by analyzing the spectrum of the empirical second-moment matrix of the gradients of the response function. By mapping the training data onto their respective subspace, the objective function and the constraint functions are efficiently approximated with low-dimensional GE-kriging models. In each cycle of the BO procedure, a new point is found by maximizing the constrained expected improvement function within a low-dimensional polytope, and it is mapped back to the original space for model evaluation. In this way, the computational costs are significantly reduced when compared with standard GE-kriging. We illustrate and assess the proposed approach with numerical experiments ranging from academic benchmark problems to aerodynamic engineering applications. The experimental results show that the proposed method is promising for optimizing high-dimensional expensive objective functions, especially for problems that exhibit a clear low-dimensional active subspace.

Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method

In inverse problems, the parameters of a model are estimated based on observations of the model response. The Bayesian approach is powerful for solving such problems; one formulates a prior distribution for the parameter state that is updated with the observations to compute the posterior parameter distribution. Solving for the posterior distribution can be challenging when, e.g., prior and posterior significantly differ from one another and/or the parameter space is high-dimensional. We use a sequence of importance sampling measures that arise by tempering the likelihood to approach inverse problems exhibiting a significant distance between prior and posterior. Each importance sampling measure is identified by cross-entropy minimization as proposed in the context of Bayesian inverse problems in Engel et al. [J. Comput. Phys., 473 (2023), 111746]. To efficiently address problems with high-dimensional parameter spaces, we set up the minimization procedure in a low-dimensional subspace of the original parameter space. The principal idea is to analyze the spectrum of the second-moment matrix of the gradient of the log-likelihood function to identify a suitable subspace. Following Zahm et al. [Math. Comp., 91 (2022), pp. 1789–1835], an upper bound on the Kullback–Leibler divergence between full-dimensional and subspace posterior is provided, which can be utilized to determine the effective dimension of the inverse problem corresponding to a prescribed approximation error bound. We suggest heuristic criteria for optimally selecting the number of model and model gradient evaluations in each iteration of the importance sampling sequence. We investigate the performance of this approach using examples from engineering mechanics set in various parameter space dimensions.

Minimum-correction second-moment matching: theory, algorithms and applications

We address the problem of finding the closest matrix $$\tilde{\varvec{U}}$$ to a given $$\varvec{U}$$ under the constraint that a prescribed second-moment matrix $$\tilde{\varvec{P}}$$ must be matched, i.e. $$\tilde{\varvec{U}}^{\mathrm {T}}\tilde{\varvec{U}}=\tilde{\varvec{P}}$$ . We obtain a closed-form formula for the unique global optimizer $$\tilde{\varvec{U}}$$ for the full-rank case, that is related to $$\varvec{U}$$ by an SPD (symmetric positive definite) linear transform. This result is generalized to rank-deficient cases as well as to infinite dimensions. We highlight the geometric intuition behind the theory and study the problem’s rich connections to minimum congruence transform, generalized polar decomposition, optimal transport, and rank-deficient data assimilation. In the special case of $$\tilde{\varvec{P}}=\varvec{I}$$ , minimum-correction second-moment matching reduces to the well-studied optimal orthonormalization problem. We investigate the general strategies for numerically computing the optimizer and analyze existing polar decomposition and matrix square root algorithms. We modify and stabilize two Newton iterations previously deemed unstable for computing the matrix square root, such that they can now be used to efficiently compute both the orthogonal polar factor and the SPD square root. We then verify the higher performance of the various new algorithms using benchmark cases with randomly generated matrices. Lastly, we complete two applications for the stochastic Lorenz-96 dynamical system in a chaotic regime. In reduced subspace tracking using dynamically orthogonal equations, we maintain the numerical orthonormality and continuity of time-varying base vectors. In ensemble square root filtering for data assimilation, the prior samples are transformed into posterior ones by matching the covariance given by the Kalman update while also minimizing the corrections to the prior samples.

Anomalous Behavior of the Terrestrial Electric Field Intensity at Multiple Frequencies of Relativistic Binary Star Systems

The vertical component of the terrestrial electric field intensity has been proven to be spectrally localized at multiple frequencies of forty-three relativistic binary star systems. The used four time series contain multiannual observations of the terrestrial electric field intensity in the infralow frequency range. The used method is eigenoscopy developed by the present authors. The matrix of second sample moments is computed at finite time spans. The eigenvectors of the moment matrix and their amplitude spectra are computed and analyzed. A database of the discovered anomalous behavior has been built.

Comment on: Pseudo-True SDFs in Conditional Asset Pricing Models

Hansen and Jagannathan (1997) is a classic paper in empirical finance that provides a simple and appealing method for evaluating asset pricing models. It does so by quantifying the degree of misspecification among competing theories of the stochastic discount factor (SDF). The theories relevant for such an analysis typically imply the existence of r moment restrictions that can be expressed in the form ... Before getting into the specifics of the present article, let us rewind the clock and review how we got here. The measurement of model misspecification is the point of emphasis for Hansen and Jagannathan (1997) (HJ hereafter). Let us define the SDF candidate y(θ) to be a correctly specified if there exist values of θ for which (1) holds with equality. In this correctly specified case, generalized method of moments (GMM, Hansen (1982)) may be employed to consistently estimate θ⁠. Let us define the candidate SDF to be misspecified if there are no values of θ for which (1) holds with equality. Suppose we were to employ a GMM estimator of (1) while using the second-moment matrix of payoffs (or returns) to weight the GMM criterion function. HJ show that the square root of the minimized GMM objective function using this matrix has a special interpretation:

Calibrating second-moment matrix for better shape adaptation with bias term from directional filter bank

Anisotropic diffusion is an important noise reduction process in many pattern recognition systems. The local orientations and shape are encoded in a descriptor, called second-moment matrix, to be used as the central part of the process. For a noisy image, it becomes hard to find the elements of the matrix with reasonable accuracy. Two layers of subsequent Gaussian smoothing are applied conventionally: One is referred to as noise scale and the other integration scale. It is still not guaranteed that a second-moment matrix is correctly aligned with underlying pattern flow directions. It is largely believed that the integration scale handles this orientation discrepancy. Therefore, some researchers suggested a space-dependent integration scale strategy. We propose here a calibration strategy using directional filterbank (DFB). The second-moment matrix is rotated to go along the proper ridge directions with an angle correction term from DFB. The experiments conducted show promise for the calibrated anisotropic diffusion process in terms of improved recognition rate over that of the un-calibrated process.

New reformulations of distributionally robust shortest path problem

This paper considers a stochastic version of the shortest path problem, namely the Distributionally Robust Stochastic Shortest Path Problem (DRSSPP) on directed graphs. In this model, each arc has a deterministic cost and a random delay. The mean vector and the second-moment matrix of the uncertain data are assumed to be known, but the exact information of the distribution is unknown. A penalty occurs when the given delay constraint is not satisfied. The objective is to minimize the sum of the path cost and the expected path delay penalty. As this problem is NP-hard, we propose new reformulations and approximations using a sequence of semidefinite programming problems which provide tight lower bounds. Finally, numerical tests are conducted to illustrate the tightness of the bounds and the value of the proposed distributionally robust approach.

О построении моделей прогнозирования деятельности предприятия методом собственных состояний

In article the problem of forecasting of enterprises development is considered. For the solution of problem the eigenstate method is offered to use. At calculation of coefficients of eigenstates either covariance matrix or matrix of the initial second moments is offered to use. Building of enterprise model is carried out with use of those eigenstates which either satisfy conditions of effective enterprise development or most fully describe the main tendencies of enterprise development. The algorithm of forecasting represents iterative procedure of calculation of principal component of eigenstates. Research of efficiency of iterative algorithm is carried out on the example of forecasting of activity of the oil and gas company. Estimate of accuracy of the presented iterative algorithm is implemented by cross-validation procedure. Value of forecasting error received by means of iterative algorithm is compared with forecasting error value of the regression analysis.

An Affine SIFT Matching Algorithm Based on Local Patch Shape Estimation

In this paper we present an affine SIFT matching method to achieve reliable correspondence points in stereo matching with large viewpoint changes. We extended the affine invariant of the conventional SIFT approach by estimating the shape of the local patch around the interest point. Since we can obtain the scale information by SIFT detector, a second moment matrix (SMM) descriptor was employed to describe the shape. Furthermore, by comparing the shapes of the potential matches, we can normalize the template of SIFT descriptor and obtain the initial affine transformation. At last, we applied the iterative based method to achieve a fine registration with the estimated initial transformation parameters. The experiment results show that the proposed method is more robust to viewpoint changes and the accuracy of registration is better than feature based methods.

An Affine SIFT Matching Algorithm Based on Local Patch Shape Estimation

In this paper we present an affine SIFT matching method to achieve reliable correspondence points in stereo matching with large viewpoint changes. We extended the affine invariant of the conventional SIFT approach by estimating the shape of the local patch around the interest point. Since we can obtain the scale information by SIFT detector, a second moment matrix (SMM) descriptor was employed to describe the shape. Furthermore, by comparing the shapes of the potential matches, we can normalize the template of SIFT descriptor and obtain the initial affine transformation. At last, we applied the iterative based method to achieve a fine registration with the estimated initial transformation parameters. The experiment results show that the proposed method is more robust to viewpoint changes and the accuracy of registration is better than feature based methods.

Distributionally robust stochastic shortest path problem

Abstract This paper considers a stochastic version of the shortest path problem, the Distributionally Robust Stochastic Shortest Path Problem(DRSSPP) on directed graphs. In this model, the arc costs are deterministic, while each arc has a random delay. The mean vector and the second-moment matrix of the uncertain data are assumed known, but the exact information of the distribution is unknown. A penalty occurs when the given delay constraint is not satisfied. The objective is to minimize the sum of the path cost and the expected path delay penalty. As it is NP-hard, we approximate the DRSSPP with a semidefinite programming (SDP for short) problem, which is solvable in polynomial time and provides tight lower bounds.

Hessian-Based Affine Adaptation of Salient Local Image Features

Affine covariant local image features are a powerful tool for many applications, including matching and calibrating wide baseline images. Local feature extractors that use a saliency map to locate features require adaptation processes in order to extract affine covariant features. The most effective extractors make use of the second moment matrix (SMM) to iteratively estimate the affine shape of local image regions. This paper shows that the Hessian matrix can be used to estimate local affine shape in a similar fashion to the SMM. The Hessian matrix requires significantly less computation effort than the SMM, allowing more efficient affine adaptation. Experimental results indicate that using the Hessian matrix in conjunction with a feature extractor that selects features in regions with high second order gradients delivers equivalent quality correspondences in less than 17% of the processing time, compared to the same extractor using the SMM.

Mixed 0-1 Linear Programs Under Objective Uncertainty: A Completely Positive Representation

In this paper, we analyze mixed 0-1 linear programs under objective uncertainty. The mean vector and the second-moment matrix of the nonnegative objective coefficients are assumed to be known, but the exact form of the distribution is unknown. Our main result shows that computing a tight upper bound on the expected value of a mixed 0-1 linear program in maximization form with random objective is a completely positive program. This naturally leads to semidefinite programming relaxations that are solvable in polynomial time but provide weaker bounds. The result can be extended to deal with uncertainty in the moments and more complicated objective functions. Examples from order statistics and project networks highlight the applications of the model. Our belief is that the model will open an interesting direction for future research in discrete and linear optimization under uncertainty.

Moment-screen method for wave propagation in a refractive medium with random scattering

Direct numerical solution of a parabolic equation (PE) for the second moment of the sound field in a refracting medium with random scattering is described. The method determines the mean-square sound pressure without requiring generation of random realizations of the propagation medium. The second-moment matrix is factored into components that are independently propagated with a conventional PE algorithm. A moment screen is periodically applied to attenuate the coherence of the wavefield, much as phase screens are often applied in the method of random realizations. An example involving upwind and downwind propagation in the near-ground atmosphere shows that the new direct method converges to an accurate solution faster than the method of random realizations and is particularly well suited to calculations at low frequencies.

Testing the APT with Maximum Sharpe Ratio of Extracted Factors

This paper tests the asymptotic arbitrage pricing theory (APT) with the factors extracted from individual stocks. It is shown that the asymptotic APT fails if and only if the number of unbounded eigenvalues of the second-moment matrix of excess returns exceeds that of the variance matrix of excess returns by one. A test is developed using this result on the eigenvectors extracted by the Connor-Korajczyk (CK) method. The test statistic is shown to be related to the maximum Sharpe ratio among portfolios of all individual stocks. The empirical evidence, supplemented by simulation results, lends support to the implication of the asymptotic arbitrage pricing theory that the unconditional pricing error is small.

Bias and uncertainty in regression-calibrated models of groundwater flow in heterogeneous media

Groundwater models need to account for detailed but generally unknown spatial variability (heterogeneity) of the hydrogeologic model inputs. To address this problem we replace the large, m-dimensional stochastic vector β that reflects both small and large scales of heterogeneity in the inputs by a lumped or smoothed m-dimensional approximation γθ ∗, where γ is an interpolation matrix and θ ∗ is a stochastic vector of parameters. Vector θ ∗ has small enough dimension to allow its estimation with the available data. The consequence of the replacement is that model function f( γθ ∗) written in terms of the approximate inputs is in error with respect to the same model function written in terms of β, f( β), which is assumed to be nearly exact. The difference f( β) − f( γθ ∗), termed model error, is spatially correlated, generates prediction biases, and causes standard confidence and prediction intervals to be too small. Model error is accounted for in the weighted nonlinear regression methodology developed to estimate θ ∗ and assess model uncertainties by incorporating the second-moment matrix of the model errors into the weight matrix. Techniques developed by statisticians to analyze classical nonlinear regression methods are extended to analyze the revised method. The analysis develops analytical expressions for bias terms reflecting the interaction of model nonlinearity and model error, for correction factors needed to adjust the sizes of confidence and prediction intervals for this interaction, and for correction factors needed to adjust the sizes of confidence and prediction intervals for possible use of a diagonal weight matrix in place of the correct one. If terms expressing the degree of intrinsic nonlinearity for f( β) and f( γθ ∗) are small, then most of the biases are small and the correction factors are reduced in magnitude. Biases, correction factors, and confidence and prediction intervals were obtained for a test problem for which model error is large to test robustness of the methodology. Numerical results conform with the theoretical analysis.

SU ( N ) Wigner–Racah algebra for the matrix of second moments of embedded Gaussian unitary ensemble of random matrices

Recently Pluhar and Weidenmüller [Ann. Phys. (N.Y.) 297, 344 (2002)] showed that the eigenvectors of the matrix of second moments of embedded Gaussian unitary ensemble of random matrices generated by k-body interactions [EGUE(k)] for m fermions in N single particle states are SU(N) Wigner coefficients and derived also an expression for the eigenvalues. Going beyond this work, we will show that the eigenvalues of this matrix are square of a SU(N) Racah coefficient and thus the matrix of second moments of EGUE(k) is solved completely by SU(N) Wigner–Racah algebra.

Review of thek-body embedded ensembles of Gaussian random matrices

The embedded ensembles were introduced by Mon and French as physically more plausible stochastic models of many--body systems governed by one--and two--body interactions than provided by standard random--matrix theory. We review several approaches aimed at determining the spectral density, the spectral fluctuation properties, and the ergodic properties of these ensembles: moments methods, numerical simulations, the replica trick, the eigenvector decomposition of the matrix of second moments and supersymmetry, the binary correlation approximation, and the study of correlations between matrix elements.

Symmetry Properties of the k-Body Embedded Unitary Gaussian Ensemble of Random Matrices

We extend the recent study of the k-body embedded Gaussian ensembles by L. Benet, T. Rupp, and H. A. Weidenmüller (2001, Benet, Phys. Rev. Lett.87, 101601-1 and 2001, Ann. Phys. (N.Y.)292, 67) and by T. Asaga, L. Benet, T. Rupp, and H. A. Weidenmüller (cond-mat/0107363 and cond-mat/0107364). We show that central results of these papers can be derived directly from the symmetry properties of both the many-particle states and the random k-body interaction. We offer new insight into the structure of the matrix of second moments of the embedded ensemble and of the supersymmetry approach. We extend the concept of the embedded ensemble and define it purely group-theoretically.

Coherence-enhancing diffusion of colour images

Many image processing problems require the enhancement of coherent flow-like structures. This can be accomplished in a natural way by combining anisotropic diffusion filtering and texture analysis by means of the structure tensor (second-moment matrix, interest operator). In this paper an extension of these ideas to vector-valued images is presented. A structure tensor for vector-valued images is constructed as the mean of the structure tensors of each channel. This idea generalizes Di Zenzo's gradient for colour images by introducing an additional integration scale. The common structure tensor is used for steering the diffusion processes in each channel. The additional integration scale turns out to be crucial for orientation smoothing and it leads to significantly improved filter results. After analysing the role of all filter parameters, examples from different application areas are presented, and it is demonstrated that this type of diffusion filtering is highly robust under additive Gaussian noise.