Empirical Eigenvectors Research Articles

A note on the prediction error of principal component regression in high dimensions

We analyze the prediction error of principal component regression (PCR) and prove high probability bounds for the corresponding squared risk conditional on the design. Our first main result shows that PCR performs comparably to the oracle method obtained by replacing empirical principal components by their population counterparts, provided that an effective rank condition holds. On the other hand, if the latter condition is violated, then empirical eigenvalues start to have a significant upward bias, resulting in a self-induced regularization of PCR. Our approach relies on the behavior of empirical eigenvalues, empirical eigenvectors and the excess risk of principal component analysis in high-dimensional regimes.

On the eigenstructure of covariance matrices with divergent spikes

For a generalization of Johnstone’s spiked model, a covariance matrix with eigenvalues all one but M of them, the number of features N comparable to the number of samples n:N=N(n),M=M(n),γ−1≤Nn≤γ where γ∈(0,∞), we obtain consistency rates in the form of CLTs for separated spikes tending to infinity fast enough whenever M grows slightly slower than n:limn→∞ logn logn M(n)=0. Our results fill a gap in the existing literature in which the largest range covered for the number of spikes has been o(n1∕6) and reveal a certain degree of flexibility for the centering in these CLTs inasmuch as it can be empirical, deterministic, or a sum of both. Furthermore, we derive consistency rates of their corresponding empirical eigenvectors to their true counterparts, which turn out to depend on the relative growth of these eigenvalues.

Simple: Statistical Inference on Membership Profiles in Large Networks

AbstractNetwork data are prevalent in many contemporary big data applications in which a common interest is to unveil important latent links between different pairs of nodes. Yet a simple fundamental question of how to precisely quantify the statistical uncertainty associated with the identification of latent links still remains largely unexplored. In this paper, we propose the method of statistical inference on membership profiles in large networks (SIMPLE) in the setting of degree-corrected mixed membership model, where the null hypothesis assumes that the pair of nodes share the same profile of community memberships. In the simpler case of no degree heterogeneity, the model reduces to the mixed membership model for which an alternative more robust test is also proposed. Both tests are of the Hotelling-type statistics based on the rows of empirical eigenvectors or their ratios, whose asymptotic covariance matrices are very challenging to derive and estimate. Nevertheless, their analytical expressions are unveiled and the unknown covariance matrices are consistently estimated. Under some mild regularity conditions, we establish the exact limiting distributions of the two forms of SIMPLE test statistics under the null hypothesis and contiguous alternative hypothesis. They are the chi-square distributions and the noncentral chi-square distributions, respectively, with degrees of freedom depending on whether the degrees are corrected or not. We also address the important issue of estimating the unknown number of communities and establish the asymptotic properties of the associated test statistics. The advantages and practical utility of our new procedures in terms of both size and power are demonstrated through several simulation examples and real network applications.

Bayesian surface regression versus spatial spectral nonparametric curve regression

COVID-19 incidence is analyzed at the provinces of the Spanish Communities in the Iberian Peninsula during the period February–October, 2020. Two infinite-dimensional regression approaches, surface regression and spatial curve regression, are proposed. In the first one, Bayesian maximum a posteriori (MAP) estimation is adopted in the approximation of the pure point spectrum of the temporal regression residual autocorrelation operator. Thus, an alternative to the moment-based estimation methodology developed in Ruiz-Medina, Miranda and Espejo (2019) is derived. Additionally, spatial curve regression is considered. A nonparametric estimator of the spectral density operator, based on the spatial periodogram operator, is computed to approximate the spatial correlation between curves. Dimension reduction is achieved by projection onto the empirical eigenvectors of the long-run spatial covariance operator. Cross-validation procedures are implemented to test the performance of the two functional regression approaches.

Connectivity-informed adaptive regularization for generalized outcomes.

One of the challenging problems in neuroimaging is the principled incorporation of information from different imaging modalities. Data from each modality are frequently analyzed separately using, for instance, dimensionality reduction techniques, which result in a loss of mutual information. We propose a novel regularization method, generalized ridgified Partially Empirical Eigenvectors for Regression (griPEER), to estimate associations between the brain structure features and a scalar outcome within the generalized linear regression framework. griPEER improves the regression coefficient estimation by providing a principled approach to use external information from the structural brain connectivity. Specifically, we incorporate a penalty term, derived from the structural connectivity Laplacian matrix, in the penalized generalized linear regression. In this work, we address both theoretical and computational issues and demonstrate the robustness of our method despite incomplete information about the structural brain connectivity. In addition, we also provide a significance testing procedure for performing inference on the estimated coefficients. Finally, griPEER is evaluated both in extensive simulation studies and using clinical data to classify HIV+ and HIV- individuals.

Estimating the principal components of correlation matrices from all their empirical eigenvectors

We consider the problem of estimating the principal components of a population covariance matrix from a limited number of measurement data. Using a combination of random matrix and information-theoretic tools, we show that all the eigenmodes of the sample correlation matrices are informative, and not only the top ones. We show how this information can be exploited when prior information about the principal component, such as whether it is localized or not, is available by mapping the estimation problem onto the search for the ground state of a spin-glass–like effective Hamiltonian encoding the prior. Results are illustrated numerically on the spiked covariance model.

Non-asymptotic adaptive prediction in functional linear models

Functional linear regression has recently attracted considerable interest. Many works focus on asymptotic inference. In this paper we consider in a non asymptotic framework a simple estimation procedure based on functional Principal Regression. It revolves in the minimization of a least square contrast coupled with a classical projection on the space spanned by the m first empirical eigenvectors of the covariance operator of the functional sample. The novelty of our approach is to select automatically the crucial dimension m by minimization of a penalized least square contrast. Our method is based on model selection tools. Yet, since this kind of methods consists usually in projecting onto known non-random spaces, we need to adapt it to empirical eigenbasis made of data-dependent–hence random–vectors. The resulting estimator is fully adaptive and is shown to verify an oracle inequality for the risk associated to the prediction error and to attain optimal minimax rates of convergence over a certain class of ellipsoids. Our strategy of model selection is finally compared numerically with cross-validation.

Structured penalties for functional linear models—partially empirical eigenvectors for regression

One of the challenges with functional data is incorporating geometric structure, or local correlation, into the analysis. This structure is inherent in the output from an increasing number of biomedical technologies, and a functional linear model is often used to estimate the relationship between the predictor functions and scalar responses. Common approaches to the problem of estimating a coefficient function typically involve two stages: regularization and estimation. Regularization is usually done via dimension reduction, projecting onto a predefined span of basis functions or a reduced set of eigenvectors (principal components). In contrast, we present a unified approach that directly incorporates geometric structure into the estimation process by exploiting the joint eigenproperties of the predictors and a linear penalty operator. In this sense, the components in the regression are 'partially empirical' and the framework is provided by the generalized singular value decomposition (GSVD). The form of the penalized estimation is not new, but the GSVD clarifies the process and informs the choice of penalty by making explicit the joint influence of the penalty and predictors on the bias, variance and performance of the estimated coefficient function. Laboratory spectroscopy data and simulations are used to illustrate the concepts.

Dimensional model reduction in non‐linear finite element dynamics of solids and structures

AbstractA general approach to the dimensional reduction of non‐linear finite element models of solid dynamics is presented. For the Newmark implicit time‐discretization, the computationally most expensive phase is the repeated solution of the system of linear equations for displacement increments. To deal with this, it is shown how the problem can be formulated in an approximation (Ritz) basis of much smaller dimension. Similarly, the explicit Newmark algorithm can be also written in a reduced‐dimension basis, and the computation time savings in that case follow from an increase in the stable time step length.In addition, the empirical eigenvectors are proposed as the basis in which to expand the incremental problem. This basis achieves approximation optimality by using computational data for the response of the full model in time to construct a reduced basis which reproduces the full system in a statistical sense. Because of this ‘global’ time viewpoint, the basis need not be updated as with reduced bases computed from a linearization of the full finite element model.If the dynamics of a finite element model is expressed in terms of a small number of basis vectors, the asymptotic cost of the solution with the reduced model is lowered and optimal scalability of the computational algorithm with the size of the model is achieved. At the same time, numerical experiments indicate that by using reduced models, substantial savings can be achieved even in the pre‐asymptotic range. Furthermore, the algorithm parallelizes very efficiently.The method we present is expected to become a useful tool in applications requiring a large number of repeated non‐linear solid dynamics simulations, such as convergence studies, design optimization, and design of controllers of mechanical systems. Copyright © 2001 John Wiley & Sons, Ltd.

Min-norm interpretations and consistency of MUSIC, MODE and ML

The multiple signal characterization (MUSIC) approach, its generalization to correlated signals known as the method of direction estimation (MODE), and the deterministic maximum likelihood (ML) approach for bearing estimation in array processing are shown to be signal subspace fitting approaches in a minimum norm sense. MODE, for example, is shown to be an approach in which the array manifold is linearly estimated from principal empirical eigenvectors in a minimum weighted Frobenius norm sense. Using the min-norm interpretations, a unified proof for strong consistency of the three approaches is provided for stationary and ergodic signals.

Empirical determination of intermediate-coupling amplitudes and transition rates from spectroscopic data.

A simple general method is presented whereby measured spectroscopic energy-level data can be used to empirically construct the eigenvectors of intermediate coupling for an arbitrary configuration. These empirical eigenvectors can be used to elucidate the transition rates into, out of, and among the levels of the configuration by separating the effects of intermediate coupling from the radial transition moment. Predictive applications to the ${\mathit{np}}^{3}$ configurations of the N and P isoelectronic sequences are presented.

Distribution of sulfur dioxide over metropolitan St. Louis, as described by empirical Eigenvectors, and its relation to meteorological parameters

Patterns of sulfur dioxide concentrations at St. Louis, Missouri, are described objectively and are related to meteorological parameters. The patterns were derived from mean 24-hour SO 2 concentrations measured at 30 locations on 49 days during the winter of 1964–1965. Eigenvectors, or empirical orthogonal functions, were used for this analysis, which showed that the observations were sufficiently interrelated that several representative patterns of SO 2 throughout the metropolitan area could be delineated. The first three such derived patterns accounted for nearly two-thirds of the variance of the observations. The day-to-day variability of the derived patterns could be largely explained by local meteorological conditions. The multiple correlations between each of the first three eigenvector coefficients and significant meteorological parameters were greater than 0.82; the largest such correlation was 0.89.

Empirical Eigenvectors of Sea-Level Pressure, Surface Temperature and Precipitation Complexes over North America

Abstract The combined representation of fields of three climatic variables with empirical orthogonal functions, herein referred to as eigenvectors, is discussed. The eigenvectors are derived from measurements of monthly mean sea-level pressure, surface temperature and precipitation at 23 points in North America for 25 Januarys. Selected eigenvectors of the individual climatic variables are presented; however, the major part of the paper is devoted to the presentation of eigenvectors consisting of combinations of three climatic variables. Empirical eigenvectors derived from fields of two or more meteorological variables have been used in statistical prediction models, but none of the studies to date displayed examples of these eigenvectors or discussed the internal consistency of the combined representations. In this paper it is shown that the structure of the covariances between the three climatic variables,as portrayed by the combined representations, is consistent with synoptic experience. This result i...