Linear Subspace Methods Research Articles

Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling

Linear subspace methods that provide sufficient reconstruction of the data, such as PCA, offer an efficient way of dealing with missing pixels, outliers, and occlusions that often appear in the visual data. Discriminative methods, such as LDA, which, on the other hand, are better suited for classification tasks, are highly sensitive to corrupted data. We present a theoretical framework for achieving the best of both types of methods: An approach that combines the discrimination power of discriminative methods with the reconstruction property of reconstructive methods which enables one to work on subsets of pixels in images to efficiently detect and reject the outliers. The proposed approach is therefore capable of robust classification with a high-breakdown point. We also show that subspace methods, such as CCA, which are used for solving regression tasks, can be treated in a similar manner. The theoretical results are demonstrated on several computer vision tasks showing that the proposed approach significantly outperforms the standard discriminative methods in the case of missing pixels and images containing occlusions and outliers.

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

We present a modular linear discriminant analysis (LDA) approach for face recognition. A set of observers is trained independently on different regions of frontal faces and each observer projects face images to a lower-dimensional subspace. These lower-dimensional subspaces are computed using LDA methods, including a new algorithm that we refer to as direct, weighted LDA or DW-LDA. DW-LDA combines the advantages of two recent LDA enhancements, namely direct LDA (D-LDA) and weighted pairwise Fisher criteria. Each observer performs recognition independently and the results are combined using a simple sum-rule. Experiments compare the proposed approach to other face recognition methods that employ linear dimensionality reduction. These experiments demonstrate that the modular LDA method performs significantly better than other linear subspace methods. The results also show that D-LDA does not necessarily perform better than the well-known principal component analysis followed by LDA approach. This is an important and significant counterpoint to previously published experiments that used smaller databases. Our experiments also indicate that the new DW-LDA algorithm is an improvement over D-LDA.

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

We present a modular linear discriminant analysis (LDA) approach for face recognition. A set of observers is trained independently on different regions of frontal faces and each observer projects face images to a lower-dimensional subspace. These lower-dimensional subspaces are computed using LDA methods, including a new algorithm that we refer to as direct, weighted LDA or DW-LDA. DW-LDA combines the advantages of two recent LDA enhancements, namely direct LDA (D-LDA) and weighted pairwise Fisher criteria. Each observer performs recognition independently and the results are combined using a simple sum-rule. Experiments compare the proposed approach to other face recognition methods that employ linear dimensionality reduction. These experiments demonstrate that the modular LDA method performs significantly better than other linear subspace methods. The results also show that D-LDA does not necessarily perform better than the well-known principal component analysis followed by LDA approach. This is an important and significant counterpoint to previously published experiments that used smaller databases. Our experiments also indicate that the new DW-LDA algorithm is an improvement over D-LDA.

Statistically stable ultrasonic imaging in random media.

Analysis of array data from acoustic scattering in a random medium with a small number of isolated targets is performed in order to image and, thereby, localize the spatial position of each target. Because the host medium has random fluctuations in wave speed, the background medium is itself a source of scattered energy. It is assumed, however, that the targets are sufficiently larger and/or more reflective than the background fluctuations so that a clear distinction can be made between targets and background scatterers. In numerical simulations nonreflective boundary conditions are used so as to isolate the effects of the host randomness from those of the spatial boundaries, which can then be treated in a separate analysis. It is shown that the key to successful imaging is finding statistically stable functionals of the data whose extreme values provide estimates of scatterer locations. The best ones are related to the eigenfunctions and eigenvalues of the array response matrix, just as one might expect from prior work on array data processing in complex scattering media having homogeneous backgrounds. The specific imaging functionals studied include matched-field processing and linear subspace methods, such as MUSIC (MUtiple SIgnal Classification). But statistical stability is not characteristic of the frequency domain, which is often the province of these methods. By transforming back into the time domain after first diagonalizing the array data in the frequency domain, one can take advantage of both the time-domain stability and the frequency-domain orthogonality of the relevant eigenfunctions.

Time-reversal acoustics and maximum-entropy imaging

A common problem in acoustics imaging is target location using either passive or active data inversion. Time-reversal methods in acoustics have the important characteristic that they provide a means of determining the eigenfunctions and eigenvalues of the scattering operator for either of these two cases. Each eigenfunction may often be approximately associated with an individual scatterer. The resulting decoupling of the scattered field from a collection of targets is a very useful aid to localizing the targets, and suggests a number of imaging and localization algorithms. Two of these are linear subspace methods and maximum-entropy imaging.

Principal component analysis of the dynamic response measured by fMRI: a generalized linear systems framework

Principal component analysis (PCA) is one of several structure-seeking multivariate statistical techniques, exploratory as well as inferential, that have been proposed recently for the characterization and detection of activation in both PET and fMRI time series data. In particular, PCA is data driven and does not assume that the neural or hemodynamic response reaches some steady state, nor does it involve correlation with any pre-defined or exogenous experimental design template. In this paper, we present a generalized linear systems framework for PCA based on the singular value decomposition (SVD) model for representation of spatio–temporal fMRI data sets. Statistical inference procedures for PCA, including point and interval estimation will be introduced without the constraint of explicit hypotheses about specific task-dependent effects. The principal eigenvectors capture both the spatial and temporal aspects of fMRI data in a progressive fashion; they are inherently matched to unique and uncorrelated features and are ranked in order of the amount of variance explained. PCA also acts as a variation reduction technique, relegating most of the random noise to the trailing components while collecting systematic structure into the leading ones. Features summarizing variability may not directly be those that are the most useful. Further analysis is facilitated through linear subspace methods involving PC rotation and strategies of projection pursuit utilizing a reduced, lower-dimensional natural basis representation that retains most of the information. These properties will be illustrated in the setting of dynamic time-series response data from fMRI experiments involving pharmacological stimulation of the dopaminergic nigro-striatal system in primates.

Epipolar Geometry and Linear Subspace Methods: A New Approach to Weak Calibration

This paper addresses the problem of estimating the epipolar geometry from point correspondences between two images taken by uncalibrated perspective cameras. It is shown that Jepson‘s and Heeger‘s linear subspace technique for infinitesimal motion estimation can be generalized to the finite motion case by choosing an appropriate basis for projective space. This yields a linear method for weak calibration. The proposed algorithm has been implemented and tested on both real and synthetic images, and it is compared to other linear and non-linear approaches to weak calibration.

Multigrid preconditioned Krylov subspace methods for three-dimensional numerical solutions of the incompressible Navier–Stokes equations

We consider numerical methods for the incompressible Reynolds averaged Navier–Stokes equations discretized by finite difference techniques on non-staggered grids in body-fitted coordinates. A segregated approach is used to solve the pressure–velocity coupling problem. Several iterative pressure linear solvers including Krylov subspace and multigrid methods and their combination have been developed to compare the efficiency of each method and to design a robust solver. Three-dimensional numerical experiments carried out on scalar and vector machines and performed on different fluid flow problems show that a combination of multigrid and Krylov subspace methods is a robust and efficient pressure solver.