True Subspace Research Articles

A Subspace-Inclusive Sampling Method for the Computational Design of Compositionally Graded Alloys

Abstract Compositionally graded alloys, a subclass of functionally graded materials (FGMs), utilize localized variations in composition with a single metal part to achieve higher performance than traditional single material parts. In previous work [Kirk, T., Galvan, E., Malak, R., and Arroyave, R., 2018, “Computational Design of Gradient Paths in Additively Manufactured Functionally Graded Materials,” J. Mech. Des., 140, p. 111410. 10.1115/1.4040816], the authors presented a computational design methodology that avoids common issues which limit a gradient alloy’s feasibility, such as deleterious phases, and optimizes for performance objectives. However, the previous methodology only samples the interior of a composition space, meaning designed gradients must include all elements in the space throughout the gradient. Because even small amounts of additional alloying elements can introduce new deleterious phases, this characteristic often neglects potentially simpler solutions to otherwise unsolvable problems and, consequently, discourages the addition of new elements to the state space. The present work improves upon the previous methodology by introducing a sampling method that includes subspaces with fewer elements in the design search. The new method samples within an artificially expanded form of the state space and projects samples outside the true region to the nearest true subspace. This method is evaluated first by observing the sample distribution in each subspace of a 3D, 4D, and 5D state space. Next, a parametric study in a synthetic 3D problem compares the performance of the new sampling scheme to the previous methodology. Lastly, the updated methodology is applied to design a gradient from stainless steel to equiatomic NiTi that has practical uses such as embedded shape memory actuation and for which the previous methodology fails to find a feasible path.

Effect of Sparse Array Geometry on Estimation of Co-array Signal Subspace

This paper considers the effect of sparse array geometry on the co-array signal subspace estimation error for Direction-of-Arrival (DOA) estimation. The second largest singular value of the signal covariance matrix plays an important role in controlling the distance between the true subspace and its estimate. For a special case of two closely-spaced sources impinging on the array, we explicitly compute the second largest singular value of the signal covariance matrix and show that it can be significantly larger for a nested array when compared against a uniform linear array with same number of sensors.

Distributed Principal Subspace Analysis for Partitioned Big Data: Algorithms, Analysis, and Implementation

Principal Subspace Analysis (PSA) -- and its sibling, Principal Component Analysis (PCA) -- is one of the most popular approaches for dimensionality reduction in signal processing and machine learning. But centralized PSA/PCA solutions are fast becoming irrelevant in the modern era of big data, in which the number of samples and/or the dimensionality of samples often exceed the storage and/or computational capabilities of individual machines. This has led to the study of distributed PSA/PCA solutions, in which the data are partitioned across multiple machines and an estimate of the principal subspace is obtained through collaboration among the machines. It is in this vein that this paper revisits the problem of distributed PSA/PCA under the general framework of an arbitrarily connected network of machines that lacks a central server. The main contributions of the paper in this regard are threefold. First, two algorithms are proposed in the paper that can be used for distributed PSA/PCA, with one in the case of data partitioned across samples and the other in the case of data partitioned across (raw) features. Second, in the case of sample-wise partitioned data, the proposed algorithm and a variant of it are analyzed, and their convergence to the true subspace at linear rates is established. Third, extensive experiments on both synthetic and real-world data are carried out to validate the usefulness of the proposed algorithms. In particular, in the case of sample-wise partitioned data, an MPI-based distributed implementation is carried out to study the interplay between network topology and communications cost as well as to study the effects of straggler machines on the proposed algorithms.

A Sequential Subspace Method for Millimeter Wave MIMO Channel Estimation

Data transmission over the mmWave in fifth-generation wireless networks aims to support very high speed wireless communications. A substantial increase in spectrum efficiency for mmWave transmission can be achieved by using advanced hybrid precoding, for which accurate channel state information is the key. Rather than estimating the entire channel matrix, directly estimating subspace information, which contains fewer parameters, does have enough information to design transceivers. However, the large channel use overhead and associated computational complexity in the existing channel subspace estimation techniques are major obstacles to deploy the subspace approach for channel estimation. In this paper, we propose a sequential two-stage subspace estimation method that can resolve the overhead issues and provide accurate subspace information. Utilizing a sequential method enables us to avoid manipulating the entire high-dimensional training signal, which greatly reduces the complexity. Specifically, in the first stage, the proposed method samples the columns of channel matrix to estimate its column subspace. Then, based on the obtained column subspace, it optimizes the training signals to estimate the row subspace. For a channel with $N_r$ receive antennas and $N_t$ transmit antennas, our analysis shows that the proposed technique only requires $O(N_t)$ channel uses, while providing a guarantee of subspace estimation accuracy. By theoretical analysis, it is shown that the similarity between the estimated subspace and the true subspace is linearly related to the signal-to-noise ratio (SNR), i.e., $O(\text{SNR})$, at high SNR, while quadratically related to the SNR, i.e., $O(\text{SNR}^2)$, at low SNR. Simulation results show that the proposed sequential subspace method can provide improved subspace accuracy, normalized mean squared error, and spectrum efficiency over existing methods.

Latent Complete Row Space Recovery for Multi-View Subspace Clustering

Multi-view subspace clustering has been applied to applications such as image processing and video surveillance, and has attracted increasing attention. Most existing methods learn view-specific self-representation matrices, and construct a combined affinity matrix from multiple views. The affinity construction process is time-consuming, and the combined affinity matrix is not guaranteed to reflect the whole true subspace structure. To overcome these issues, the Latent Complete Row Space Recovery (LCRSR) method is proposed. Concretely, LCRSR is based on the assumption that the multi-view observations are generated from an underlying latent representation, which is further assumed to collect the authentic samples drawn exactly from multiple subspaces. LCRSR is able to recover the row space of the latent representation, which not only carries complete information from multiple views but also determines the subspace membership under certain conditions. LCRSR does not involve the graph construction procedure and is solved with an efficient and convergent algorithm, thereby being more scalable to large-scale datasets. The effectiveness and efficiency of LCRSR are validated by clustering various kinds of multi-view data and illustrated in the background subtraction task.

Properties of the partial Cholesky factorization and application to reduced-rank adaptive beamforming

Reduced-rank adaptive beamforming is a well established and efficient methodology, notably for disturbance covariance matrices which are the sum of a strong low-rank component (interference) and a scaled identity matrix (thermal noise). Eigenvalue or singular decomposition is often used to achieve rank reduction. In this paper, we study and analyze an alternative, namely a partial Cholesky factorization, as a means to retrieve interference subspace and to compute reduced-rank beamformers. First, we study the angles between the true subspace and that obtained from partial Cholesky factorization of the covariance matrix. Then, a statistical analysis is carried out in finite samples. Using properties of partitioned Wishart matrices, we provide a stochastic representation of the beamformer based on partial Cholesky factorization and of the corresponding signal to interference and noise ratio loss. We show that the latter follows approximately a beta distribution, similarly to the beamformer based on eigenvalue decomposition. Finally, numerical simulations are presented which indicate that a reduced-rank adaptive beamformer based on partial Cholesky factorization incurs almost no loss, and can even perform better in some scenarios than its eigenvalue or singular value-based counterpart.

A convex formulation for high-dimensional sparse sliced inverse regression

SummarySliced inverse regression is a popular tool for sufficient dimension reduction, which replaces covariates with a minimal set of their linear combinations without loss of information on the conditional distribution of the response given the covariates. The estimated linear combinations include all covariates, making results difficult to interpret and perhaps unnecessarily variable, particularly when the number of covariates is large. In this paper, we propose a convex formulation for fitting sparse sliced inverse regression in high dimensions. Our proposal estimates the subspace of the linear combinations of the covariates directly and performs variable selection simultaneously. We solve the resulting convex optimization problem via the linearized alternating direction methods of multiplier algorithm, and establish an upper bound on the subspace distance between the estimated and the true subspaces. Through numerical studies, we show that our proposal is able to identify the correct covariates in the high-dimensional setting.

Subspace Estimation From Incomplete Observations: A High-Dimensional Analysis

We present a high-dimensional analysis of three popular algorithms, namely, Oja's method, GROUSE and PETRELS, for subspace estimation from streaming and highly incomplete observations. We show that, with proper time scaling, the time-varying principal angles between the true subspace and its estimates given by the algorithms converge weakly to deterministic processes when the ambient dimension $n$ tends to infinity. Moreover, the limiting processes can be exactly characterized as the unique solutions of certain ordinary differential equations (ODEs). A finite sample bound is also given, showing that the rate of convergence towards such limits is $\mathcal{O}(1/\sqrt{n})$. In addition to providing asymptotically exact predictions of the dynamic performance of the algorithms, our high-dimensional analysis yields several insights, including an asymptotic equivalence between Oja's method and GROUSE, and a precise scaling relationship linking the amount of missing data to the signal-to-noise ratio. By analyzing the solutions of the limiting ODEs, we also establish phase transition phenomena associated with the steady-state performance of these techniques.

Zonal structure of unbounded external-flow and aerodynamics

This paper starts from the far-field behaviors of velocity field in externally unbounded flow. We find that the well-known algebraic decay of disturbance velocity as derived kinematically is too conservative. Once the kinetics are taken into account by working on the fundamental solutions of far-field linearized Navier–Stokes equations, it is proven that the furthest far-field zone adjacent to the uniform fluid at infinity must be unsteady, viscous and compressible, where all disturbances degenerate to sound waves that decay exponentially. But this optimal rate does not exist in some commonly used simplified flow models, such as steady flow, incompressible flow and inviscid flow, because they actually work in true subspaces of the unbounded free space, which are surrounded by further far fields of different nature. This finding naturally leads to a zonal structure of externally unbounded flow field. The significance of the zonal structure is demonstrated by its close relevance to existing theories of aerodynamic force and moment in external flows, including the removal of the difficulties or paradoxes inherent in the simplified models.

Highly Efficient Framework for Predicting Interactions Between Proteins.

Protein-protein interactions (PPIs) play a central role in many biological processes. Although a large amount of human PPI data has been generated by high-throughput experimental techniques, they are very limited compared to the estimated 130 000 protein interactions in humans. Hence, automatic methods for human PPI-detection are highly desired. This work proposes a novel framework, i.e., Low-rank approximation-kernel Extreme Learning Machine (LELM), for detecting human PPI from a protein's primary sequences automatically. It has three main steps: 1) mapping each protein sequence into a matrix built on all kinds of adjacent amino acids; 2) applying the low-rank approximation model to the obtained matrix to solve its lowest rank representation, which reflects its true subspace structures; and 3) utilizing a powerful kernel extreme learning machine to predict the probability for PPI based on this lowest rank representation. Experimental results on a large-scale human PPI dataset demonstrate that the proposed LELM has significant advantages in accuracy and efficiency over the state-of-art approaches. Hence, this work establishes a new and effective way for the automatic detection of PPI.

Steady-State Statistical Performance Analysis of Subspace Tracking Methods

Subspace tracking using low-complexity methods is a common research area in signal processing. While numerical properties of available methods are extensively addressed in literature, there is a paucity of statistical results on limiting behavior of subspace tracking methods. In this contribution, we will study steady-state error of DPM, MALASE, Champagne's PA, Karasalo, and PAST methods. We show that DPM, MALASE, and PAST are approximations of the PA method, while Karasalo's method is equivalent to the PA method in convergence region. A tradeoff between steady-state error of methods and their local convergence rate is demonstrated. A nonstationary signal model is assumed, in which true subspace is slowly and randomly varying. Simulation results show PA and PAST are asymptotically close to the optimum sample covariance matrix (SCM), while this is not true for DPM and MALASE in stationary signal case. In the steady state, these methods can be expressed with a unified approach using stochastic first-order difference equations, whose fixed points are calculated. The results can be used to determine optimal step-size in terms of tolerable total signal subspace projection error. Simulation results confirm the validity of theoretical error calculations.

Robust DOA Estimation Using Independent Component Analysis

The paper deals with robust direction of arrival (DOA) estimation based on independent component analysis (ICA) in the presence of sensor gain perturbation and unknown noise fields. It has been shown that the noise subspace based estimator suffers from the drawback of being very sensitivity to orthogonality between the estimated noise subspace and true signal subspace. However, no single subspace feature extraction gives satisfactory results under different environments, but the proposed two-step subspace feature extraction method can efficiently cope with the limits. The noise-subspace demixing procedure of complex-valued FastICA is presented here, which is based on minor component analysis (MCA) for individual noise-eigenvector projection, and Newton’s iteration algorithm exploiting the minimization of the approximate negentropy of non-Gaussian signal for array signal processing. When a new noise subspace is attained, the proposed method can form the maximizing orthogonality especially for imperfect antenna array and make to facilitate the conventional noise-subspace based estimate method. Several computer simulation examples confirm the effectiveness of the proposed method and the performance improvement over the MUSIC estimator.

Robust Recovery of Subspace Structures by Low-Rank Representation

In this paper, we address the subspace clustering problem. Given a set of data samples (vectors) approximately drawn from a union of multiple subspaces, our goal is to cluster the samples into their respective subspaces and remove possible outliers as well. To this end, we propose a novel objective function named Low-Rank Representation (LRR), which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary. It is shown that the convex program associated with LRR solves the subspace clustering problem in the following sense: When the data is clean, we prove that LRR exactly recovers the true subspace structures; when the data are contaminated by outliers, we prove that under certain conditions LRR can exactly recover the row space of the original data and detect the outlier as well; for data corrupted by arbitrary sparse errors, LRR can also approximately recover the row space with theoretical guarantees. Since the subspace membership is provably determined by the row space, these further imply that LRR can perform robust subspace clustering and error correction in an efficient and effective way.

Minimum Mean Square Distance Estimation of a Subspace

We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace $U$ and its estimate $\hat{U}$ may not be adequate as the MSE is not the natural metric in the Grassmann manifold. As an alternative, we propose to carry out subspace estimation by minimizing the mean square distance (MSD) between $U$ and its estimate, where the considered distance is a natural metric in the Grassmann manifold, viz. the distance between the projection matrices. We show that the resulting estimator is no longer the posterior mean of $U$ but entails computing the principal eigenvectors of the posterior mean of $U U^{T}$. Derivation of the MMSD estimator is carried out in a few illustrative examples including a linear Gaussian model for the data and a Bingham or von Mises Fisher prior distribution for $U$. In all scenarios, posterior distributions are derived and the MMSD estimator is obtained either analytically or implemented via a Markov chain Monte Carlo simulation method. The method is shown to provide accurate estimates even when the number of samples is lower than the dimension of $U$. An application to hyperspectral imagery is finally investigated.

An Analysis of Linear Subspace Approaches for Computer Vision and Pattern Recognition

Linear subspace analysis (LSA) has become rather ubiquitous in a wide range of problems arising in pattern recognition and computer vision. The essence of these approaches is that certain structures are intrinsically (or approximately) low dimensional: for example, the factorization approach to the problem of structure from motion (SFM) and principal component analysis (PCA) based approach to face recognition. In LSA, the singular value decomposition (SVD) is usually the basic mathematical tool. However, analysis of the performance, in the presence of noise, has been lacking. We present such an analysis here. First, the "denoising capacity" of the SVD is analysed. Specifically, given a rank-r matrix, corrupted by noise--how much noise remains in the rank-r projected version of that corrupted matrix? Second, we study the "learning capacity" of the LSA-based recognition system in a noise-corrupted environment. Specifically, LSA systems that attempt to capture a data class as belonging to a rank-r column space will be affected by noise in both the training samples (measurement noise will mean the learning samples will not produce the "true subspace") and the test sample (which will also have measurement noise on top of the ideal clean sample belonging to the "true subspace"). These results should help one to predict aspects of performance and to design more optimal systems in computer vision, particularly in tasks, such as SFM and face recognition. Our analysis agrees with certain observed phenomenon, and these observations, together with our simulations, verify the correctness of our theory.

Laser–ultrasound image restoration using independent components analysis

A novel image enhancement method for laser ultrasound images is proposed. The method is based on Independent Component Analysis (ICA). We assume that the speckle noise comes from a signal source, which accompanies but is independent of the 'true signal source' and that their statistical characteristics are not the same. Thus the image restoration problem can also be described as 'signal source separation and enhancement problem. Then we classify the basis images that span into two different subspaces, namely 'true signal subspace' and 'speckle subspace'. Based on the classification result, we build our spatially adaptive filter and obtain the recovered image after a mixing transform. Our experimental results show that the proposed method achieves an improvement as compared to other methods for laser ultrasound images of structured objects blurred by speckle noises.

Spherical subspace tracking for efficient, high performance adaptive signal processing applications

In this paper we develop and analyze multilevel spherical subspace updating algorithms. These algorithms track eigenvectors and/or eigensubspaces associated with time-varying data that arises in adaptive signal processing, direction of arrival (DOA) or frequency tracking problems. Spherical subspaces can be created by setting groups of eigenvalues to their average value. Because of the sphericity, the subspace basis vectors can be reoriented to deflate the update problem and reduce computation. In this paper, we focus on a four-level update that efficiently tracks the signal subspace and generates just enough eigenlevel information to make strongly consistent signal subspace rank estimates. The four-level method is O( nr) per update when n is the data dimension and r is the signal subspace tank. In order to analyze the multilevel algorithms, we use Ljung's method to derive ordinary differential equations (ODEs) that characterize the asymptotic eigenlevel and subspace projection operator trajectories. Under certain weak assumptions, we prove that the multilevel algorithms asymptotically converge with probability one to the true subspace and eigenlevel values. We also present a four-level MDL-based rank detection scheme and show it to be strongly consistent. Simulation studies support the theoretical results and demonstrate the performance of our methods with frequency tracking data.

Estimation of direction-of-arrivals based on the array manifold space

In this paper, a high resolution technique for estimating DOAs of spatially close source signals is presented. It is observed that the array manifold over a sector of interest is rank deficient and the dimension of the array manifold space, which is the range space of the array manifold, is less than the number of sensors in the array. The true signal subspace is a subspace in the array manifold space. A novel technique is provided that searches for the signal subspace in this array manifold space. The resulting estimated signal subspace has minimum principal angles with the data signal subspace generated by eigen-decomposing the covariance matrix of the array data vector. It is proved that the proposed estimator is asymptotically consistent and the estimated signal subspace is closer to the true signal subspace than the data signal subspace formed by MUSIC. The proposed novel technique has better performance than the MUSIC algorithm. Its performance is comparable to MLE and MD-MUSIC yet it requires only one-dimensional searches and is computationally much less intense. Simulation results are presented to show the effectiveness of the proposed technique, and comparisons with MUSIC, MLE, and MD-MUSIC algorithms are also included.