Projection Subspaces Research Articles

Support vector machine with discriminative low‐rank embedding

AbstractSupport vector machine (SVM) is a binary classifier widely used in machine learning. However, neglecting the latent data structure in previous SVM can limit the performance of SVM and its extensions. To address this issue, the authors propose a novel SVM with discriminative low‐rank embedding (LRSVM) that finds a discriminative latent low‐rank subspace more suitable for SVM classification. The extension models of LRSVM are introduced by imposing different orthogonality constraints to prevent computational inaccuracies. A detailed derivation of the authors’ iterative algorithms are given that is essentially for solving the SVM on the low‐rank subspace. Additionally, some theorems and properties of the proposed models are presented by the authors. It is worth mentioning that the subproblems of the proposed algorithms are equivalent to the standard or the weighted linear discriminant analysis (LDA) problems. This indicates that the projection subspaces obtained by the authors’ algorithms are more suitable for SVM classification compared to those from the LDA method. The convergence analysis for the authors proposed algorithms are also provided. Furthermore, the authors conduct experiments on various machine learning data sets to evaluate the algorithms. The experiment results show that the authors’ algorithms perform significantly better than other algorithms, which indicates their superior abilities on classification tasks.

A Proposed Clustering Algorithm for Efficient Clustering of High-Dimensional Data

To partition transaction data values, clustering algorithms are used. To analyse the relationships between transactions, similarity measures are utilized. Similarity models based on vectors perform well with low-dimensional data. High-dimensional data values are clustered using subspace clustering techniques. Clustering high-dimensional data is difficult due to the curse of dimensionality. Projective clustering seeks out projected clusters in subsets of a data space's dimensions. In high-dimensional data space, a probability model represents predicted clusters. A model-based fuzzy projection clustering method to find clusters with overlapping boundaries in different projection subspaces. The system employs the Model Based Projective Clustering (MPC) method. To cluster high-dimensional data, projective clustering algorithms are used. A subspace clustering technique is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been optimized. A subspace clustering approach is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been improved.

Derivative Interpolating Subspace Frameworks for Nonlinear Eigenvalue Problems

We first consider the problem of approximating a few eigenvalues of a proper rational matrix-valued function closest to a prescribed target. It is assumed that the proper rational matrix-valued function is expressed in the transfer function form $H(s) = C (sI - A)^{-1} B$, where the middle factor is large, whereas the number of rows of $C$ and the number of columns of $B$ are equal and small. We propose a subspace framework that performs two-sided projections on the state-space representation of $H(\cdot)$, commonly employed in model reduction and giving rise to a reduced transfer function. At every iteration, the projection subspaces are expanded to attain Hermite interpolation conditions at the eigenvalues of the reduced transfer function closest to the target, which in turn leads to a new reduced transfer function. We prove in theory that, when a sequence of eigenvalues of the reduced transfer functions converges to an eigenvalue of the full problem, it converges at least at a quadratic rate. In the second part, we extend the proposed framework to locate the eigenvalues of a general square large-scale nonlinear meromorphic matrix-valued function $T(\cdot)$, where we exploit a representation ${\mathcal R}(s) = C(s) A(s)^{-1} B(s) - D(s)$ defined in terms of the block components of $T(\cdot)$. The numerical experiments illustrate that the proposed framework is reliable in locating a few eigenvalues closest to the target point, and that, with respect to runtime, it is competitive to established methods for nonlinear eigenvalue problems.

Projection-compliant database generation

Synthesizing data using declarative formalisms has been persuasively advocated in contemporary data generation frameworks. In particular, they specify operator output volumes through row-cardinality constraints. However, thus far, adherence to these volumetric constraints has been limited to the Filter and Join operators. A critical deficiency is the lack of support for the Projection operator, which is at the core of basic SQL constructs such as Distinct, Union and Group By. The technical challenge here is that cardinality unions in multi-dimensional space, and not mere summations, need to be captured in the generation process. Further, dependencies across different data subspaces need to be taken into account. We address the above lacuna by presenting PiGen , a dynamic data generator that incorporates Projection cardinality constraints in its ambit. The design is based on a projection subspace division strategy that supports the expression of constraints using optimized linear programming formulations. Further, techniques of symmetric refinement and workload decomposition are introduced to handle constraints across different projection subspaces. Finally, PiGen supports dynamic generation, where data is generated on-demand during query processing, making it amenable to Big Data environments. A detailed evaluation on workloads derived from real-world and synthetic benchmarks demonstrates that PiGen can accurately and efficiently model Projection outcomes, representing an essential step forward in customized database generation.

Advanced variations of two-dimensional principal component analysis for face recognition

The two-dimensional principal component analysis (2DPCA) has been one of the basic methods of developing artificial intelligent algorithms. To increase the feasibility, we propose a new general ridge regression model for 2DPCA and variations, with extracting low dimensional features under two projection subspaces. A new relaxed 2DPCA under the quaternion framework is proposed to utilize the label (if known) and color information to compute the essential features of generalization ability with optimization algorithms. The 2DPCA-based approaches for face recognition are also improved by weighting each principle component a scatter measure, which increases efficiently the rate of face recognition. In numerical experiments on well-known standard databases, the R2DPCA approach has high generalization ability and achieves a higher recognition rate than the state-of-the-art 2DPCA-like methods, and has better performance than the basic deep learning methods such as CNNs, DBNs, and DNNs in the small-sample case.

Principal component analysis for multivariate extremes

In the probabilistic framework of multivariate regular variation, the first order behavior of heavy-tailed random vectors above large radial thresholds is ruled by a homogeneous limit measure. For a high dimensional vector, a reasonable assumption is that the support of this measure is concentrated on a lower dimensional subspace, meaning that certain linear combinations of the components are much likelier to be large than others. Identifying this subspace and thus reducing the dimension will facilitate a refined statistical analysis. In this work we apply Principal Component Analysis (PCA) to a re-scaled version of radially thresholded observations. Within the statistical learning framework of empirical risk minimization, our main focus is to analyze the squared reconstruction error for the exceedances over large radial thresholds. We prove that the empirical risk converges to the true risk, uniformly over all projection subspaces. As a consequence, the best projection subspace is shown to converge in probability to the optimal one, in terms of the Hausdorff distance between their intersections with the unit sphere. In addition, if the exceedances are re-scaled to the unit ball, we obtain finite sample uniform guarantees to the reconstruction error pertaining to the estimated projection subspace. Numerical experiments illustrate the capability of the proposed framework to improve estimators of extreme value parameters.

An alternative extended block Arnoldi method for solving low-rank Sylvester equations

Projection methods based on Krylov subspaces are an effective tool for solving low-rank Sylvester matrix equations in the large scale cases. The initial problem is projected onto a sequence of nested subspaces. At each step of the process, a reduced Sylvester equation, whose coefficient matrices are restrictions of the original coefficient matrices to the projection subspaces, are solved by a direct method. In this paper, after recalling some important properties about the extended block Arnoldi process, we propose an alternative approach that enables us to determine approximate solutions by solving projected and reduced Sylvester equations whose coefficient matrices are restrictions of the inverse of the original matrices. Some numerical experiments are given in order to illustrate the performance of our approach.

Robust Adaptive Beamforming With Subspace Projection and Covariance Matrix Reconstruction

In this paper, we present a subspace projection and covariance matrix reconstruction (SPCMR) algorithm for adaptive beamforming to improve the robustness against large SV mismatch. The SPCMR algorithm consists of two parts: projection subspaces estimation and interference-plus-noise covariance matrix (INCM) reconstruction. Specifically, we estimate two projection subspaces containing the signal component and obtain the signal SV from their intersection. The first projection subspace is estimated from the constructed signal covariance matrix via the distortionless responses principle. The second one is gotten according to the subspace proximity between the nominal signal SV and the eigenvectors of the sample covariance matrix. Subsequently, the interference SVs are estimated by using the Capon spatial estimator, and each interference power is obtained via the oblique projectors. After that, an accurate INCM is reconstructed, and the SPCMR beamformer is proposed. The simulation results show that the SPCMR algorithm is robust to several model mismatches and outperforms other adaptive algorithms.

Unsupervised Joint Subspace and Dictionary Learning for Enhanced Cross-Domain Person Re-Identification

Person re-identification (Re-ID) has drawn increasing attention from both academia and industry due to its great potentials in surveillance applications. Most existing research efforts have attempted to tackle cross-view variation in single-domain person Re-ID. However, there is still a lack of effective approaches to cross-domain person Re-ID problem. In this paper, an unsupervised joint subspace and dictionary learning (UJSDL) framework is proposed to address the cross-domain person Re-ID problem, where both cross-view (i.e., across different cameras in the same network of cameras) and cross-domain (i.e., across different network of cameras) variation are jointly addressed. In particular, to reduce the impact of cross-view distribution variation, the graph Laplacian approach is used to project the images from different camera views in each domain into a shared subspace. To alleviate the impact of cross-domain distribution variation, a shared dictionary is learned from all the projection subspaces such that the discriminative information from both the labeled source datasets and the unlabeled target dataset are well encoded. To efficiently solve the joint subspace and dictionary learning task, an alternating optimization algorithm is presented. We used multiple different feature sets and conducted experiments on multiple benchmark datasets as the target domain. The results demonstrate that UJSDL outperforms the state-of-the-art approaches.

Interpolatory model reduction of parameterized bilinear dynamical systems

Interpolatory projection methods for model reduction of nonparametric linear dynamical systems have been successfully extended to nonparametric bilinear dynamical systems. However, this has not yet occurred for parametric bilinear systems. In this work, we aim to close this gap by providing a natural extension of interpolatory projections to model reduction of parametric bilinear dynamical systems. We introduce necessary conditions that the projection subspaces must satisfy to obtain parametric tangential interpolation of each subsystem transfer function. These conditions also guarantee that the parameter sensitivities (Jacobian) of each subsystem transfer function are matched tangentially by those of the corresponding reduced-order model transfer function. Similarly, we obtain conditions for interpolating the parameter Hessian of the transfer function by including additional vectors in the projection subspaces. As in the parametric linear case, the basis construction for two-sided projections does not require computing the Jacobian or the Hessian.

A set-to-set nearest neighbor approach for robust and efficient face recognition with image sets

Set-to-set face recognition has drawn much attention thanks to its rich set information. We propose a robust and efficient Set-to-Set Nearest Neighbor Classification (S2S-NNC) approach for face recognition by using the maximum weighted correlation between sets in low-dimensional projection subspaces. A pair of face sets is represented as two sets of Mutual Typical Samples (MTS) based on their maximum weighted correlation, and the S2S distance is equivalent to that between two sets of MTS. For the variation of objects within a set, the faces are partitioned into patches and projected onto a correlation subspace to find the MTS between two sets. Furthermore, we develop a S2S-NNC approach for image set-based face recognition. Compared with existing approaches, the S2S-NNC unifies the image-to-image, image-to-set and set-to-set recognition problems into one model. Experimental results show the S2S-NNC approach significantly outperforms the state-of-art approaches on large video samples and small occluded samples.

A global rational Arnoldi method for model reduction

In this paper, we propose two new approaches for model order reduction of large-scale multi-input multi-output (MIMO) linear time invariant dynamical systems (LTI). These methods are based on a generalization of the global Arnoldi algorithm which is used to generate projection subspaces. An adaptive procedure for the selection of shift parameters is proposed and some simple Arnoldi relations are established in order to compute error bounds on the transfer function. Numerical experiments showing the effectiveness of these approaches are provided.

Distributed Subspace Consensus in the Presence of Dynamic In-Network Disturbance

This paper studies distributed strategies for average-consensus of arbitrary vectors in multiagent systems, when the interagent information exchange is corrupted by the agents states within the same network. In particular, each neighboring state received by an agent has an additive component that consists of projections of the states at other agents; the agents corrupting this exchange are unknown to the receiving agent and may also change over time. We model such in-network disturbance with a dynamic disturbance graph over the agents, in addition to the static graph over which consensus is implemented. The problem in its full generality is quite challenging and in an attempt to simplify, we assume two particular disturbance cases: 1) sender based and 2) receiver based . In the former case, we assume that the (null spaces of the) projection subspaces are only known at the senders; while in the latter case, we assume this knowledge only at the receivers. We provide a concrete example of static, flat-fading multiple-input multiple-output channels to support this disturbance model. In the aforementioned context, we cast an algebraic structure over the disturbance subspaces and show that the average is reachable in a subspace whose dimension is complementary to the maximal dimension of the disturbance subspaces. To develop the results, we introduce the notion of information alignment to align the intended message to the null space of the unintended disturbance. We derive the conditions under which this alignment is invertible, that is, the intended message can be recovered. A major contribution of this work is to show that local protocols exist for (subspace) consensus even when the disturbance over the network spans the entire vector space.

On the Incommensurability Phenomenon

Suppose that two large, multi-dimensional data sets are each noisy measurements of the same underlying random process, and principal components analysis is performed separately on the data sets to reduce their dimensionality. In some circumstances it may happen that the two lower-dimensional data sets have an inordinately large Procrustean fitting-error between them. The purpose of this manuscript is to quantify this phenomenon. In particular, under specified conditions, the square Procrustean fitting-error of the two normalized lower-dimensional data sets is (asymptotically) a convex combination (via a correlation parameter) of the Hausdorff distance between the projection subspaces and the maximum possible value of the square Procrustean fitting-error for normalized data. We show how this gives rise to the incommensurability phenomenon, and we employ illustrative simulations and also use real data to explore how the incommensurability phenomenon may have an appreciable impact.

Convergence of the Galyorkin method of approximate solving parabolic equation with weight integral condition

In a Hilbert space an abstract linear parabolic equation with a nonlocal weight integral condition is resolved approximatelymaking use of theGalyorkinmethod. Assumptions on projection subspaces are oriented on a usage of finite element method. We consider the case of projection subspaces built by the uniform partition of domain as well as the case of arbitrary projection subspaces. We obtain the errors estimations for approximate solutions and establish estimates of the convergence rate, exact by the order.

A statistical training data cleaning strategy for the PCA-based chiller sensor fault detection, diagnosis and data reconstruction method

This paper presents a statistical training data cleaning strategy for PCA-based chiller sensor Fault Detection, Diagnosis and Data Reconstruction method. Finding and removing outliers from the original training data set, the training data quality can be improved by the presented data-cleaning strategy. This can enhance the efficiency of the fault detection and increase the accuracy of the data reconstruction. Outliers cannot be easily found in the original data set used for training the PCA model. These outliers would severely affect the projection directions of PCA's two orthogonal subspaces (PC subspace and Residual subspace). Therefore, the threshold of Q-statistic is changed by the unexpected projection subspaces so that the detection efficiency of the sensor fault is decreased. The Euclidean distance was employed as an index to detect outliers from the original training data. In order to achieve optimal training data for the sensor FDDR, the z-scores of each sample's Euclidean Distance were employed as the key to remove the outliers. A field measured data set of a screw water-cooled chiller was used to validate the presented strategy. Results demonstrate that the quality of the training data is optimized and sensor fault detection efficiency, as well as the reconstruction data accuracy, is improved when compared to the normal PCA method.

Compact model order reduction of weakly nonlinear systems by associated transform

SummaryWe advance a recently proposed approach, called the associated transform, for computing slim projection matrices serving high‐order Volterra transfer functions in the context of weakly nonlinear model order reduction (NMOR). The innovation is to carry out an association of multivariate (Laplace) variables in high‐order multiple‐input multiple‐output transfer functions to generate univariate single‐s transfer functions. In contrast to conventional projection‐based NMOR which finds projection subspaces about every si in multivariate transfer functions, only that about a single s is required in the proposed approach. This leads to much more compact reduced‐order models without compromising accuracy. Specifically, the proposed NMOR procedure first converts the original set of Volterra transfer functions into a new set of linear transfer functions, which then allows direct utilization of linear MOR techniques for modeling weakly nonlinear systems with either single‐tone or multi‐tone inputs. An adaptive algorithm is also given to govern the selection of appropriate basis orders in different Volterra transfer functions. Numerical examples then verify the effectiveness of the proposed scheme. Copyright © 2015 John Wiley & Sons, Ltd.

On linear regression models in infinite dimensional spaces with scalar response

In functional linear regression, the parameters estimation involves solving a non necessarily well-posed problem and it has points of contact with a range of methodologies, including statistical smoothing, deconvolution and projection on finite-dimensional subspaces. We discuss the standard approach based explicitly on functional principal components analysis, nevertheless the choice of the number of basis components remains something subjective and not always properly discussed and justified. In this work we discuss inferential properties of least square estimation in this context with different choices of projection subspaces, as well as we study asymptotic behaviour increasing the dimension of subspaces.

H-fold Symmetric quasicrystallography from affine Coxeter groups

One can obtain the quasicrystallographic structures from the projection of the higher dimensional lattices into 2D or 3D subspaces. Here we introduce a general technique applicable to any higher dimensional lattice described by the affine Coxeter groups. It is pointed out that the Coxeter number h and the Coxeter exponents play an important role in determining the principal planes onto which the lattice to be projected. The quasicrystal structures obtained by projection display the dihedral symmetry of order 2h . The projection subspaces are determined by using the eigenvectors and the corresponding eigenvalues of the Cartan matrix. Examples are given for 12-fold symmetric quasicrystal structures obtained by projections of the lattices determined by the affine Coxeter-Weyl groups Wa (F4), Wa (B6), and Wa (E6). The reflection generators R1 and R2 of the dihedral group D12 can be obtained as the products of the generators of the Coxeter-Weyl groups. It is noted that the quasicrystal structures obtained from the lattices Wa (F4) and Wa (B6) are compatible with the experimental data.

Twelvefold symmetric quasicrystallography from the latticesF4,B6andE6

One possible way to obtain the quasicrystallographic structure is the projection of the higher-dimensional lattice into two- or three-dimensional subspaces. Here a general technique applicable to any higher-dimensional lattice is introduced. The Coxeter number and the integers of the Coxeter exponents of a Coxeter–Weyl group play a crucial role in determining the plane onto which the lattice is to be projected. The quasicrystal structures display the dihedral symmetry of order twice that of the Coxeter number. The eigenvectors and the corresponding eigenvalues of the Cartan matrix are used to determine the set of orthonormal vectors inn-dimensional Euclidean space which lead to suitable choices for the projection subspaces. The maximal dihedral subgroup of the Coxeter–Weyl group is identified to determine the symmetry of the quasicrystal structure. Examples are given for 12-fold symmetric quasicrystal structures obtained by projecting the higher-dimensional lattices determined by the affine Coxeter–Weyl groupsWa(F4),Wa(B6) andWa(E6). These groups share the same Coxeter numberh= 12 with different Coxeter exponents. The dihedral subgroupD12of the Coxeter groups can be obtained by defining two generatorsR1andR2as the products of generators of the Coxeter–Weyl groups. The reflection generatorsR1andR2operate in the Coxeter planes where the Coxeter elementR1R2of the Coxeter–Weyl group represents the rotation of order 12. The canonical (strip, equivalently, cut-and-project technique) projections of the lattices determine the nature of the quasicrystallographic structures with 12-fold symmetry as well as the crystallographic structures with fourfold and sixfold symmetry. It is noted that the quasicrystal structures obtained from the latticesWa(F4) andWa(B6) are compatible with some experimental results.