Eigendecomposition Of Matrix Research Articles

Subspace choices for the Celis-Dennis-Tapia problem

Grapiglia et al. (2013) proved subspace properties for the Celis-Dennis-Tapia (CDT) problem. If a subspace with lower dimension is appropriately chosen to satisfy subspace properties, then one can solve the CDT problem in that subspace so that the computational cost can be reduced. We show how to find subspaces that satisfy subspace properties for the CDT problem, by using the eigendecomposition of the Hessian matrix of the objection function. The dimensions of the subspaces are investigated. We also apply the subspace technologies to the trust region subproblem and the quadratic optimization with two quadratic constraints.

Approximate normalized cuts without Eigen-decomposition

Most traditional weighted graph clustering algorithms are solved by spectral method, which is only suitable for small scale datasets because of the high space and time complexity. How to reduce the computational complexity of graph cut clustering to process the massive complex data has become a challenging problem. To overcome this problem, we design an approximate normalized cuts algorithm without eigen-decomposition for large scale clustering. On the one hand, the space requirement of normalized cut is decreased by sampling a few data points to infer the global features of dataset instead of using the whole affinity matrix; on the other hand, the graph cut clustering procedure is accelerated in an iterative way that using the approximate weighted kernel k-means to optimize the objective function of normalized cut, which avoids the direct eigen-decomposition of Laplacian matrix. We also analyze the approximation error of the proposed algorithm and compare it with other state-of-the-arts clustering algorithms on several benchmark datasets. The experimental results demonstrate that our method can efficiently do the clustering when the number of data objects exceeds tens of thousands.

Option Pricing in Some Non-Lévy Jump Models

This paper considers pricing European options in a large class of one-dimensional Markovian jump processes known as subordinate diffusions, which are obtained by time changing a diffusion process with an independent Lévy or additive random clock. These jump processes are non-Lévy in general, and they can be viewed as a natural generalization of many popular Lévy processes used in finance. Subordinate diffusions offer richer jump behavior than Lévy processes and they have found a variety of applications in financial modeling. The pricing problem for these processes presents unique challenges, as existing numerical PIDE schemes fail to be efficient and the applicability of transform methods to many subordinate diffusions is unclear. We develop a novel method based on a finite difference approximation of spatial derivatives and matrix eigendecomposition, and it can deal with diffusions that exhibit various types of boundary behavior. Since financial payoffs are typically not smooth, we apply a smoothing technique and use extrapolation to speed up convergence. We provide convergence and error analysis and perform various numerical experiments to show that the proposed method is fast and accurate. Extension to pricing path-dependent options will be investigated in a followup paper.

Outsourcing Eigen-Decomposition and Singular Value Decomposition of Large Matrix to a Public Cloud

Cloud computing enables customers with limited computational resources to outsource their huge computation workloads to the cloud with massive computational power. However, in order to utilize this computing paradigm, it presents various challenges that need to be addressed, especially security. As eigen-decomposition (ED) and singular value decomposition (SVD) of a matrix are widely applied in engineering tasks, we are motivated to design secure, correct, and efficient protocols for outsourcing the ED and SVD of a matrix to a malicious cloud in this paper. In order to achieve security, we employ efficient privacy-preserving transformations to protect both the input and output privacy. In order to check the correctness of the result returned from the cloud, an efficient verification algorithm is employed. A computational complexity analysis shows that our protocols are highly efficient. We also introduce an outsourcing principle component analysis as an application of our two proposed protocols.

A general solution strategy of modified power method for higher mode solutions

A general solution strategy of the modified power iteration method for calculating higher eigenmodes has been developed and applied in continuous energy Monte Carlo simulation. The new approach adopts four features: 1) the eigen decomposition of transfer matrix, 2) weight cancellation for higher modes, 3) population control with higher mode weights, and 4) stabilization technique of statistical fluctuations using multi-cycle accumulations. The numerical tests of neutron transport eigenvalue problems successfully demonstrate that the new strategy can significantly accelerate the fission source convergence with stable convergence behavior while obtaining multiple higher eigenmodes at the same time. The advantages of the new strategy can be summarized as 1) the replacement of the cumbersome solution step of high order polynomial equations required by Booth's original method with the simple matrix eigen decomposition, 2) faster fission source convergence in inactive cycles, 3) more stable behaviors in both inactive and active cycles, and 4) smaller variances in active cycles. Advantages 3 and 4 can be attributed to the lower sensitivity of the new strategy to statistical fluctuations due to the multi-cycle accumulations. The application of the modified power method to continuous energy Monte Carlo simulation and the higher eigenmodes up to 4th order are reported for the first time in this paper. Modified power method is applied to continuous energy Monte Carlo simulation.Transfer matrix is introduced to generalize the modified power method.All mode based population control is applied to get the higher eigenmodes.Statistic fluctuation can be greatly reduced using accumulated tally results.Fission source convergence is accelerated with higher mode solutions.

Robust beamforming using beam‐to‐reference weighting diagonal loading and Bayesian framework

A novel robust diagonal loading (DL) algorithm against steering vector mismatch is proposed. The beam-to-reference ratio is estimated first and then used as a weighting factor for variable DL. The Bayesian framework is applied eventually to obtain the weight vector. In contrast to traditional methods, the proposed method achieves good performance without using any optimisation toolboxes or matrix eigendecomposition operations.

Self-Tuning p-Spectral Clustering Based on Shared Nearest Neighbors

Cognitive computing needs to handle large amounts of data and information. Spectral clustering is a powerful data mining tool based on algebraic graph theory. Because of the solid theoretical foundation and good clustering performance, spectral clustering has aroused extensive attention of academia in recent years. Spectral clustering transforms the data clustering problem into the graph partitioning problem. Cheeger cut is an optimized graph partitioning criterion. To minimize the objective function of Cheeger cut, the eigen-decomposition of p-Laplacian matrix is required. However, the clustering results are sensitive to the selection of similarity measurement and the parameter p of p-Laplacian matrix. Therefore, we propose a self-tuning p-spectral clustering algorithm based on shared nearest neighbors (SNN-PSC). This algorithm uses shared nearest neighbors to measure the similarities of data couples and then applies fruit fly optimization algorithm to find the optimal parameters p of p-Laplacian matrix that leads to better data classification. Experiments show that SNN-PSC algorithm can produce more balanced clusters and has strong adaptability and robustness compared to traditional spectral clustering algorithms.

An Improved ESPRIT Algorithm for Fast Frequency Estimation

An improved fast ESPRIT frequency estimation algorithm is present for reducing the computational load existing in frequency estimation of subspace rotational in variance technology (ESPRIT) algorithm. The improved algorithm does not need eigen-decomposition of auto-covariance matrix, using the sampling signal delay data to construct two sub-arrays with the same array shape, through generalized eigen-decomposition of the matrix pencil constructed by the two sub-arrays cross-covariance matrix, to achieve a fast frequency estimation of signal. The simulation results show that the frequency estimated performance of the improved ESPRIT algorithm is comparable to the standard ESPRIT algorithm, and its computational load can be reduced to the fifteen percent of the standard ESPRIT algorithm, so can be used in real time processing system.

An Efficient and Effective Multiple Empirical Kernel Learning Based on Random Projection

Multiple empirical kernel learning (MEKL) is demonstrated to be flexible and effective due to introducing multiple kernels. But MEKL also brings a large computational complexity in practice. Therefore, in this paper we adopt the random projection (RP) technique to efficiently construct the low-dimensional feature space, and then develop an efficient and effective MEKL named MEKLRP so as to decrease the computational complexity. The proposed MEKLRP randomly selects a subset $$S'$$S? of $$p$$p samples from the whole training set $$S$$S of $$N$$N samples, and then utilizes $$S'$$S? to generate $$M$$M different EKMs $$\{\Phi ^{rpe}_l(x)\}_{l=1}^M$${?lrpe(x)}l=1M. Following that, MEKLRP maps each sample $$x$$x into $$\Phi _l^{rpe}(x), l=1...M$$?lrpe(x),l=1...M. Finally, MEKLRP applies the transformed samples into our previous MEKL framework. We highlight the contributions of the MEKLRP as follows. Firstly, the MEKLRP adopts the random characteristic of RP and efficiently decreases the computational cost of the matrix eigen-decomposition from $$O(N^3)$$O(N3) to $$O(p^3)$$O(p3). Secondly, the MEKLRP maintains an approximate separability at one certain margin and preserves most of the discriminant information in a low-dimensional space since the characteristic of RP in kernel-based learning. Thirdly, the MEKLRP behaves a lower generalization risk bound than its corresponding original learning machine according to the Rademacher complexity.

Consistent feature selection and its application to face recognition

In this paper we consider feature selection for face recognition using both labeled and unlabeled data. We introduce the weighted feature space in which the global separability between different classes is maximized and the local similarity of the neighboring data points is preserved. By integrating the global and local structures, a general optimization framework is formulated. We propose a simple solution to this problem, avoiding the matrix eigen-decomposition procedure which is often computationally expensive. Experimental results demonstrate the efficacy of our approach and confirm that utilizing labeled and unlabeled data together does help feature selection with small number of labeled samples.

An adaptive robust fuzzy beamformer for steering vector mismatch and reducing interference and noise

This paper presents an efficient beamformer design that accounts for both signal steering vector mismatches and the trade-off between interference and noise reduction. The empirical results show that substantial performance degradation occurs when the exact signal steering vector for the desired signal is not known. Furthermore, total rejection of interference may increase noise, and vice versa. Therefore, an adaptive robust fuzzy beamformer was designed by adopting the method of fuzzy systems to modify both the value of the signal steering vector and the interference-to-noise ratio. This method uses the first-order Taylor expansion of the object function to modify the mismatches of the signal steering vector, and uses the signal covariance matrix eigendecomposition to adjust the ratio of interference reduction to noise reduction. Simulations confirm that the proposed scheme performance is substantially improved and more robust if the effects of the signal steering vector mismatches and the interference to noise ratio are considered in the beamformer design which is based on expert knowledge.

基于Hessian矩阵本征分解的目标中心线提取算法

基于Hessian矩阵本征分解的目标中心线提取算法

Random matrix theory analysis of the dominant mode rejection beamformer

The Dominant Mode Rejection (DMR) beamformer developed by Abraham and Owsley [Proc. Oceans, 1990] determines the beamformer weights from the sensor covariance matrix eigendecomposition. The weights are designed to reject signals contained in the dominant subspace, which is defined by the eigenvectors associated with the largest eigenvalues. In previous work, we developed a model for the mean notch depth (ND) of the DMR beamformer from random matrix theory (RMT) results on the sample eigenvector fidelity [IEEE Stat. Sig. Proc. workshop, 2012]. While ND is useful, other metrics such as white noise gain (WNG) and signal to interference and noise ratio (SINR) are of great interest. WNG characterizes the beamformer robustness to mismatch, and SINR quantifies overall performance. SINR loss is defined as the ratio of the SINR for a beamformer designed using sample statistics to the SINR for the optimal beamformer designed using ensemble statistics. This talk extends our previous work by considering the relationship among ND, WNG, and SINR for the DMR beamformer. A surprising result obtained from RMT is that for a single loud interferer and twice as many snapshots as sensors, the expected SINR loss depends only on the number of snapshots. [Work supported by ONR.]

Synthetic aperture radar autofocus based on projection approximation subspace tracking

An eigenvector method for maximum-likelihood estimation (MLE) of phase error has better algorithmic performance than phase gradient autofocus (PGA), which is implemented by the simultaneous processing of multiple-pulse vectors of the range-compressed data. However, this method requires eigendecomposition of the sample covariance matrix, which is a computationally expensive task and also limits the real-time application. In order to overcome such difficulty, this study proposes a novel autofocus algorithm using the projection approximation subspace tracking (PAST) approach. With this methodology, the computational cost can be reduced effectively to the level of PGA via avoiding the procedures of covariance matrix estimation and eigendecomposition. Monte Carlo tests and real synthetic aperture radar (SAR) data validate that although undergoing performance loss compare with the original multiple-pulse MLE algorithm, the new approach outperforms the mostly used PGA.

Multiple Sample Data Spectroscopic Clustering of Large Datasets Using Nyström Extension

In this article, we focus on computational aspects of spectral clustering algorithms that have recently shown promising results in machine learning, statistics, and computer vision. These algorithms cluster observations (of size n) into groups by investigating eigenvectors of an affinity matrix or its Laplacian matrix, both of which are size n × n. However, when the sample size is large, the computation involved in the matrix eigen-decomposition is expensive or even infeasible. To overcome the computation hurdle, subsampling techniques, such as the Nyström extension, have been used to approximate eigenvectors of large matrices. We study statistical properties of this approximation and their influence on the accuracy of various spectral clustering algorithms. We show that the perturbation of the spectrum due to subsampling could lead to a large discrepancy among clustering results. In order to provide accurate and stable results for large datasets, we propose a method to combine multiple subsamples using data spectroscopic clustering and the Nyström extension. In addition, we propose a sparse approximation of the eigenvectors to further speed up computation. Simulation and experiments on real datasets show that our approaches work quickly and provide reasonable results that are more stable across samples than the single sample approach. This article has supplementary material online.

PERFORMANCE ANALYSIS OF POLARIZATION-SPACE-TIME THREE-DOMAIN JOINT PROCESSING FOR CLUTTER SUPPRESSION IN AIRBORNE RADAR

An optimum polarization-space-time joint domain pro- cessing (PST-JDP) technique is proposed for clutter suppression which adequately adopts the three-domain information including the polar- ization, space and Doppler frequency information of the radar echo. The study shows that the polarization information together with the space and Doppler frequency information are efiective to signiflcantly enhance the clutter suppression performance for airborne radar. Sev- eral new techniques, (i.e., the covariance matrix eigendecomposition, the spectral analysis and the resolution grid method), are utilized for deriving the performance of the optimum PST-JDP. The main factors which afiect on the performance of clutter rejection are the clutter degree of polarization, statistical distance of polarization between tar- get and clutter, Doppler frequency of target and input clutter-to-noise ratio. The new optimum PST-JDP method outperforms signiflcantly the traditional optimum space-time processing technology, especially in the case of the slowly or tangentially moving target. The simulation verifles the correctness and e-ciency of the model.

Partitioning networks into clusters and residuals with average association

We investigate the problem of detecting clusters exhibiting higher-than-average internal connectivity in networks of interacting systems. We show how the average association objective formulated in the context of spectral graph clustering leads naturally to a clustering strategy where each system is assigned to at most one cluster. A residual set is formed of the systems that are not members of any cluster. Maximization of the average association objective leads to a discrete optimization problem, which is difficult to solve, but a relaxed version can be solved using an eigendecomposition of the connectivity matrix. A simple approach to extracting clusters from a relaxed solution is described and developed by applying a variance maximizing solution to the relaxed solution, which leads to a method with increased accuracy and sensitivity. Numerical studies of theoretical connectivity models and of synchronization clusters in a lattice of coupled Lorenz oscillators are conducted to show the efficiency of the proposed approach. The method is applied to an experimentally obtained human resting state functional magnetic resonance imaging dataset and the results are discussed.

Speed up kernel discriminant analysis

Linear discriminant analysis (LDA) has been a popular method for dimensionality reduction, which preserves class separability. The projection vectors are commonly obtained by maximizing the between-class covariance and simultaneously minimizing the within-class covariance. LDA can be performed either in the original input space or in the reproducing kernel Hilbert space (RKHS) into which data points are mapped, which leads to kernel discriminant analysis (KDA). When the data are highly nonlinear distributed, KDA can achieve better performance than LDA. However, computing the projective functions in KDA involves eigen-decomposition of kernel matrix, which is very expensive when a large number of training samples exist. In this paper, we present a new algorithm for kernel discriminant analysis, called Spectral Regression Kernel Discriminant Analysis (SRKDA). By using spectral graph analysis, SRKDA casts discriminant analysis into a regression framework, which facilitates both efficient computation and the use of regularization techniques. Specifically, SRKDA only needs to solve a set of regularized regression problems, and there is no eigenvector computation involved, which is a huge save of computational cost. The new formulation makes it very easy to develop incremental version of the algorithm, which can fully utilize the computational results of the existing training samples. Moreover, it is easy to produce sparse projections (Sparse KDA) with a L 1-norm regularizer. Extensive experiments on spoken letter, handwritten digit image and face image data demonstrate the effectiveness and efficiency of the proposed algorithm.

Exploiting semantic proximities for content search over p2p networks

In this paper, we address the issue of content search over peer-to peer networks. We use the concept of semantic proximity that exploits the commonalities of interests exhibited among peer users so as to decompose the network into semantic clusters. We initially define search entropy, as a metric indicating the average number of packets required to locate the requested content. Then, spectral clustering is used to organize the peer nodes into semantic clusters so that (a) the probability that a node locates content within its own cluster is maximized, while simultaneously; (b) the respective probability of finding this content outside this cluster is minimized. The proposed semantic partitioning algorithm is then extended into a hierarchical two-tier scheme, in which practical issues arising for the deployment of a peer-to-peer (p2p) application can be more easily addressed. After the system has been initialized, a dynamic algorithm places new users that join the p2p network into appropriately selected clusters and also handles peer departures without the need for matrix eigen decomposition process which is necessary for the assessment of the initial static partitioning. Our experimental results validate that (a) our static partitioning outperforms traditional and novel search techniques and (b) our dynamic algorithm is able to efficiently track the system’s progression maintaining the search entropy close to the initially assessed levels.

Pathological rate matrices: from primates to pathogens

BackgroundContinuous-time Markov models allow flexible, parametrically succinct descriptions of sequence divergence. Non-reversible forms of these models are more biologically realistic but are challenging to develop. The instantaneous rate matrices defined for these models are typically transformed into substitution probability matrices using a matrix exponentiation algorithm that employs eigendecomposition, but this algorithm has characteristic vulnerabilities that lead to significant errors when a rate matrix possesses certain 'pathological' properties. Here we tested whether pathological rate matrices exist in nature, and consider the suitability of different algorithms to their computation.ResultsWe used concatenated protein coding gene alignments from microbial genomes, primate genomes and independent intron alignments from primate genomes. The Taylor series expansion and eigendecomposition matrix exponentiation algorithms were compared to the less widely employed, but more robust, Padé with scaling and squaring algorithm for nucleotide, dinucleotide, codon and trinucleotide rate matrices. Pathological dinucleotide and trinucleotide matrices were evident in the microbial data set, affecting the eigendecomposition and Taylor algorithms respectively. Even using a conservative estimate of matrix error (occurrence of an invalid probability), both Taylor and eigendecomposition algorithms exhibited substantial error rates: ~100% of all exonic trinucleotide matrices were pathological to the Taylor algorithm while ~10% of codon positions 1 and 2 dinucleotide matrices and intronic trinucleotide matrices, and ~30% of codon matrices were pathological to eigendecomposition. The majority of Taylor algorithm errors derived from occurrence of multiple unobserved states. A small number of negative probabilities were detected from the Padé algorithm on trinucleotide matrices that were attributable to machine precision. Although the Padé algorithm does not facilitate caching of intermediate results, it was up to 3× faster than eigendecomposition on the same matrices.ConclusionDevelopment of robust software for computing non-reversible dinucleotide, codon and higher evolutionary models requires implementation of the Padé with scaling and squaring algorithm.