Sparse Singular Value Decomposition Research Articles

Fast hierarchical low-rank view factor matrices for thermal irradiance on planetary surfaces

We present an algorithm for compressing the radiosity view factor model commonly used in radiation heat transfer and computer graphics. We use a format inspired by the hierarchical off-diagonal low rank format, where elements are recursively partitioned using a quadtree or octree and blocks are compressed using a sparse singular value decomposition—the hierarchical matrix is assembled using dynamic programming. The motivating application is time-dependent thermal modeling on vast planetary surfaces, with a focus on permanently shadowed craters which receive energy through indirect irradiance. In this setting, shape models are comprised of a large number of triangular facets which conform to a rough surface. At each time step, a quadratic number of triangle-to-triangle scattered fluxes must be summed; that is, as the sun moves through the sky, we must solve the same view factor system of equations for a potentially unlimited number of time-varying righthand sides. We first conduct numerical experiments with a synthetic spherical cap-shaped crater, where the equilibrium temperature is analytically available. We also test our implementation with triangle meshes of planetary surfaces derived from digital elevation models recovered by orbiting spacecraft. Our results indicate that the compressed view factor matrix can be assembled in quadratic time, which is comparable to the time it takes to assemble the full view matrix itself. Memory requirements during assembly are reduced by a large factor. Finally, for a range of compression tolerances, the size of the compressed view factor matrix and the speed of the resulting matrix vector product both scale linearly (as opposed to quadratically for the full matrix), resulting in orders of magnitude savings in processing time and memory space.

Robust integrative biclustering for multi-view data.

In many biomedical research, multiple views of data (e.g. genomics, proteomics) are available, and a particular interest might be the detection of sample subgroups characterized by specific groups of variables. Biclustering methods are well-suited for this problem as they assume that specific groups of variables might be relevant only to specific groups of samples. Many biclustering methods exist for detecting row-column clusters in a view but few methods exist for data from multiple views. The few existing algorithms are heavily dependent on regularization parameters for getting row-column clusters, and they impose unnecessary burden on users thus limiting their use in practice. We extend an existing biclustering method based on sparse singular value decomposition for single-view data to data from multiple views. Our method, integrative sparse singular value decomposition (iSSVD), incorporates stability selection to control Type I error rates, estimates the probability of samples and variables to belong to a bicluster, finds stable biclusters, and results in interpretable row-column associations. Simulations and real data analyses show that integrative sparse singular value decomposition outperforms several other single- and multi-view biclustering methods and is able to detect meaningful biclusters. iSSVD is a user-friendly, computationally efficient algorithm that will be useful in many disease subtyping applications.

Weak Signal Detection Based on Combination of Sparse Representation and Singular Value Decomposition

Due to the inevitable acquisition system noise and strong background noise, it is often difficult to detect the features of weak signals. To solve this problem, sparse representation can effectively extract useful information according to the sparse characteristics of signals. However, sparse representation is less effective against non-Gaussian white noise. Therefore, a novel SRSVD method combining sparse representation and singular value decomposition is proposed to further improve the denoising performance of the algorithm. All the signal components highly matched with the dictionary are extracted by sparse representation, and then each component of singular value decomposition is weighted by the evaluation index, PMI, which can indicate the component with useful information in the signal, so that the signal denoising performance of the algorithm is greatly improved. The performance of the proposed method is verified by the processing of weak signals in the circuit and early fault signals of bearings. The results show that SRSVD can successfully suppress noise interference. Compared with other existing methods, SRSVD has better denoising performance.

Negative binomial factor regression with application to microbiome data analysis.

The human microbiome provides essential physiological functions and helps maintain host homeostasis via the formation of intricate ecological host‐microbiome relationships. While it is well established that the lifestyle of the host, dietary preferences, demographic background, and health status can influence microbial community composition and dynamics, robust generalizable associations between specific host‐associated factors and specific microbial taxa have remained largely elusive. Here, we propose factor regression models that allow the estimation of structured parsimonious associations between host‐related features and amplicon‐derived microbial taxa. To account for the overdispersed nature of the amplicon sequencing count data, we propose negative binomial reduced rank regression (NB‐RRR) and negative binomial co‐sparse factor regression (NB‐FAR). While NB‐RRR encodes the underlying dependency among the microbial abundances as outcomes and the host‐associated features as predictors through a rank‐constrained coefficient matrix, NB‐FAR uses a sparse singular value decomposition of the coefficient matrix. The latter approach avoids the notoriously difficult joint parameter estimation by extracting sparse unit‐rank components of the coefficient matrix sequentially, effectively delivering interpretable bi‐clusters of taxa and host‐associated factors. To solve the nonconvex optimization problems associated with these factor regression models, we present a novel iterative block‐wise majorization procedure. Extensive simulation studies and an application to the microbial abundance data from the American Gut Project (AGP) demonstrate the efficacy of the proposed procedure. In the AGP data, we identify several factors that strongly link dietary habits and host life style to specific microbial families.

Object-Based Classification Framework of Remote Sensing Images With Graph Convolutional Networks

Object-based image classification (OBIC) on very-high-resolution (VHR) remote sensing (RS) images is utilized in a wide range of applications. Nowadays, many existing OBIC methods only focus on features of each object itself, neglecting the contextual information among adjacent objects and resulting in low classification accuracy. Inspired by a spectral graph theory, we construct a graph structure from objects generated from VHR RS images and propose an OBIC framework based on truncated sparse singular value decomposition and graph convolutional network (GCN) model, aiming to make full use of relativities among objects and produce an accurate classification. Through conducting experiments on two annotated RS image data sets, our framework obtained 97.2% and 66.9% overall accuracy, respectively, in automatic and manual object segmentation circumstances, within a processing time of about 1/100 of convolutional neural network (CNN)-based methods’ training time.

Network-based cancer genomic data integration for pattern discovery

BackgroundSince genes involved in the same biological modules usually present correlated expression profiles, lots of computational methods have been proposed to identify gene functional modules based on the expression profiles data. Recently, Sparse Singular Value Decomposition (SSVD) method has been proposed to bicluster gene expression data to identify gene modules. However, this model can only handle the gene expression data where no gene interaction information is integrated. Ignoring the prior gene interaction information may produce the identified gene modules hard to be biologically interpreted.ResultsIn this paper, we develop a Sparse Network-regularized SVD (SNSVD) method that integrates a prior gene interaction network from a protein protein interaction network and gene expression data to identify underlying gene functional modules. The results on a set of simulated data show that SNSVD is more effective than the traditional SVD-based methods. The further experiment results on real cancer genomic data show that most co-expressed modules are not only significantly enriched on GO/KEGG pathways, but also correspond to dense sub-networks in the prior gene interaction network. Besides, we also use our method to identify ten differentially co-expressed miRNA-gene modules by integrating matched miRNA and mRNA expression data of breast cancer from The Cancer Genome Atlas (TCGA). Several important breast cancer related miRNA-gene modules are discovered.ConclusionsAll the results demonstrate that SNSVD can overcome the drawbacks of SSVD and capture more biologically relevant functional modules by incorporating a prior gene interaction network. These identified functional modules may provide a new perspective to understand the diagnostics, occurrence and progression of cancer.

Integrative and sparse singular value decomposition method for biclustering analysis in multi-sources dataset

Multi-sources dataset analysis has been researched for a few years and it aims at improving the potential performance for discoveries by combining data from different sources. Meanwhile, biclustering analysis, widely applied for biology, chemistry, and so on, is a powerful tool that allows the clustering of rows and columns simultaneously. This paper proposes an integrative and sparse singular value decomposition (ISSVD) method for biclustering analysis in multi-sources datasets, which seeks the common left and individual right singular vectors, and finds the interpretable row-column associations. Simulation studies demonstrate that our model can handle ​with the different scales between the multi-source data, and obtain better performance than other popular biclustering methods. Two real-world data examples are also presented.

A Novel Sparse Graph-Regularized Singular Value Decomposition Model and Its Application to Genomic Data Analysis.

Learning the gene coexpression pattern is a central challenge for high-dimensional gene expression analysis. Recently, sparse singular value decomposition (SVD) has been used to achieve this goal. However, this model ignores the structural information between variables (e.g., a gene network). The typical graph-regularized penalty can be used to incorporate such prior graph information to achieve more accurate discovery and better interpretability. However, the existing approach fails to consider the opposite effect of variables with negative correlations. In this article, we propose a novel sparse graph-regularized SVD model with absolute operator (AGSVD) for high-dimensional gene expression pattern discovery. The key of AGSVD is to impose a novel graph-regularized penalty ( | u|T L| u| ). However, such a penalty is a nonconvex and nonsmooth function, so it brings new challenges to model solving. We show that the nonconvex problem can be efficiently handled in a convex fashion by adopting an alternating optimization strategy. The simulation results on synthetic data show that our method is more effective than the existing SVD-based ones. In addition, the results on several real gene expression data sets show that the proposed methods can discover more biologically interpretable expression patterns by incorporating the prior gene network.

Automatic sparse principal component analysis

The wide availability of computers enables us to accumulate a huge amount of data, thus effective tools to extract information from the huge volume of data have become critical. Principal component analysis (PCA) is a useful and traditional tool for dimensionality reduction of massive high‐dimensional datasets. Recently, sparse principal component (PC) loading estimation based on L1‐type regularization has drawn a large amount of attention. Although sparse PCA makes interpretation easily and performs dimension reduction without disturbance from noisy features, the existing studies on sparse PCA were based on an arbitrary number of PCs without any statistical justification. We propose a novel method, called as automatic sparse PCA, which can perform PC selection and sparse PC loading estimation, simultaneously. For PC selection, we first develop sparse singular value decomposition (sparse SVD), then incorporate sparsity into PC loading estimation. The proposed method enables us to perform dimension reduction and PC loading estimation, simultaneously. Furthermore, we can perform PCA without disturbance from noisy features. It can be seen through Monte Carlo experiments that the proposed automatic sparse PCA outperforms sparse structure identification and reconstructing data based on low‐dimensional projection. The proposed method is also applied to a number of real datasets and it can be also seen that our method achieves effectiveness for estimation accuracy and interpreting PCA results.

Generalized co-sparse factor regression

Multivariate regression techniques are commonly applied to explore the associations between large numbers of outcomes and predictors. In real-world applications, the outcomes are often of mixed types, including continuous measurements, binary indicators, and counts, and the observations may also be incomplete. Building upon the recent advances in mixed-outcome modeling and sparse matrix factorization, generalized co-sparse factor regression (GOFAR) is proposed, which utilizes the flexible vector generalized linear model framework and encodes the outcome dependency through a sparse singular value decomposition (SSVD) of the integrated natural parameter matrix. To avoid the estimation of the notoriously difficult joint SSVD, GOFAR proposes both sequential and parallel unit-rank estimation procedures. By combining the ideas of alternating convex search and majorization–minimization, an efficient algorithm is developed to solve the sparse unit-rank problem and implemented in the R package gofar. Extensive simulation studies and two real-world applications demonstrate the effectiveness of the proposed approach.

ASSVD: Adaptive Sparse Singular Value Decomposition for High Dimensional Matrices

In this paper, an adaptive sparse singular value decomposition (ASSVD) algorithm is proposed to estimate the signal matrix when only one data matrix is observed and there is high dimensional white noise, in which we assume that the signal matrix is low-rank and has sparse singular vectors, i.e. it is a simultaneously low-rank and sparse matrix. It is a structured matrix since the non-zero entries are confined on some small blocks. The proposed algorithm estimates the singular values and vectors separable by exploring the structure of singular vectors, in which the recent developments in Random Matrix Theory known as anisotropic Marchenko-Pastur law are used. And then we prove that when the signal is strong in the sense that the signal to noise ratio is above some threshold, our estimator is consistent and outperforms over many state-of-the-art algorithms. Moreover, our estimator is adaptive to the data set and does not require the variance of the noise to be known or estimated. Numerical simulations indicate that ASSVD still works well when the signal matrix is not very sparse.

Doppler Distortion Removal in Wayside Circular Microphone Array Signals

Condition monitoring and fault diagnosis (CMFD) of key moving sources has attracted more and more attentions in recent years. Among many CMFD strategies, the wayside acoustic signal-based method can detect slight failure efficiently in time with a low-cost system. However, the nonlinear time-varying distance between the moving source and the microphone array would bring in serious Doppler distortion in the acquired signals. The Doppler distortion would result in serious amplitude distortion, frequency shift, and band extension, which is an obstacle for accurate CMFD. Considering the superior time-spatial characteristics of circular microphone array, this paper presents a short-time sparse singular value decomposition (SSVD) method based on wayside circular microphone array signals to eliminate the existed Doppler distortion. The procedure of the presented method is summarized as follows. First, the Doppler distortion array signals are decomposed into a series of array segments with a sliding window. Afterward, the corresponding moving angles (MAs) of each array segments are calculated with the introduced SSVD method. Finally, the fitting time-varying MA curve and time-domain interpolation resampling are employed to eliminate the Doppler distortion. Simulation results validated that the proposed method could achieve better localization accuracy under heavy background noise as compared to other traditional methods. Moreover, the effectiveness of the proposed method has been further verified with real experiments, which indicates excellent potential application prospects for moving vehicles, trains, planes, and so on.

Doppler distortion elimination using short-time sparse singular value decomposition strategy for wayside acoustic source fault diagnosis

With the rapid development of transportation, wayside condition monitoring and fault diagnosis of key acoustic sources has attracted considerable attentions because of low cost and high efficiency. However, serious Doppler distortion exists in the wayside acquired signals when the monitored moving source passes by the system at high velocity, which makes it difficult for condition monitoring and fault diagnosis. This paper presents a novel method involving short-time sparse singular value decomposition to eliminate Doppler distortion in the wayside acquired signals based on a microphone array. The procedure of the proposed short-time sparse singular value decomposition is performed as follows. First, the Doppler distorted array signals are decomposed into a series of array segments by a sliding window with a proper window length. Afterwards, the time-varying direction of arrival of the corresponding array segments is acquired by individual sparse singular value decomposition. Then the fitting time-varying directions of arrival and time-domain interpolation resampling are employed to correct the Doppler distorted signals for recovering the objective characteristic frequency. Simulation analysis has validated that under heavy background noise situation, better localization accuracy and effectiveness of the proposed short-time sparse singular value decomposition could be achieved in comparison with other strategies like short-time multiple signal classification. Besides, the real data cases have verified the effectiveness of the proposed method, which shows great potential applications in wayside condition monitoring and fault diagnosis system for moving vehicles, trains, planes, etc.

SOFAR: Large-Scale Association Network Learning.

Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsity and orthogonality have been two largely incompatible goals. To accommodate both features, in this paper we suggest the method of sparse orthogonal factor regression (SOFAR) via the sparse singular value decomposition with orthogonality constrained optimization to learn the underlying association networks, with broad applications to both unsupervised and supervised learning tasks such as biclustering with sparse singular value decomposition, sparse principal component analysis, sparse factor analysis, and spare vector autoregression analysis. Exploiting the framework of convexity-assisted nonconvex optimization, we derive nonasymptotic error bounds for the suggested procedure characterizing the theoretical advantages. The statistical guarantees are powered by an efficient SOFAR algorithm with convergence property. Both computational and theoretical advantages of our procedure are demonstrated with several simulations and real data examples.

Group-sparse SVD Models via $L_1$- and $L_0$-norm Penalties and Their Applications in Biological Data

Sparse Singular Value Decomposition (SVD) models have been proposed for biclustering high dimensional gene expression data to identify block patterns with similar expressions. However, these models do not take into account prior group effects upon variable selection. To this end, we first propose group-sparse SVD models with group Lasso ( $GL_1$ G L 1 -SVD) and group $L_0$ L 0 -norm penalty ( $GL_0$ G L 0 -SVD) for non-overlapping group structure of variables. However, such group-sparse SVD models limit their applicability in some problems with overlapping structure. Thus, we also propose two group-sparse SVD models with overlapping group Lasso ( $OGL_1$ O G L 1 -SVD) and overlapping group $L_0$ L 0 -norm penalty ( $OGL_0$ O G L 0 -SVD). We first adopt an alternating iterative strategy to solve $GL_1$ G L 1 -SVD based on a block coordinate descent method, and $GL_0$ G L 0 -SVD based on a projection method. The key of solving $OGL_1$ O G L 1 -SVD is a proximal operator with overlapping group Lasso penalty. We employ an alternating direction method of multipliers (ADMM) to solve the proximal operator. Similarly, we develop an approximate method to solve $OGL_0$ O G L 0 -SVD. Applications of these methods and comparison with competing ones using simulated data demonstrate their effectiveness. Extensive applications of them onto several real gene expression data with gene prior group knowledge identify some biologically interpretable gene modules.

Integration of sparse singular vector decomposition and statistical process control for traffic monitoring and quality of service improvement in mission-critical communication networks

Mission-Critical Communication Networks (MCCNs) are wireless networks whose malfunction can cause significant problems. The nature of MCCNs puts an extremely high standard on the Quality of Service (QoS). QoS assurance starts from monitoring and change/anomaly detection of network packets data. This problem has been primarily studied by the research community of communication networks, in which the existing methods fall short for providing a privacy-preserving, minimum-disruption, global monitoring tool. Another relevant research area is Multivariate Statistical Process Control (MSPC), in which generic methods have been developed for monitoring high-dimensional data streams. These methods do not account for the special data distribution and correlation structure of packet streams. Nor are they efficient enough to suit real-time analytics in MCCNs. We propose a method called Sparse Singular Value Decomposition (SSVD)-MSPC. SSVD-MSPC addresses the aforementioned limitations and additionally provides key capabilities toward QoS improvement, including monitoring, fault identification, and fault characterization. Extensive case studies are conducted for MCCNs that experience faults of different magnitudes and various temporal shapes. SSVD-MSPC achieves good performance across the different settings in comparison with existing methods.

Sequential Co-Sparse Factor Regression

ABSTRACTIn multivariate regression models, a sparse singular value decomposition of the regression component matrix is appealing for reducing dimensionality and facilitating interpretation. However, the recovery of such a decomposition remains very challenging, largely due to the simultaneous presence of orthogonality constraints and co-sparsity regularization. By delving into the underlying statistical data-generation mechanism, we reformulate the problem as a supervised co-sparse factor analysis, and develop an efficient computational procedure, named sequential factor extraction via co-sparse unit-rank estimation (SeCURE), that completely bypasses the orthogonality requirements. At each step, the problem reduces to a sparse multivariate regression with a unit-rank constraint. Nicely, each sequentially extracted sparse and unit-rank coefficient matrix automatically leads to co-sparsity in its pair of singular vectors. Each latent factor is thus a sparse linear combination of the predictors and may influence only a subset of responses. The proposed algorithm is guaranteed to converge, and it ensures efficient computation even with incomplete data and/or when enforcing exact orthogonality is desired. Our estimators enjoy the oracle properties asymptotically; a non-asymptotic error bound further reveals some interesting finite-sample behaviors of the estimators. The efficacy of SeCURE is demonstrated by simulation studies and two applications in genetics. Supplementary materials for this article are available online.

A Novel Sparse Penalty for Singular Value Decomposition

Singular value decomposition (SVD) is a tool widely used in data denoising, matrix approximation, recommendation system, text mining and computer vision. A majority of applications prefer sparse singular vectors to capture inherent structures and patterns of the input data so that the results are interpretable. We present a novel penalty for SVD to achieve sparsity. Comparing with the traditional penalties, the proposed penalty is scale, dimensional insensitive and bounded between 0 and 1, which are in favor of controlling sparsity. Regulated by the penalty, we provide an efficient algorithm to project a vector onto a given sparse level in O(n) expected time. The efficient projection algorithm serve as a drudge for sparse SVD (SSVD). In experiments, SSVD is efficient and could capture the latent structures and patterns of the input data.

Neural Substrates Related to Motor Memory with Multiple Timescales in Sensorimotor Adaptation.

Recent computational and behavioral studies suggest that motor adaptation results from the update of multiple memories with different timescales. Here, we designed a model-based functional magnetic resonance imaging (fMRI) experiment in which subjects adapted to two opposing visuomotor rotations. A computational model of motor adaptation with multiple memories was fitted to the behavioral data to generate time-varying regressors of brain activity. We identified regional specificity to timescales: in particular, the activity in the inferior parietal region and in the anterior-medial cerebellum was associated with memories for intermediate and long timescales, respectively. A sparse singular value decomposition analysis of variability in specificities to timescales over the brain identified four components, two fast, one middle, and one slow, each associated with different brain networks. Finally, a multivariate decoding analysis showed that activity patterns in the anterior-medial cerebellum progressively represented the two rotations. Our results support the existence of brain regions associated with multiple timescales in adaptation and a role of the cerebellum in storing multiple internal models.

Discovering Recurrent Copy Number Aberrations in Complex Patterns via Non-Negative Sparse Singular Value Decomposition.

Recurrent copy number aberrations (RCNAs) in multiple cancer samples are strongly associated with tumorigenesis, and RCNA discovery is helpful to cancer research and treatment. Despite the emergence of numerous RCNA discovering methods, most of them are unable to detect RCNAs in complex patterns that are influenced by complicating factors including aberration in partial samples, co-existing of gains and losses and normal-like tumor samples. Here, we propose a novel computational method, called non-negative sparse singular value decomposition (NN-SSVD), to address the RCNA discovering problem in complex patterns. In NN-SSVD, the measurement of RCNA is based on the aberration frequency in a part of samples rather than all samples, which can circumvent the complexity of different RCNA patterns. We evaluate NN-SSVD on synthetic dataset by comparison on detection scores and Receiver Operating Characteristics curves, and the results show that NN-SSVD outperforms existing methods in RCNA discovery and demonstrate more robustness to RCNA complicating factors. Applying our approach on a breast cancer dataset, we successfully identify a number of genomic regions that are strongly correlated with previous studies, which harbor a bunch of known breast cancer associated genes.