Rank Reduction Research Articles

Generalized Additive Models

Generalized additive models are generalized linear models in which the linear predictor includes a sum of smooth functions of covariates, where the shape of the functions is to be estimated. They have also been generalized beyond the original generalized linear model setting to distributions outside the exponential family and to situations in which multiple parameters of the response distribution may depend on sums of smooth functions of covariates. The widely used computational and inferential framework in which the smooth terms are represented as latent Gaussian processes, splines, or Gaussian random effects is reviewed, paying particular attention to the case in which computational and theoretical tractability is obtained by prior rank reduction of the model terms. An empirical Bayes approach is taken, and its relatively good frequentist performance discussed, along with some more overtly frequentist approaches to model selection. Estimation of the degree of smoothness of component functions via cross validation or marginal likelihood is covered, alongside the computational strategies required in practice, including when data and models are reasonably large. It is briefly shown how the framework extends easily to location-scale modeling, and, with more effort, to techniques such as quantile regression. Also covered are the main classes of smooths of multiple covariates that may be included in models: isotropic splines and tensor product smooth interaction terms.

Multi-Source Information Graph Embedding with Ensemble Learning for Link Prediction

Link prediction is a key technique for connecting entities and relationships in a graph reasoning field. It leverages known information about the graph structure data to predict missing factual information. Previous studies have either focused on the semantic representation of a single triplet or on the graph structure data built on triples. The former ignores the association between different triples, and the latter ignores the true meaning of the node itself. Furthermore, common graph-structured datasets inherently face challenges, such as missing information and incompleteness. In light of this challenge, we present a novel model called Multi-source Information Graph Embedding with Ensemble Learning for Link Prediction (EMGE), which can effectively improve the reasoning of link prediction. Ensemble learning is systematically applied throughout the model training process. At the data level, this approach enhances entity embeddings by integrating structured graph information and unstructured textual data as multi-source information inputs. The fusion of these inputs is effectively addressed by introducing an attention mechanism. During the training phase, the principle of ensemble learning is employed to extract semantic features from multiple neural network models, facilitating the interaction of enriched information. To ensure effective model learning, a novel loss function based on contrastive learning is devised, effectively minimizing the discrepancy between predicted values and the ground truth. Moreover, to enhance the semantic representation of graph nodes in link prediction, two rules are introduced during the aggregation of graph structure information. These rules incorporate the concept of spreading activation, enabling a more comprehensive understanding of the relationships between nodes and edges in the graph. During the testing phase, the EMGE model is validated on three datasets, including WN18RR, FB15k-237, and a private Chinese financial dataset. The experimental results demonstrate a reduction in the mean rank (MR) by 0.2 times, an improvement in the mean reciprocal rank (MRR) by 5.9%, and an increase in the Hit@1 by 12.9% compared to the baseline model.

Fusion and classification algorithm of octacalcium phosphate production based on XRD and FTIR data

The present manuscript tested an automated analysis sequence to provide a decision support system to track the OCP synthesis from α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\alpha$$\\end{document}-TCP over time. Initially, the XRD and FTIR signals from a hundredfold scaled-up hydrolysis of OCP from α\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\alpha$$\\end{document}-TCP were fused and modeled by the curve fitting based on the significantly established maxima from the literature and nine features extracted from the fitted shapes. Afterward, the analysis sequence enclosed the machine learning techniques for feature ranking, spatial filtering, and dimensionality reduction to support the automatic recognition of the synthesis stages. The proposed analysis pipeline for OCP identification might be the foundation for a decision support system explicitly targeting OCP synthesis. Future projects will exploit the suggested methodology for pinpointing the OCP production over time (including the intermediary phases present in the OCP formation) and for evaluating whether biological variables might be merged with biomaterial properties to build a unified model of tissue response to the implant.

Sensor Network Localization via Riemannian Conjugate Gradient and Rank Reduction

Sensor Network Localization via Riemannian Conjugate Gradient and Rank Reduction

Deep Learning Model Compression With Rank Reduction in Tensor Decomposition.

Large neural network models are hard to deploy on lightweight edge devices demanding large network bandwidth. In this article, we propose a novel deep learning (DL) model compression method. Specifically, we present a dual-model training strategy with an iterative and adaptive rank reduction (RR) in tensor decomposition. Our method regularizes the DL models while preserving model accuracy. With adaptive RR, the hyperparameter search space is significantly reduced. We provide a theoretical analysis of the convergence and complexity of the proposed method. Testing our method for the LeNet, VGG, ResNet, EfficientNet, and RevCol over MNIST, CIFAR-10/100, and ImageNet datasets, our method outperforms the baseline compression methods in both model compression and accuracy preservation. The experimental results validate our theoretical findings. For the VGG-16 on CIFAR-10 dataset, our compressed model has shown a 0.88% accuracy gain with 10.41 times storage reduction and 6.29 times speedup. For the ResNet-50 on ImageNet dataset, our compressed model results in 2.36 times storage reduction and 2.17 times speedup. In federated learning (FL) applications, our scheme reduces 13.96 times the communication overhead. In summary, our compressed DL method can improve the image understanding and pattern recognition processes significantly.

Joint Reconstruction and Multiple Attenuation Using One-Step Randomized-Order Damped Rank Reduction Method

Joint Reconstruction and Multiple Attenuation Using One-Step Randomized-Order Damped Rank Reduction Method

Direction of Arrival Estimation With Gain-Phase Uncertainties in Unknown Nonuniform Noise

This paper considers the problem of direction-of-arrival (DOA) estimation for narrowband source signals under the coexistence of gain-phase uncertainties and nonuniform noise. Firstly, by exploiting the low-rank property of the covariance of signal components, and the diagonal property of noise covariance, an optimization problem with low-rank constraints is formulated to estimate the nonuniform noise covariance matrix, which is then solved efficiently by the alternating direction method of multipliers (ADMM). Secondly, a linear transformation operator is employed to construct a rank reduction (RARE) estimator, and subsequently an initial DOA and gain-phase uncertainties are obtained via one-dimensional (1-D) spectral search. With initial estimates, an augmented DOA estimation is finally achieved by constructing a weighted and structured sparse total least-squares (WSS-TLS) optimization problem, which is solved using the block coordinate descent algorithm. Simulation results show the superior performance of the proposed method, and it achieves better tradeoff between computational complexity and performance, compared to existing methods.

Precise Recovery of Corrupted Synchrophasors Based on Autoregressive Bayesian Low-Rank Factorization and Adaptive K-Medoids Clustering

Phasor measurement unit (PMU) data quality problems, such as data loss and modification caused by communication contingencies and cyber attacks, threaten the reliability and stability of modern power systems. In this paper, autoregressive Bayesian low-rank matrix/Hankel tensor factorization (BLMF/BLTF) algorithms are proposed to recover corrupted synchrophasor measurements by leveraging the latent electrical-temporal correlations between data points. To mitigate the impact of consecutive data anomalies on recovery, adaptive k-Medoids clustering based on dynamic time wrapping (DTW) distance and Canopy clustering is introduced to preprocess raw data. The effectiveness of the proposed algorithms in recovering missing or modified synchrophasor measurements is demonstrated through comprehensive case studies. Compared with previous works, the proposed algorithms rely on simpler prior hypotheses and can achieve higher recovery precision. Rank reduction and low-cost rolling prediction techniques make the proposed algorithms suitable not only for processing offline data blocks but also for online real-time recovery of PMU data streams.

Who gets left behind by left behind places?

We document that children growing up in places left behind by today's economy experience lower levels of social mobility as adults. Using a longitudinal database that tracks over 20,000 places in the USA from 1980 to 2018, we identify two kinds of left behind places: the 'long-term left behind' that have struggled over long periods of history; and 'recently left-behind' places where conditions have deteriorated. Compared to children of similar baseline household income levels, we find that exposure to left behind places is associated with a 4-percentile reduction in adult income rank. Children fare considerably better when exposed to places where conditions are improving. These outcomes vary across prominent social and spatial categories and are compounded when nearby places are also experiencing hardship. Based on these findings, we argue that left behind places are having 'scarring effects' on children that could manifest long into the future, exacerbating the intergenerational challenges faced by low-income households and communities. Improvements in local economic conditions and outmigration to more prosperous places are, therefore, unlikely to be full remedies for the problems created by left behind places.

DRR: An open-source multi-platform package for the damped rank-reduction method and its applications in seismology

We present an open-source multi-platform package for the damped rank-reduction (DRR) method, that has been widely used in various problems in seismology. The DRR method is an optimally improved rank-reduction (RR) method by damping the singular values using an analytically derived threshold. The DRR method can separate the useful signal and unwanted noise in multi-channel seismic data better than the standard RR method, resulting in spatially more coherent seismic events without damaging the critical subtleties. There have been some advanced methodological improvements or ingenious implementations of the DRR method in the past decade, as well as dozens of new applications in different areas of seismology, most of which will be included in the presented package. The DRR package is composed of two parts: MATdrr and Pydrr, corresponding to the Matlab and Python versions, respectively. The current DRR package includes localized DRR, optimally damped rank-reduction (ODRR), off-the-grid reconstruction, de-aliased reconstruction, and many reproducible examples in different areas of seismology. The DRR package will be continuously developed to include more functionalities in the future.

Localization of mixed‐field sources based on fourth‐order cumulant matrix rank reduction

AbstractIn this letter, a localization algorithm for mixed‐field sources based on fourth‐order cumulant matrix rank reduction is proposed. First, a cumulant matrix is constructed, and a spectral peak search function associated with the direction‐of‐arrival (DOA) of the sources only is constructed using the rank reduction principle to achieve DOA estimation. Then the range estimation is also obtained by a spectral peak search. Compared to existing algorithms, the algorithm is able to utilize all the elements of the cumulant matrix and has a higher accuracy and degrees‐of‐freedom. The simulation results verify the excellence of the algorithm.

Does exposure to democracy decrease health inequality?

Abstract Exposure to (a liberal) democracy can have an impact on both the political attention and visibility of the needs of marginalized populations, as well as the design of health policies that can influence the distribution of population health. This paper investigates the effect of exposure to democracy, that is, the number of years spent in a democracy as measured by democracy indexes, on various measures of inequality in self-reported health across European countries. We use an instrumental variable strategy to leverage the potential endogeneity of a country’s exposure to democracy, drawing on both bivariate (socioeconomic) and univariate health inequality measures. Our estimates provide evidence that an additional year in a democracy reduces both bivariate (income-related) health inequality and overall (univariate) health inequality. Our preferred specification suggests a two-point rank reduction in inequality with an additional year under a democracy. The effect is mainly driven by a reduction of “health poverty” alongside other effects.

Tensor rank reduction via coordinate flows

Recently, there has been a growing interest in efficient numerical algorithms based on tensor networks and low-rank techniques to approximate high-dimensional functions and solutions to high-dimensional PDEs. In this paper, we propose a new tensor rank reduction method based on coordinate transformations that can greatly increase the efficiency of high-dimensional tensor approximation algorithms. The idea is simple: given a multivariate function, determine a coordinate transformation so that the function in the new coordinate system has smaller tensor rank. We restrict our analysis to linear coordinate transformations, which gives rise to a new class of functions that we refer to as tensor ridge functions. Leveraging Riemannian gradient descent on matrix manifolds we develop an algorithm that determines a quasi-optimal linear coordinate transformation for tensor rank reduction. The results we present for rank reduction via linear coordinate transformations open the possibility for generalizations to larger classes of nonlinear transformations.

Factorization of the Matrix of Orthogonal Projector on a Subspace

A deep factorization of the orthogonal projection matrix onto a subspace is obtained. The LQ decomposition is used. For construction of an orthogonal matrix Q the method of successive rank reduction is applied.

Simultaneous random noise attenuation and three‐dimensional seismic‐data interpolation with faster adaptive rank reduction

AbstractRank‐reduction‐based simultaneous random noise attenuation and three‐dimensional seismic‐data interpolation has recently become a hot topic in reflection seismology. However, the rank of traditional methods is fixed without considering the variation of signal‐to‐noise ratio on different frequency components, leading to serious residual noise and further affecting the following processing and interpretation tasks. In addition, traditional methods also heavily rely on the application of singular value decomposition technique for rank reduction, which is proven to be computationally expensive for large‐scale data. Thus, a fast‐adaptive rank‐reduction method is proposed in this study. First, the information entropy theory is introduced to adaptively select the optimal rank at various frequencies by calculating the increment of singular entropy. Second, we propose a fast Random Block Krylov algorithm and a subspace multiplexing technique to replace the singular value decomposition algorithm used in traditional methods. The proposed method can significantly improve computational efficiency and yield better seismic‐data reconstruction performance than traditional methods. Applications of the proposed approach on both synthetic and field seismic data demonstrate its superior performance over a well‐known rank‐reduction‐based method, that is the random multi‐channel singular spectrum analysis, in terms of recovered signal‐to‐noise ratio and visual view.

Three dimensional seismic data reconstruction based on truncated nuclear norm

The rank reduction method is widely used to reconstruct three dimensional (3D) seismic data. The traditional multichannel singular spectrum analysis (MSSA) utilizes truncated singular value decomposition (TSVD) to approximately solve the rank function of the block Hankel structure matrix constructed by frequency slices of seismic data. However, the TSVD algorithm discards all nonzero singular values except a few largest singular values and ignores the useful seismic information in small nonzero singular values. To further utilize the seismic data information contained in these small singular values, this paper proposes a method to recover 3D seismic data with truncated nuclear norm (TNN). The proposed method imposes different constraints on large singular values and small singular values. Essentially, the TNN is closer to the rank function than the TSVD. Finally, the proposed model is addressed by the alternating direction method of multipliers, and closed-form solutions are used to update the optimization variables. The experimental results demonstrate that the proposed method achieves superior reconstruction results than the traditional MSSA method.

Mixed source localization considering mutual coupling and unknown nonuniform noise under exact spatial geometry

A mixed far-field (FF) and near-field (NF) source localization method in the presence of unknown mutual coupling and nonuniform noise is proposed in this paper, where the exact spatial geometry considering propagation attenuation is adopted. First, the nonuniform noise covariance and FF direction of arrival (DOA) are estimated by the iterative least-squares subspace (ILSS) technique and the principle of rank reduction (RARE), respectively. Then, the NF source components are extracted by applying the oblique projection technique. Finally, by choosing three relative sensor positions and constructing four special cumulant matrices properly, NF DOA and range information are obtained in closed-form expressions, where a scheme to avoid ambiguity is exploited. Simulation result show that the proposed method can yield an improved performance in comparison with existing state-of-the-art methods.

ClusterMaker2: a major update to clusterMaker, a multi-algorithm clustering app for Cytoscape

BackgroundSince the initial publication of clusterMaker, the need for tools to analyze large biological datasets has only increased. New datasets are significantly larger than a decade ago, and new experimental techniques such as single-cell transcriptomics continue to drive the need for clustering or classification techniques to focus on portions of datasets of interest. While many libraries and packages exist that implement various algorithms, there remains the need for clustering packages that are easy to use, integrated with visualization of the results, and integrated with other commonly used tools for biological data analysis. clusterMaker2 has added several new algorithms, including two entirely new categories of analyses: node ranking and dimensionality reduction. Furthermore, many of the new algorithms have been implemented using the Cytoscape jobs API, which provides a mechanism for executing remote jobs from within Cytoscape. Together, these advances facilitate meaningful analyses of modern biological datasets despite their ever-increasing size and complexity.ResultsThe use of clusterMaker2 is exemplified by reanalyzing the yeast heat shock expression experiment that was included in our original paper; however, here we explored this dataset in significantly more detail. Combining this dataset with the yeast protein–protein interaction network from STRING, we were able to perform a variety of analyses and visualizations from within clusterMaker2, including Leiden clustering to break the entire network into smaller clusters, hierarchical clustering to look at the overall expression dataset, dimensionality reduction using UMAP to find correlations between our hierarchical visualization and the UMAP plot, fuzzy clustering, and cluster ranking. Using these techniques, we were able to explore the highest-ranking cluster and determine that it represents a strong contender for proteins working together in response to heat shock. We found a series of clusters that, when re-explored as fuzzy clusters, provide a better presentation of mitochondrial processes.ConclusionsclusterMaker2 represents a significant advance over the previously published version, and most importantly, provides an easy-to-use tool to perform clustering and to visualize clusters within the Cytoscape network context. The new algorithms should be welcome to the large population of Cytoscape users, particularly the new dimensionality reduction and fuzzy clustering techniques.

Flexible Hyperspectral Anomaly Detection Using Weighted Nuclear Norm

It has been demonstrated that nuclear-norm-based low-rank representation is capable of modeling cluttered backgrounds in hyperspectral images (HSIs) for robust anomaly detection. However, minimizing the nuclear norm regularizes each singular value equally during rank reduction, which restricts the capacity and flexibility of modeling the major structures of the background. To address this problem, we propose detection of anomaly pixels in HSIs using the weighted nuclear norm, which can preserve the major singular values during rank reduction. We present a down-up sampling scheme to remove plausible anomaly pixels from the image as much as possible and learn a robust principal component analysis (PCA) background dictionary. From a dictionary, we develop a weighted nuclear-norm minimization model to represent the background with a low-rank coefficients matrix that can be effectively optimized using the standard alternating direction method of multipliers (ADMM). Due to the flexible modeling capacity using the weighted nuclear norm, anomaly pixels can be distinguished from the background with the reconstruction error. The experimental results on two real HSIs datasets demonstrate the effectiveness of the proposed method for anomaly detection.

Non-negative low-rank approximations for multi-dimensional arrays on statistical manifold

Although low-rank approximation of multi-dimensional arrays has been widely discussed in linear algebra, its statistical properties remain unclear. In this paper, we use information geometry to uncover a statistical picture of non-negative low-rank approximations. First, we treat each input array as a probability distribution using a log-linear model on a poset, where a structure of an input array is realized as a partial order. We then describe the low-rank condition of arrays as constraints on parameters of the model and formulate the low-rank approximation as a projection onto a subspace that satisfies such constraints, where parameters correspond to coordinate systems of a statistical manifold. Second, based on information-geometric analysis of low-rank approximation, we point out the unexpected relationship between the rank-1 non-negative low-rank approximation and mean-field approximation, a well-established method in physics that uses a one-body problem to approximate a many-body problem. Third, our theoretical discussion leads to a novel optimization method of non-negative low-rank approximation, called Legendre Tucker rank reduction. Because the proposed method does not use the gradient method, it does not require tuning parameters such as initial position, learning rate, and stopping criteria. In addition, the flexibility of the log-linear model enables us to treat the problem of non-negative multiple matrix factorization (NMMF), a variant of low-rank approximation with shared factors. We find the best rank-1 NMMF formula as a closed form and develop a rapid rank-1 NMF method for arrays with missing entries based on the closed form, called A1GM.