Projective Non-negative Matrix Factorization Research Articles

Longitudinal Relationship Between Brain Atrophy Patterns, Cognitive Decline, and Cerebrospinal Fluid Biomarkers in Alzheimer's Disease Explored by Orthonormal Projective Non-Negative Matrix Factorization.

Longitudinal magnetic resonance imaging (MRI) has been proposed for tracking the progression of Alzheimer's disease (AD) through the assessment of brain atrophy. Detection of brain atrophy patterns in patients with AD as the longitudinal disease tracker. We used a refined version of orthonormal projective non-negative matrix factorization (OPNMF) to identify six distinct spatial components of voxel-wise volume loss in the brains of 83 subjects with AD from the ADNI3 cohort relative to healthy young controls from the ABIDE study. We extracted non-negative coefficients representing subject-specific quantitative measures of regional atrophy. Coefficients of brain atrophy were compared to subjects with mild cognitive impairment and controls, to investigate the cross-sectional and longitudinal associations between AD biomarkers and regional atrophy severity in different groups. We further validated our results in an independent dataset from ADNI2. The six non-overlapping atrophy components represent symmetric gray matter volume loss primarily in frontal, temporal, parietal and cerebellar regions. Atrophy in these regions was highly correlated with cognition both cross-sectionally and longitudinally, with medial temporal atrophy showing the strongest correlations. Subjects with elevated CSF levels of TAU and PTAU and lower baseline CSF Aβ42 values, demonstrated a tendency toward a more rapid increase of atrophy. The present study has applied a transferable method to characterize the imaging changes associated with AD through six spatially distinct atrophy components and correlated these atrophy patterns with cognitive changes and CSF biomarkers cross-sectionally and longitudinally, which may help us better understand the underlying pathology of AD.

Mapping the macrostructure and microstructure of the in vivo human hippocampus using diffusion MRI.

The hippocampus is classically divided into mesoscopic subfields which contain varying microstructure that contribute to their unique functional roles. It has been challenging to characterize this microstructure with current magnetic resonance based neuroimaging techniques. In this work, we used diffusion magnetic resonance imaging (dMRI) and a novel surface-based approach in the hippocampus which revealed distinct microstructural distributions of neurite density and dispersion, T1w/T2w ratio as a proxy for myelin content, fractional anisotropy, and mean diffusivity. We used the neurite orientation dispersion and density imaging (NODDI) model optimized for grey matter diffusivity to characterize neurite density and dispersion. We found that neurite dispersion was highest in the cornu ammonis (CA) 1 and subiculum subfields which likely captures the large heterogeneity of tangential and radial fibres, such as the Schaffer collaterals, perforant path, and pyramidal neurons. Neurite density and T1w/T2w were highest in the subiculum and CA3 and lowest in CA1, which may reflect known myeloarchitectonic differences between these subfields. Using a simple logistic regression model, we showed that neurite density, dispersion, and T1w/T2w measures were separable across the subfields, suggesting that they may be sensitive to the known variability in subfield cyto- and myeloarchitecture. We report macrostructural measures of gyrification, thickness, and curvature that were in line with ex vivo descriptions of hippocampal anatomy. We employed a multivariate orthogonal projective non-negative matrix factorization (OPNNMF) approach to capture co-varying regions of macro- and microstructure across the hippocampus. The clusters were highly variable along the medial-lateral (proximal-distal) direction, likely reflecting known differences in morphology, cytoarchitectonic profiles, and connectivity. Finally, we show that by examining the main direction of diffusion relative to canonical hippocampal axes, we could identify regions with stereotyped microstructural orientations that may map onto specific fibre pathways, such as the Schaffer collaterals, perforant path, fimbria, and alveus. These results highlight the value of combining in vivo dMRI with computational approaches for capturing hippocampal microstructure, which may provide useful features for understanding cognition and for diagnosis of disease states.

Exploring the influence of functional architecture on cortical thickness networks in early psychosis – A longitudinal study

Cortical thickness reductions differ between individuals with psychotic disorders and comparison subjects even in early stages of illness. Whether these reductions covary as expected by functional network membership or simply by spatial proximity has not been fully elucidated. Through orthonormal projective non-negative matrix factorization, cortical thickness measurements in functionally-annotated regions from MRI scans of early-stage psychosis and matched healthy controls were reduced in dimensionality into features capturing positive covariance. Rather than matching the functional networks, the covarying regions in each feature displayed a more localized spatial organization. With Bayesian belief networks, the covarying regions per feature were arranged into a network topology to visualize the dependency structure and identify key driving regions. The features demonstrated diagnosis-specific differences in cortical thickness distributions per feature, identifying reduction-vulnerable spatial regions. Differences in key cortical thickness features between psychosis and control groups were delineated, as well as those between affective and non-affective psychosis. Clustering of the participants, stratified by diagnosis and clinical variables, characterized the clinical traits that define the cortical thickness patterns. Longitudinal follow-up revealed that in select clusters with low baseline cortical thickness, clinical traits improved over time. Our study represents a novel effort to characterize brain structure in relation to functional networks in healthy and clinical populations and to map patterns of cortical thickness alterations among ESP patients onto clinical variables for a better understanding of brain pathophysiology.

Scalable NMF via linearly optimized data compression.

Orthonormal projective non-negative matrix factorization (opNMF) has been widely used in neuroimaging and clinical neuroscience applications to derive representations of the brain in health and disease. The non-negativity and orthonormality constraints of opNMF result in intuitive and well-localized factors. However, the advantages of opNMF come at a steep computational cost that prohibits its use in large-scale data. In this work, we propose novel and scalable optimization schemes for orthonormal projective non-negative matrix factorization that enable the use of the method in large-scale neuroimaging settings. We replace the high-dimensional data matrix with its corresponding singular value decomposition (SVD) and QR decompositions and combine the decompositions with opNMF multiplicative update algorithm. Empirical validation of the proposed methods demonstrated significant speed-up in computation time while keeping memory consumption low without compromising the accuracy of the solution.

Scalable Orthonormal Projective NMF via Diversified Stochastic Optimization.

The increasing availability of large-scale neuroimaging initiatives opens exciting opportunities for discovery science of human brain structure and function. Data-driven techniques, such as Orthonormal Projective Non-negative Matrix Factorization (opNMF), are well positioned to explore multivariate relationships in big data towards uncovering brain organization. opNMF enjoys advantageous interpretability and reproducibility compared to commonly used matrix factorization methods like Principal Component Analysis (PCA) and Independent Component Analysis (ICA), which led to its wide adoption in clinical computational neuroscience. However, applying opNMF in large-scale cohort studies is hindered by its limited scalability caused by its accompanying computational complexity. In this work, we address the computational challenges of opNMF using a stochastic optimization approach that learns over mini-batches of the data. Additionally, we diversify the stochastic batches via repulsive point processes, which reduce redundancy in the mini-batches and in turn lead to lower variance in the updates. We validated our framework on gray matter tissue density maps estimated from 1000 subjects part of the Open Access Series of Imaging (OASIS) dataset. We demonstrated that operations over mini-batches of data yield significant reduction in computational cost. Importantly, we showed that our novel optimization does not compromise the accuracy or interpretability of factors when compared to standard opNMF. The proposed model enables new investigations of brain structure using big neuroimaging data that could improve our understanding of brain structure in health and disease.

Bi-level optimization-based projective non-negative matrix factorization

Non-negative matrix factorization (NMF) is a powerful tool for image data analysis and has been utilized in a range of applications involving data classification, clustering, and image processing. Nevertheless, NMF and most of its variants are unsupervised learning algorithms and do not take account of the projection property of the basis image matrix. These flaws will have a negative impact on their classification performance. To resolve the mentioned drawbacks of NMF-based methods, this paper establishes a novel projective non-negative matrix factorization (PNMF) framework using the class-label information and deals with the model via a bi-level quadratic optimization (BLO) strategy. The convergence of the proposed BLO-PNMF algorithm is theoretically analyzed and proved in detail, and also verified by experiments. Our approach and the related state-of-the-art methods are evaluated and compared on three face databases. The experimental results indicate that the BLO-PNMF method can extract more localized features for image data representation and achieve superior performance in face recognition.

Novel Gyrification Networks Reveal Links with Psychiatric Risk Factors in Early Illness.

Adult gyrification provides a window into coordinated early neurodevelopment when disruptions predispose individuals to psychiatric illness. We hypothesized that the echoes of such disruptions should be observed within structural gyrification networks in early psychiatric illness that would demonstrate associations with developmentally relevant variables rather than specific psychiatric symptoms. We employed a new data-driven method (Orthogonal Projective Non-Negative Matrix Factorization) to delineate novel gyrification-based networks of structural covariance in 308 healthy controls. Gyrification within the networks was then compared to 713 patients with recent onset psychosis or depression, and at clinical high-risk. Associations with diagnosis, symptoms, cognition, and functioning were investigated using linear models. Results demonstrated 18 novel gyrification networks in controls as verified by internal and external validation. Gyrification was reduced in patients in temporal-insular, lateral occipital, and lateral fronto-parietal networks (pFDR < 0.01) and was not moderated by illness group. Higher gyrification was associated with better cognitive performance and lifetime role functioning, but not with symptoms. The findings demonstrated that gyrification can be parsed into novel brain networks that highlight generalized illness effects linked to developmental vulnerability. When combined, our study widens the window into the etiology of psychiatric risk and its expression in adulthood.

A Sparse Dimensionality Reduction Approach Based on False Nearest Neighbors for Nonlinear Fault Detection

As a newly emerging multivariate statistical process monitoring method, non-negative matrix factorization (NMF) and its variants avoid the positive and negative cancellations between the extracted features because of the purely additive combination of non-negative components. Thus, they make a good match with this reality that the negative values of both observations and decomposed components are physically meaningless in many kinds of industrial processes. However, these methods are effective only for linearly separable problems and are not suitable for dealing with nonlinear process monitoring. In this article, the kernel-based method is integrated into the projective NMF (KPNMF) to improve the accuracy of fault detection, and the appropriate multiplicative update method is proven to be convergent. Furthermore, inspired by the false nearest neighbors (FNNs) method, a new dimensionality reduction approach (KPNMF-FNN) is presented to further reduce the original variables for determining the smallest dimension regression vector needed. Compared with the traditional methods, the proposed approach can greatly reduce the time and storage space required on the premise of maintaining a high fault detection rate and low false alarm rate. The experimental results on the Tennessee Eastman benchmark process and the pumping unit system show that the proposed algorithms have excellent performance and can effectively detect faults under the circumstances of retaining only the top 60% of the original variables.

ScPNMF: sparse gene encoding of single cells to facilitate gene selection for targeted gene profiling.

ABSTRACT:MotivationSingle-cell RNA sequencing (scRNA-seq) captures whole transcriptome information of individual cells. While scRNA-seq measures thousands of genes, researchers are often interested in only dozens to hundreds of genes for a closer study. Then, a question is how to select those informative genes from scRNA-seq data. Moreover, single-cell targeted gene profiling technologies are gaining popularity for their low costs, high sensitivity and extra (e.g. spatial) information; however, they typically can only measure up to a few hundred genes. Then another challenging question is how to select genes for targeted gene profiling based on existing scRNA-seq data.ResultsHere, we develop the single-cell Projective Non-negative Matrix Factorization (scPNMF) method to select informative genes from scRNA-seq data in an unsupervised way. Compared with existing gene selection methods, scPNMF has two advantages. First, its selected informative genes can better distinguish cell types. Second, it enables the alignment of new targeted gene profiling data with reference data in a low-dimensional space to facilitate the prediction of cell types in the new data. Technically, scPNMF modifies the PNMF algorithm for gene selection by changing the initialization and adding a basis selection step, which selects informative bases to distinguish cell types. We demonstrate that scPNMF outperforms the state-of-the-art gene selection methods on diverse scRNA-seq datasets. Moreover, we show that scPNMF can guide the design of targeted gene profiling experiments and the cell-type annotation on targeted gene profiling data.Availability and implementationThe R package is open-access and available at https://github.com/JSB-UCLA/scPNMF. The data used in this work are available at Zenodo: https://doi.org/10.5281/zenodo.4797997.Supplementary information Supplementary data are available at Bioinformatics online.

Patterns of Structural Covariance Abnormalities and Clinical Correlations in Schizophrenia

Structural covariance networks provide data-driven and biologically-relevant brain parcellations, potentially increasing power to detect disease effects. Here, we utilized orthogonal projective non-negative matrix factorization (opNMF) to investigate patterns of structural covariance (PSCs) in schizophrenia (SCZ), and their clinical correlates in a large, multi-site dataset.

Individual Differences in Intrinsic Brain Networks Predict Symptom Severity in Autism Spectrum Disorders.

The neurobiology of heterogeneous neurodevelopmental disorders such as Autism Spectrum Disorders (ASD) is still unknown. We hypothesized that differences in subject-level properties of intrinsic brain networks were important features that could predict individual variation in ASD symptom severity. We matched cases and controls from a large multicohort ASD dataset (ABIDE-II) on age, sex, IQ, and image acquisition site. Subjects were matched at the individual level (rather than at group level) to improve homogeneity within matched case-control pairs (ASD: n = 100, mean age = 11.43years, IQ = 110.58; controls: n = 100, mean age = 11.43years, IQ = 110.70). Using task-free functional magnetic resonance imaging, we extracted intrinsic functional brain networks using projective non-negative matrix factorization. Intrapair differences in strength in subnetworks related to the salience network (SN) and the occipital-temporal face perception network were robustly associated with individual differences in social impairment severity (T = 2.206, P = 0.0301). Findings were further replicated and validated in an independent validation cohort of monozygotic twins (n = 12; 3 pairs concordant and 3 pairs discordant for ASD). Individual differences in the SN and face-perception network are centrally implicated in the neural mechanisms of social deficits related to ASD.

Background image-assisted divide-and-conquer reconstruction method for ECT

Electrical capacitance tomography (ECT) is a potential image-based measurement technology for monitoring time-varying industrial processes, but its applicability is challenged by low-quality images. To address this conundrum, a two-stage reconstruction (TSR) method with more effective priors and optimizer is presented in this work. In the first stage, the random vector functional link network (RVFLN) is developed to calculate a data-dependent background image, and a new distributed computing method is developed to achieve efficient training. To decrease the expensive computation load in the RVFLN training, the regularized projective nonnegative matrix factorization (RPNMF) method is developed to diminish the size of the sample data. A new optimization problem (OP) is proposed to model the RPNMF problem, which is solved by a new optimizer efficiently. In the second stage, a new OP that achieves the confluence of the domain knowledge-based prior related to imaging objects and the measurement physics is built, and a new divide-and-conquer optimizer is devised to solve the OP. To reduce the difficulty of parameter adjustment, the background image is used to initialize the proposed computing algorithm, and such treatment not only achieves the simultaneous fusion of the domain knowledge-based prior and the data-dependent prior but also decreases the computational difficulty. Numerical results show that the TSR algorithm is not only robust but also can achieve high precision reconstruction in comparison with popular imaging techniques.

Robust Ensemble Manifold Projective Non-Negative Matrix Factorization for Image Representation

Projective non-negative matrix factorization (PNMF) as a variant of NMF has received considerable attention. However, the existing PNMF methods can be further improved from two aspects. On the one hand, the square loss function that is intended to measure the reconstruction error is sensitive to noise. On the other hand, it is non-trivial to estimate the intrinsic manifold of the feature space in a principal manner. So current paper is an attempt that has proposed a new method named as robust ensemble manifold projective non-negative matrix factorization (REPNMF) for image representation. Specifically, REPNMF not only assesses the influence of noise by imposing a spare noise matrix for image reconstruction, but it also assumes that the intrinsic manifold exists in a convex hull of certain pre-given manifold candidates. We aim to remove noise from the data and find the optimized combination of candidate manifolds to approximate the intrinsic manifold simultaneously. We develop iterative multiplicative updating rules for the optimization of REPNMF along with its convergence proof. The experimental results on four image datasets verify that REPNMF is superior as compare to other related state-of-the-art methods.

Face Recognition based on Linear Projective Non-negative Matrix Factorization with Kullback-Leibler Divergence

In order to solve the problem of feature extraction and classification of Non-negative Matrix Factorization (NMF) in face recognition, a Linear Projective Non-negative Matrix Factorization method based on Kullback-Leibler divergence (LP-NMF-DIV) is proposed. In LP-NMF-DIV, an objective function of Kullback-Leibler divergence is considered. The Taylor series expansion and the Newton iteration formula are used to find the root formula. The iterative algorithm of basis matrix and linear transformation matrix are derived, and the convergence of the algorithm is proved. The results of experimental show that the algorithm is convergent. In face recognition, compared with some typical NMF methods, the algorithm has high recognition accuracy; when the ranks of the basis matrices are set to different values, the algorithm is stable. The method for LP-NMF-DIV is effective.

Hybrid projective nonnegative matrix factorization based on α-divergence and the alternating least squares algorithm

Nonnegative Matrix Factorization (NMF) is a linear dimensionality reduction technique for extracting hidden and intrinsic features of high-dimensional data sets. Recently, several Projective NMF (P-NMF) methods have been proposed for the purpose of resolving issues associated with the standard NMF approach. Experimental results show that P-NMF algorithms outperform the standard NMF method in some aspects. But some basic issues still affect the existing NMF and P-NMF methods, these include slow convergence rate, low reconstruction accuracy and dense basis factors. In this article, we propose a new and generalized hybrid algorithm by combining the concept of alternating least squares with the multiplicative update rules of the α-divergence-based P-NMF method. We have conducted extensive numerical experiments on 7 real-world data sets and compared the new algorithm with several state-of-the-art methods. The attractive features and added advantages of the new algorithm include remarkable clustering performances, providing highly “orthogonal” and very sparse basis factors, and extracting distinctive and better localized features of the original data than its counterparts.

Design of various Image Compression Methods in Wireless Sensor Networks

The processing capacity and power of nodes in a Wireless Sensor Network (WSN) are restricted. The quality of the images is deficient, and the contents of the images may vary after decoding when we apply image compression algorithms in WSN. Various compression algorithms are compared in this paper. An Image Compression method based on Restricted Boltzmann Machine (RBM), Auto encoders and Non-negative Matrix Factorization (NMF), Least Square Non-Negative Matrix Factorization (LSNMF), Projective Non-Negative Matrix Factorization (PNMF) network are proposed in this paper. For the WSN, we have used a Message Queue Telemetry Transport (MQTT) protocol. We have used a three Raspberry Pi’s to build a WSN; Publisher, Broker, Subscriber. A Publisher, where it can trigger the camera and captures the images then compress it and send it to another raspberry pi which is a MQTT broker. The PSNR values for those image compression methods were analyzed and compared against each other for images evaluated from the MNIST dataset. Along with the simulation results, all these compression methods are implemented using hardware implementation. Raspberry Pi, a single-board computer with in-built Wi-Fi capabilities, was used in establishing a WSN. Message Queue Telemetry Transport (MQTT) protocol was used for transmitting the compressed images across the WSN, that offers fast and reliable transmission.

Unsupervised Negative Link Prediction in Signed Social Networks

It has been proved in a number of applications that it is useful to predict unknown social links, and link prediction has played an important role in sociological study. Although there has been a surge of pertinent approaches to link prediction, most of them focus on positive link prediction while giving few attentions to the problem of inferring unknown negative links. The inherent characteristics of negative relations present great challenges to traditional link prediction: (1) there are very few negative interaction data; (2) negative links are much sparser than positive links; (3) social data is often noisy, incomplete, and fast-evolved. This paper intends to address this novel problem by solely leveraging structural information and further proposes the UN-PNMF framework based on the projective nonnegative matrix factorization, so as to incorporate network embedding and user’s property embedding into negative link prediction. Empirical experiments on real-world datasets corroborate their effectiveness.

Adaptive non-negative projective semi-supervised learning for inductive classification

We discuss the inductive classification problem by proposing a joint framework termed Adaptive Non-negative Projective Semi-Supervised Learning (ANP-SSL). Specifically, ANP-SSL integrates the adaptive inductive label propagation, adaptive reconstruction weights learning and the neighborhood preserving projective nonnegative matrix factorization (PNMF) explicitly. To make the label prediction results more accurate, ANP-SSL incorporates the semi-supervised data representation and classification errors into regular PNMF for minimization, which can enable our ANP-SSL to perform the adaptive weights learning and label propagation over the spatially local and part-based data representations, which differs from most existing work that usually assign weights and predict labels based on the original data that often has noise and corruptions. Moreover, existing methods usually pre-assign weights before the process of label estimation, but such operation cannot ensure the learnt weights by independent step to be optimal for the subsequent classification. The combined representation error can also make the learnt reduced part-based representations of neighborhood preserving PNMF, which can potentially enhance the prediction results. By minimizing the classification error jointly over the neighborhood preserving nonnegative representation can make the embedding based classification efficient. Extensive results on several public image databases verified the effectiveness of our ANP-SSL, compared with other state-of-the-art methods.

Hybrid Projective Nonnegative Matrix Factorization With Drum Dictionaries for Harmonic/Percussive Source Separation

One of the most general models of music signals considers that such signals can be represented as a sum of two distinct components: a tonal part that is sparse in frequency and temporally stable and a transient (or percussive) part that is composed of short-term broadband sounds. In this paper, we propose a novel hybrid method built upon nonnegative matrix factorization (NMF) that decomposes the time frequency representation of an audio signal into such two components. The tonal part is estimated by a sparse and orthogonal nonnegative decomposition, and the transient part is estimated by a straightforward NMF decomposition constrained by a pre-learned dictionary of smooth spectra. The optimization problem at the heart of our method remains simple with very few hyperparameters and can be solved thanks to simple multiplicative update rules. The extensive benchmark on a large and varied music database against four state of the art harmonic/percussive source separation algorithms demonstrate the merit of the proposed approach.

Projective complex matrix factorization for facial expression recognition

In this paper, a dimensionality reduction method applied on facial expression recognition is investigated. An unsupervised learning framework, projective complex matrix factorization (proCMF), is introduced to project high-dimensional input facial images into a lower dimension subspace. The proCMF model is related to both the conventional projective nonnegative matrix factorization (proNMF) and the cosine dissimilarity metric in the simple manner by transforming real data into the complex domain. A projective matrix is then found through solving an unconstraint complex optimization problem. The gradient descent method was utilized to optimize a complex cost function. Extensive experiments carried on the extended Cohn-Kanade and the JAFFE databases show that the proposed proCMF model provides even better performance than state-of-the-art methods for facial expression recognition.