Non-negative Factorization Research Articles

Optimizing feature extraction for tampering image detection using deep learning approaches

Tamper image detection approach using deep learning involves, creating a model that can accurately identify and localize instances of image tampering, by employing advanced feature extraction methods, object detection algorithms, and optimization techniques that could be manipulated on need basis. Enhance the integrity of visual content by automating the detection of unauthorized alterations, to ensure the reliability of digital images across various applications and domains. The problem addressing the optimization feature extraction techniques involves the detection of subtle manipulations, handling diverse tampering techniques, and achieving robust performance across different types of images and scenarios. The proliferation of sophisticated image editing tools makes it challenging to detect tampered regions within images, necessitating proposed techniques for automated tamper image detection. The research work will focus on four different feature extraction algorithms such as non-negative factorization (NNF), singular value decomposition (SVD), explicit semantic analysis (ESA), principal component analysis (PCA), which are outsourced. Detecting tampered images through deep learning necessitates the meaningful selection and adjustment of several parameters to enhance the model's effectiveness. Integrating the feature extraction algorithm with the suggested methods effectively identifies critical features within the dataset, thereby improving the detection capabilities and achieving higher accuracy.

Multi-sample non-negative spatial factorization.

It is important to model biological variation when analyzing spatial transcriptomics data from multiple samples. One approach to multi-sample analysis is to spatially align samples, but this is a challenging problem. Here, we provide an alignment-free framework for generalizing a one-sample spatial factorization model to multi-sample data. Using this framework, we develop a method, called multi-sample non-negative spatial factorization (mNSF) that extends the one-sample non-negative spatial factorization (NSF) framework to a multi-sample dataset. Our model allows for a sample-specific model for the spatial correlation structure and extracts a low-dimensional representation of the data. We illustrate the performance of mNSF by simulation studies and real data. mNSF identifies true factors in simulated data, identifies shared anatomical regions across samples in real data and reveals region-specific biological functions. mNSFs performance is similar to alignment based methods when alignment is possible, but extends analysis to situations where spatial alignment is impossible. We expect multi-sample factorization methods to be a powerful class of methods for analyzing spatially resolved transcriptomics data.

Outranking-based approaches for multiple criteria partially ordered clustering: A review of existing algorithms, new proposals, and experimental comparison

We consider clustering problems that involve categorizing alternatives into partially ordered, initially undefined groups based on their performance across multiple criteria. To achieve this, we use an outranking relation model to reflect the Decision Maker's preferences. We examine various algorithms that not only group the alternatives but also order the clusters in different ways. This analysis includes innovative approaches that use distances in the space of outranking relations or detailed relation profiles, and apply orthogonal non-negative factorization to outranking matrices. Additionally, we discuss a set of measures, including two novel ones, for assessing the effectiveness of clustering when the groupings are partially ordered. Our findings are based on comprehensive computational experiments on real-world and simulated datasets. Beyond evaluating various methods using four quality metrics and computational efficiency, we explore the influence of accessible preference structures and ordering techniques on the clustering outcomes.

Generating Hidden Markov Models from Process Models Through Nonnegative Tensor Factorization

Monitoring of industrial processes is a critical capability in industry and in government to ensure reliability of production cycles, quick emergency response, and national security. Process monitoring allows users to gauge the progress of an organization in an industrial process or predict the degradation or aging of machine parts in processes taking place at a remote location. Similar to many data science applications, we usually only have access to limited raw data, such as satellite imagery, short video clips, event logs, and signatures captured by a small set of sensors. To combat data scarcity, we leverage the knowledge of Subject Matter Experts (SMEs) who are familiar with the actions of interest. SMEs provide expert knowledge of the essential activities required for task completion and the resources necessary to carry out each of these activities. Various process mining techniques have been developed for this type of analysis; typically such approaches combine theoretical process models built based on domain expert insights with ad-hoc integration of available pieces of raw data. Here, we introduce a novel mathematically sound method that integrates theoretical process models (as proposed by SMEs) with interrelated minimal Hidden Markov Models (HMM), built via nonnegative tensor factorization. Our method consolidates: (a) theoretical process models, (b) HMMs, (c) coupled nonnegative matrix-tensor factorizations, and (d) custom model selection. To demonstrate our methodology and its abilities, we apply it on simple synthetic and real world process models.

Toward deep drum source separation

In the past, the field of drum source separation faced significant challenges due to limited data availability, hindering the adoption of cutting-edge deep learning methods that have found success in other related audio applications. In this letter, we introduce StemGMD, a large-scale audio dataset of isolated single-instrument drum stems. Each audio clip is synthesized from MIDI recordings of expressive drum performances using ten real-sounding acoustic drum kits. Totaling 1224 h, StemGMD is the largest audio dataset of drums to date and the first to comprise isolated audio clips for every instrument in a canonical nine-piece drum kit. We leverage StemGMD to develop LarsNet, a novel deep drum source separation model. Through a bank of dedicated U-Nets, LarsNet can separate five stems from a stereo drum mixture faster than real-time and is shown to considerably outperform state-of-the-art nonnegative spectro-temporal factorization methods.

Natural frequency analysis of gear system with interval shaft misalignment in multidimensions

Due to the existing tolerances, the shaft misalignment of gear system is inevitably uncertain and shows interval property, which will cause the natural frequency to fluctuate in an interval range, therefore the traditional dynamics analysis that takes it as a deterministic parameter cannot reflect the interval characteristics of natural frequency. To overcome this conundrum, an improved interval eigenvalue algorithm based on matrix non-negative factorization and the Chebyshev inclusion function was proposed to find the natural frequency range of gear system under such uncertainty. First, the average meshing stiffness range of the gears with interval shaft misalignment is obtained by the Chebyshev interval approach. Then combined with the consideration of the remaining excitation parameter deviation coefficients, the global interval stiffness matrix and mass matrix of a single-stage gear system are obtained. Afterwards, the non-negative factorizations of these two interval matrices are carried out and parameter vertex theory is finally employed to obtain the upper and lower bounds of the eigenvalues. The feasibility of the applied algorithm is verified through comparison with the scanning method. Numerical results show that the uncertainty of shaft misalignment will enormously widen the system's resonance bandwidth and intensify oscillation, these will provide a new and valuable perspective for the tolerance allocation in gearbox design.

Application of Weibull Distribution to Hidden Markov Model for Non-Negative Factorization Matrix

Probabilistic Nonnegative Matrix Factorizations (NMFs) are very useful in statistics when dealing with stochastic signals such as wave fronts, share prices, and volatility as a Nonnegative Matrix Factorization (NMF) approach. Little attention has been made in the literature to developing NMF algorithms that use moving average to exploit data's temporal dependencies. A hidden Markov model (HMM) using a Weibull distribution as the output density function was created in this study. The Weibull HMM was then reformulated as a probabilistic NMF. This demonstrates the connection between the proposed HMM and NMF, and will lead to a novel probabilistic NMF approach in which the model captures temporal dependencies inherently utilizing moving average. Furthermore, the model parameters were estimated using maximum likelihood estimation (MLE). The model's adaptability was compared to the existing probabilistic NMFs models of gamma and lognormal. Our trials with US COVID-19 data revealed that the proposed technique achieves a superior balance of sparsity, the goodness of fit, and temporal modeling than gamma and lognormal models.

A Fine-Grained Regularization Scheme for Nonnegative Latent Factorization of High-Dimensional and Incomplete Tensors

A Fine-Grained Regularization Scheme for Nonnegative Latent Factorization of High-Dimensional and Incomplete Tensors

Clinico-Genomic Characterization of AML Patients Based on IL2RA (CD25) Expression Uncovers an Association with Stem Cell Signatures and FLT3-ITD Status and Informs Drug Combinations

Background Acute myeloid leukemia (AML) is a heterogeneous disease with poor outcomes, thus there remains a need to integrate molecular information to identify patients most likely to respond to drugs and combinations. Disease segmentation based on transcriptomics has provided valuable insight into disease risk and the likelihood of response to targeted compounds. IL2RA(CD25) is a receptor expressed on both AML leukemic and immune cells, and has shown initial promise as a potential dual target of leukemic blasts and regulatory T (Treg) cells in pre-clinical models (Pousse et al. Front. Oncol. 2023). We used a systems approach based on a recently described transcriptomic classifier for AML (Hamidi et al. ASH 2021) and other molecular tools to characterize the association between IL2RA expression levels and AML genomic markers, clinical features and patient outcomes. In addition, we evaluated IL2RA in relation to ex vivo drug sensitivity, to identify patient segments who are most likely to benefit from CD25-targeting drugs. Methods BEAT-AML (NCT01728402) RNAseq data from patient samples, associated ex vivo drug sensitivity data (N=283), was VOOM normalized. Patients were binned into quartiles based on IL2RA expression. Gene signature scoring was performed using xCell cell type enrichment algorithm and GSVA for Hallmark pathways and scRNAseq signature based on Van Galen et al. Cell 2019. Associations were performed with clinical outcome (log-rank test), gene signatures (Spearman correlation), mutations (Wilcoxon test) and ex vivo drug sensitivity (Kruskal Wallis test). Results We characterized the BeatAML dataset and found that elevated IL2RA expression associates with inferior overall survival (p=0.014) and high-risk features. In addition, we identified a strong association between IL2RA expression and FLT3-ITD status (Figure), as well as other genetic alterations. Using correlation analyses, we established an association between IL2RA expression levels and “primitive” AML signatures (leukemic stem cell (LSC), R=0.4, p=6.8x10-12, hematopoietic stem cell (HSC)-like, R=0.43, p=6.8x10-14) and Tregs (R=0.47, p=1.7x10-15), and an anti-correlation with promonocytic signatures (R=-0.23, p=1.1x10-4). Interestingly, HSC-like and Treg signatures were also correlated (R=0.19, p=0.00087), consistent with an association between stem cell abundance and a repressive immune microenvironment. Finally, using a multivariate model adjusting for the effect of Tregs and LSCs, the prognostic value of IL2RA remained significant (p=0.029). We previously used unsupervised machine learning clustering based on consensus non-negative factorization (cNMF) to discover novel transcription-based classification (Hamidi ASH 2021). Using this methodology, we identified a strong correlation between IL2RA expression in cNMF subtypes (p=1.6x10-10). Moreover, this method uncovered patient subtypes in which IL2RA correlates with LSC (cNMF 6.3), Treg (6.4, 6.6) or both signatures (6.1, 6.2). This differential association was independent of prognostic category, maturation state or venetoclax sensitivity (Table). Transcription-based classifiers have been shown to be highly predictive of ex vivo drug sensitivity. Consistent with an association between IL2RA expression and FLT3-ITD status, we identified strong correlations between IL2RA and sensitivity to FLT-3 and other tyrosine kinase inhibitors, providing a rationale for combining these compounds with CD25-targeting therapeutics. In contrast, there was no correlation between IL2RA levels and venetoclax AUC (p=0.25), while the cNMF classification system similarly revealed that patient segments with increased IL2RA expression had heterogeneous venetoclax sensitivity (Table), supporting CD25 targeting in patients who may not respond to venetoclax. Conclusion AML patients with elevated IL2RA expression have inferior prognosis, enrichment of stem-like and Treg signatures and FLT3-ITD alterations. Transcriptomics-based clustering could be used to guide combination therapies based on potential impact of CD25 targeting on Tregs, leukemic cells, or both populations. Ex vivo drug sensitivity analyses support combinations of CD25-targeting agents with FLT3 inhibitors, as well as venetoclax. Additional work to evaluate these and other combinations using functional assays is warranted.

Study of performance of low-rank nonnegative tensor factorization methods

Abstract In the present paper we compare two different iterative approaches to constructing nonnegative tensor train and Tucker decompositions. The first approach is based on idea of alternating projections and randomized sketching for factorization of tensors with nonnegative elements. This approach can be useful for both TT and Tucker formats. The second approach consists of two stages. At the first stage we find the unconstrained tensor train decomposition for the target array. At the second stage we use this initial approximation in order to fix it within moderate number of operations and obtain the factorization with nonnegative factors either in tensor train or Tucker model. We study the performance of these methods for both synthetic data and hyper-spectral image and demonstrate the clear advantage of the latter technique in terms of computational time and wider range of possible applications.

Nonnegative spatial factorization applied to spatial genomics

Nonnegative matrix factorization (NMF) is widely used to analyze high-dimensional count data because, in contrast to real-valued alternatives such as factor analysis, it produces an interpretable parts-based representation. However, in applications such as spatial transcriptomics, NMF fails to incorporate known structure between observations. Here, we present nonnegative spatial factorization (NSF), a spatially-aware probabilistic dimension reduction model based on transformed Gaussian processes that naturally encourages sparsity and scales to tens of thousands of observations. NSF recovers ground truth factors more accurately than real-valued alternatives such as MEFISTO in simulations, and has lower out-of-sample prediction error than probabilistic NMF on three spatial transcriptomics datasets from mouse brain and liver. Since not all patterns of gene expression have spatial correlations, we also propose a hybrid extension of NSF that combines spatial and nonspatial components, enabling quantification of spatial importance for both observations and features. A TensorFlow implementation of NSF is available from https://github.com/willtownes/nsf-paper.

Graph regularized non-negative matrix factorization with prior knowledge consistency constraint for drug–target interactions prediction

BackgroundIdentifying drug–target interactions (DTIs) plays a key role in drug development. Traditional wet experiments to identify DTIs are expensive and time consuming. Effective computational methods to predict DTIs are useful to narrow the searching scope of potential drugs and speed up the process of drug discovery. There are a variety of non-negativity matrix factorization based methods to predict DTIs, but the convergence of the algorithms used in the matrix factorization are often overlooked and the results can be further improved.ResultsIn order to predict DTIs more accurately and quickly, we propose an alternating direction algorithm to solve graph regularized non-negative matrix factorization with prior knowledge consistency constraint (ADA-GRMFC). Based on known DTIs, drug chemical structures and target sequences, ADA-GRMFC at first constructs a DTI matrix, a drug similarity matrix and a target similarity matrix. Then DTI prediction is modeled as the non-negative factorization of the DTI matrix with graph dual regularization terms and a prior knowledge consistency constraint. The graph dual regularization terms are used to integrate the information from the drug similarity matrix and the target similarity matrix, and the prior knowledge consistency constraint is used to ensure the matrix decomposition result should be consistent with the prior knowledge of known DTIs. Finally, an alternating direction algorithm is used to solve the matrix factorization. Furthermore, we prove that the algorithm can converge to a stationary point. Extensive experimental results of 10-fold cross-validation show that ADA-GRMFC has better performance than other state-of-the-art methods. In the case study, ADA-GRMFC is also used to predict the targets interacting with the drug olanzapine, and all of the 10 highest-scoring targets have been accurately predicted. In predicting drug interactions with target estrogen receptors alpha, 17 of the 20 highest-scoring drugs have been validated.

Exponential Graph Regularized Non-Negative Low-Rank Factorization for Robust Latent Representation

Non-negative matrix factorization (NMF) is a fundamental theory that has received much attention and is widely used in image engineering, pattern recognition and other fields. However, the classical NMF has limitations such as only focusing on local information, sensitivity to noise and small sample size (SSS) problems. Therefore, how to develop the NMF to improve the performance and robustness of the algorithm is a worthy challenge. Based on the bottlenecks above, we propose an exponential graph regularization non-negative low-rank factorization algorithm (EGNLRF) combining sparseness, low rank and matrix exponential. Firstly, based on the assumption that the data is corroded, we decompose a given raw data item with a data error fitting noise matrix, applying a low-rank constraint to the denoising data. Then, we perform a non-negative factorization on the resulting low-rank matrix, from which we derive the low-dimensional representation of the original matrix. Finally, we use the low-dimensional representation for graph embedding to maintain the geometry between samples. The graph embedding terms are matrix exponentiated to cope with SSS problems and nearest neighbor sensitivity. The above three steps will be incorporated into a joint framework to validate and optimize each other; therefore, we can learn latent data representations that are undisturbed by noise and preserve the local structure of known samples. We conducted simulation experiments on different datasets and verified the effectiveness of the algorithm by comparing the proposed with the lasting ones related to NMF, low rank and graph embedding.

A transfer framework based on co-matrix decomposition for undirected, high-dimensional and sparse networks

Undirected, high-dimensional and sparse network is often encountered in industrial engineering and described by symmetric, high-dimensional and sparse (SHiDS) matrix for academic investigation. Latent factor (LF) models have shown an arguably promising yet whelming capability of extracting knowledge from so little known information in the SHiDS matrix. However, most of the LF models in the form of matrix factorization (a.k.a. matrix decomposition) focus exclusively on a single domain and struggle to meet the requirements of symmetry and non-negativity of the SHiDS matrix. To address these issues, a symmetric and non-negative collective factorization (SNCF) transfer framework in the form of collective-matrix (co-matrix) decomposition is proposed. SNCF transfers a great deal of knowledge from the auxiliary domain to mitigate the sparsity of the target domain. To solve the SNCF efficiently and guarantee the symmetry and non-negativity of the target matrix, a collective batch-based non-negative optimization approach (SNCF-CBN) is put forward. Experimental results demonstrate the effectiveness of SNCF-CBN by comparing it with several state-of-the-art models on public datasets, including Protein Networks, Sonar Mines and Sonar Rocks datasets with different sparsities in cross-domain.

A novel update rule of HALS algorithm for nonnegative matrix factorization and Zangwill’s global convergence

Nonnegative Matrix Factorization (NMF) has attracted a great deal of attention as an effective technique for dimensionality reduction of large-scale nonnegative data. Given a nonnegative matrix, NMF aims to obtain two low-rank nonnegative factor matrices by solving a constrained optimization problem. The Hierarchical Alternating Least Squares (HALS) algorithm is a well-known and widely-used iterative method for solving such optimization problems. However, the original update rule used in the HALS algorithm is not well defined. In this paper, we propose a novel well-defined update rule of the HALS algorithm, and prove its global convergence in the sense of Zangwill. Unlike conventional globally-convergent update rules, the proposed one allows variables to take the value of zero and hence can obtain sparse factor matrices. We also present two stopping conditions that guarantee the finite termination of the HALS algorithm. The practical usefulness of the proposed update rule is shown through experiments using real-world datasets.

Graph Convolutional Sparse Subspace Coclustering With Nonnegative Orthogonal Factorization for Large Hyperspectral Images

Sparse subspace clustering (SSC) is a representative data clustering paradigm that has been broadly applied in the unsupervised classification of hyperspectral images (HSIs). Existing SSC methods usually produce a subspace affinity matrix between representations of hyperspectral pixels first, followed by spectral clustering for the affinity matrix. To this end, the separated framework fails to exploit the dualities contained in both features and pixels or higher order entities at the same time, and thus, it is difficult to compute coclusters simultaneously. In addition, SSC methods often require expensive computational consumption and memory capacity to approximate the spectral decomposition of the affinity matrix, thus hindering the applicability of SSC methods for large HSIs. To overcome these limitations, we propose a novel graph convolutional sparse subspace coclustering (GCSSC) model with nonnegative orthogonal factorization for large HSIs in which affinity matrix learning and spectral coclustering are integrated into a unified optimizing model to obtain the optimal clustering results. Specifically, to form a more compact self-representation, the superpixel-based adaptive dictionary construction strategy is proposed instead of the global dictionary to precisely represent the pixels. To explore the spatial–contextual and spectral neighboring characteristics between dictionary atoms, graph convolution is incorporated into the dictionary atoms to aggregate the local neighborhood information, and the affinity matrix in the proposed coclustering framework is constructed under a joint sparsity constrained representation model. To reduce high computational consumption and memory capacity, a nonnegative orthogonal factorization constraint is proposed to offer an alternative spectral clustering for hyperspectral pixels and dictionary atoms simultaneously. The clustering performance of the proposed method is evaluated for three classical HSIs, and the experimental results illustrate that the proposed method is memory and computationally efficient and outperforms the state-of-the-art HSI clustering methods.

Non-Negative Minimum Volume Factorization (NMVF) for Hyperspectral Images (HSI) Unmixing: A Hybrid Approach

Spectral unmixing is essential for exploitation of remotely sensed data of Hyperspectral Images (HSI). It amounts to the identification of a position of spectral signatures that are pure and therefore called end members and their matching fractional, draft rules abundances for every pixel in HSI. This paper aims to unmix hyperspectral data using the minimal volume method of elementary scrutiny. Moreover, the problem of optimization is solved by the implementation of the sequence of small problems that are constrained quadratically. The hard constraint in the final step for the abundance fraction is then replaced with a loss function of hinge type that accounts for outliners and noise. Existing algorithms focus on estimating the endmembers (Ems) enumeration in a sight, discerning of spectral signs of EMs, besides assessment of fractional profusion for every EM in every pixel of a sight. Nevertheless, all the stages are performed by only a few algorithms in the process of hyperspectral unmixing. Therefore, the Non-negative Minimum Volume Factorization (NMVF) algorithm is further extended by fusing it with the nonnegative matrix of robust collaborative factorization that aims to perform all the three unmixing chain steps for hyperspectral images. The major contributions of this article are in this manner: (A) it performs Simplex analysis of minimum volume for hyperspectral images with unsupervised linear unmixing is employed. (B) The simplex analysis method is configured with an exaggerated form of the elementary which is delivered by vertical component analysis (VCA). (C) The inflating factor is chosen carefully inactivating the constraints in a large majority for relating to the source fractions abundance that speeds up the algorithm. (D) The final step is making simplex analysis method robust to outliners as well as noise that replaces the profusion element positivity hard restraint by a hinge kind soft restraint, preserving the local minima having good quality. (E) The matrix factorization method is applied that is capable of performing the three major phases of the hyperspectral separation sequence. The anticipated approach can find application in a scenario where the end members are known in advance, however, it assumes that the endmembers count is corresponding to an overestimated value. The proposed method is different from other conventional methods as it begins with the overestimation of the count of endmembers wherein removing the endmembers that are redundant by the means of collaborative regularization. As demonstrated by the experimental results, proposed approach yields competitive performance comparable with widely used methods.

Colorectal cancer in Crohn's disease evaluated with genes belonging to fibroblasts of the intestinal mucosa selected by NMF

Crohn's disease (CD) is a type of chronic, inflammatory bowel disease (IBD) which affects any part of the gastrointestinal tract. This study aims to understand the mechanism which activate mucosal fibroblasts in the microenvironment of the colon in CD and colorectal carcinomas and to extract fibroblasts phenotypes via a novel framework based on non-negative factorization of matrix (NMF). The results identify a fibroblast phenotype characterized by intense pro-inflammatory activity ensured by the presence of genes belonging to the APOBEC1 family, such as APOBEC3F and APOBEC3G. These results demonstrated that there is a difference in fibroblast response in producing a pro-tumorigenic effect in CD. The different activation mechanisms could represent useful biomarkers in controlling CD development without generalizing its significance as IBD.

A machine learning approach for the factorization of psychometric data with application to the Delis Kaplan Executive Function System

While a replicability crisis has shaken psychological sciences, the replicability of multivariate approaches for psychometric data factorization has received little attention. In particular, Exploratory Factor Analysis (EFA) is frequently promoted as the gold standard in psychological sciences. However, the application of EFA to executive functioning, a core concept in psychology and cognitive neuroscience, has led to divergent conceptual models. This heterogeneity severely limits the generalizability and replicability of findings. To tackle this issue, in this study, we propose to capitalize on a machine learning approach, OPNMF (Orthonormal Projective Non-Negative Factorization), and leverage internal cross-validation to promote generalizability to an independent dataset. We examined its application on the scores of 334 adults at the Delis–Kaplan Executive Function System (D-KEFS), while comparing to standard EFA and Principal Component Analysis (PCA). We further evaluated the replicability of the derived factorization across specific gender and age subsamples. Overall, OPNMF and PCA both converge towards a two-factor model as the best data-fit model. The derived factorization suggests a division between low-level and high-level executive functioning measures, a model further supported in subsamples. In contrast, EFA, highlighted a five-factor model which reflects the segregation of the D-KEFS battery into its main tasks while still clustering higher-level tasks together. However, this model was poorly supported in the subsamples. Thus, the parsimonious two-factors model revealed by OPNMF encompasses the more complex factorization yielded by EFA while enjoying higher generalizability. Hence, OPNMF provides a conceptually meaningful, technically robust, and generalizable factorization for psychometric tools.

Graphical interpretation of non-negative factorization expecting bio-Raman research

Non-negative matrix factorization (NMF) has been frequently used in research on live cells, biomolecules, and tissues (bio-Raman research) to disentangle the complicated and large sized data. A stagnation is that NMF does not provide unique decomposition, depending on initial settings; that is, NMF returns non-negative spectral components close to the truths, but solely giving several possibilities. In this research, we visualized possible ranges of NMF in binary component system. The mechanism of NMF became more clarified and opened new viewpoints.