Kernel Nonnegative Matrix Factorization Research Articles

A Novel High-Dimensional Kernel Joint Non-Negative Matrix Factorization With Multimodal Information for Lung Cancer Study.

Judging and identifying biological activities and biomarkers inside tissues from imaging features of diseases is challenging, so correlating pathological image data with genes inside organisms is of great significance for clinical diagnosis. This paper proposes a high-dimensional kernel non-negative matrix factorization (NMF) method based on muti-modal information fusion. This algorithm can project RNA gene expression data and pathological images (WSI) into a common feature space, where the heterogeneous variables with the largest coefficient in the same projection direction form a co-module. In addition, the miRNA-mRNA and miRNA-lncRNA interaction networks in the ceRNA network are added to the algorithm as a priori information to explore the relationship between the images and the internal activities of the gene. Furthermore, the radial basis kernel function is used to calculate the feature proportion between different kinds of genes mapped in the high-dimensional feature space and projected into the common feature space to explore the gene interaction in the high-dimensional situation. The original feature matrix is regularized to improve biological correlation, and the feature factors are sparse by orthogonal constraints to reduce redundancy. Experimental results show that the proposed NMF method is better than the traditional NMF method in stability, decomposition accuracy, and robustness. Through data analysis applied to lung cancer, genes related to tissue morphology are found, such as COL7A1, CENPF and BIRC5. In addition, gene pairs with a correlation degree exceeding 0.8 are found, and potential biomarkers of significant correlation with survival are obtained such as CAPN8. It has potential application value for the clinical diagnosis of lung cancer.

Nonnegative matrix factorization with combined kernels for small data representation

Kernel nonnegative matrix factorization (KNMF) has emerged as a promising nonlinear data representation method, especially for applications with small sample sizes. Existing methods are usually based on a single kernel function, representing samples by the global or local features learned by the matrix factorization algorithm. In this paper, a combined kernel method is proposed and applied to data representation for small-sample face recognition in particular. Based on the combined kernel, which is a linear combination of the fractional power inner-product kernel and a newly defined Gaussian-type kernel, the new KNMF method is able to extract both global and local nonlinear features from the inputs. An efficient gradient decent algorithm is derived to solve the combined kernel nonnegative matrix factorization (CKNMF) problem, and a rigorous convergence proof is presented. The proposed method is experimentally evaluated on small image datasets, and the results demonstrate its superior performance than the state-of-the-art KNMF methods.

Making a Case for Application of the Unsupervised PCA Algorithm for Simultaneous and Proportional Myoelectric Intention Estimation of Individual Fingers

This article makes a case for the application of the conventional principal component analysis (PCA) unsupervised algorithm by modifying it with embedded high dimensional layers to achieve simultaneous and proportional intention estimation of individual fingers of the human body using surface electromyography (sEMG). Due to its linear nature, PCA in its original form is unfit for estimating intricate movements like individual fingers of the human hand. Hence, the PCA algorithm was improved using the application of nonlinear embedding of the original electromyography (EMG) data into a higher dimensional feature space. These embedding functions were used in single and multiple layers with their mapping functions evaluated from the PCA algorithm. The resulting algorithms from the application of these embedding functions are called nonlinear PCA (NLPCA) and multilayer NLPCA (ML-NLPCA). We compare these algorithms with the unsupervised algorithms of nonnegative matrix factorization (NMF), NMF with Hadamard product (NMF-HP), kernel NMF (kNMF), and kernel NMF-HP (kNMF-HP) from previous studies to draw comparisons in making our case. The models for each finger are trained blindly without any output values by the unsupervised algorithms to solve the individual finger estimation problem using an in- house eight-channel EMG acquisition system. It will be seen that ML-NLPCA produces the most effective results to reflect the movement of a human hand by generating simultaneous and proportional control (SPC) commands using a robot hand.

A novel general kernel-based non-negative matrix factorisation approach for face recognition

Kernel-based non-negative matrix factorisation (KNMF) is a promising nonlinear approach for image data representation using non-negative features. However, most of the KNMF algorithms are developed via a specific kernel function and thus fail to adopt other kinds of kernels. Also, they have to learn pre-image inaccurately that may influence the reliability of the method. To address these problems of KNMF, this paper proposes a novel general kernel-based non-negative matrix factorisation (GKBNNMF) method. It not only avoids pre-image learning but also is suitable for any kernel functions as well. We assume that the mapped basis images fall within the cone spanned by the mapped training data, allowing us to use arbitrary kernel function in the algorithm. The symmetric NMF strategy is exploited on kernel matrix to establish our general kernel NMF model. The proposed algorithm is proven to be convergent. The facial image datasets are selected to evaluate the performance of our method. Compared with some state-of-the-art approaches, the experimental results demonstrate that our proposed is both effective and robust.

A novel industrial process fault monitoring method based on kernel robust non-negative matrix factorization

Industrial processes are characterized by large amounts of nonlinear and noisy data, which pose a critical challenge to the accuracy and rapidity of fault detection. In this paper, an industrial process fault monitoring method based on kernel robust non-negative matrix factorization is proposed. This method uses the kernel technique to map the nonlinear data to high-dimensional linear space, where the local features of the sample will be extracted by the non-negative matrix factorization (NMF) method. However, noise signals will inevitably be mixed. Therefore, a sparse error matrix is introduced to isolate fault and noise information. Finally, a new monitoring statistics and a fault detection framework are constructed. On the TE platform, the algorithm proposed in this paper is compared with kernel principal component analysis and kernel NMF methods in nonlinear experiments and robustness experiments through two performance indicators: fault detection rate and fault delay. The results prove the effectiveness of the algorithm in this paper.

Feature Extraction Using Sparse Kernel Non-Negative Matrix Factorization for Rolling Element Bearing Diagnosis.

For early fault detection of a bearing, the localized defect generally brings a complex vibration signal, so it is difficult to detect the periodic transient characteristics from the signal spectrum using conventional bearing fault diagnosis methods. Therefore, many matrix analysis technologies, such as singular value decomposition (SVD) and reweighted SVD (RSVD), were proposed recently to solve this problem. However, such technologies also face failure in bearing fault detection due to the poor interpretability of the obtained eigenvector. Non-negative Matrix Factorization (NMF), as a part-based representation algorithm, can extract low-rank basis spaces with natural sparsity from the time–frequency representation. It performs excellent interpretability of the factor matrices due to its non-negative constraints. By this virtue, NMF can extract the fault feature by separating the frequency bands of resonance regions from the amplitude spectrogram automatically. In this paper, a new feature extraction method based on sparse kernel NMF (KNMF) was proposed to extract the fault features from the amplitude spectrogram in greater depth. By decomposing the amplitude spectrogram using the kernel-based NMF model with L1 regularization, sparser spectral bases can be obtained. Using KNMF with the linear kernel function, the time–frequency distribution of the vibration signal can be decomposed into a subspace with different frequency bands. Thus, we can extract the fault features, a series of periodic impulses, from the decomposed subspace according to the sparse frequency bands in the spectral bases. As a result, the proposed method shows a very high performance in extracting fault features, which is verified by experimental investigations and benchmarked by the Fast Kurtogram, SVD and NMF-based methods.

Hyperspectral Images Unmixing Based on Abundance Constrained Multi-Layer KNMF

Due to the low spatial resolution of the sensors, the hyperspectral images contain mixed pixels. The purpose of hyperspectral unmixing is to decompose the mixed pixels into a series of endmembers and abundance fractions. In order to improve the performance of the nonlinear unmixing algorithm for hyperspectral images, a nonlinear unmixing method, i.e., abundance constrained multi-layer kernel non-negative matrix factorization (AC-MLKNMF), is presented. Firstly, MLKNMF is presented to iteratively decompose the mixed pixels into a multi-layer structure, and then AC-MLKNMF is presented based on MLKNMF by adding the sparseness constraint and total variation regularization to the abundance to characterize the sparseness and the piecewise smooth structure of the abundance maps according to the spatial distribution characteristics of the actual ground-objects. Experimental results on synthetic and real datasets show that the proposed AC-MLKNMF can improve the hyperspectral unmixing accuracy compared with single-layer KNMF, and it is also superior to multi-layer non-negative matrix factorization, KNMF without pure pixels, kernel sparse NMF, MLKNMF.

Elastic net regularized kernel non-negative matrix factorization algorithm for clustering guided image representation

Due to the capacity of data representation, non-negative matrix factorization has been investigated widely by introducing a variety of constraints. The general processing of non-negative matrix factorization for image clustering consists of two steps: (i) achieving the r-dimensional non-negative image representations, where the rank r is set to the expected number of clusters; (ii) adopting the traditional clustering techniques to accomplish the clustering task. Nevertheless, the previous non-negative matrix factorization variants derive image representations from the original space which cannot handle the nonlinear structure of images. This paper focuses on the existing issues and proposes an elastic net regularized kernel non-negative matrix factorization algorithm for clustering guided image representation. In order to explore the nonlinear relations of images, this paper uses the kernel trick to extend original non-negative matrix factorization. A self-organized graph and elastic net regularization are incorporated into the proposed objective of kernel non-negative matrix factorization, besides, the rank is allowed to be larger than the expected number of clusters. By doing so, the graph defined in the feature space is more qualified to represent the intrinsic structure of images. As an accompanying advantage, the clusters of images can be determined using the graph directly without using the two-step trick. According to the proposed alternating update algorithm for solving the optimization problem, the image representation and clustering result can be obtained simultaneously. Extensive experiments on challenging data sets demonstrate the effectiveness of the proposed algorithm compared with the prominent non-negative matrix factorization variants for image clustering.

A kernel non-negative matrix factorization framework for single cell clustering

The emergence of single-cell RNA-sequencing is ideally placed to unravel cellular heterogeneity in biological systems, an extremely challenging problem in single cell RNA-sequencing studies. However, most current computational approaches lack the sensitivity to reliably detect nonlinear gene-gene relationships masked by dropout events. We proposed a kernel non-negative matrix factorization framework for detecting nonlinear relationships among genes, where the new kernel is developed using kernel tricks on cellular differentiability correlation. The newly constructed kernel not only provides a description on the gene-gene relationship, but also helps to build a new low-dimensional representation on the original data. Besides, we developed an efficient method for determining the optimal cluster number within each data set with the usage of Diffusion Maps. The proposed algorithm is further compared with representative algorithms: SC3 and several other state-of-the-art clustering methods, on several benchmark or real scRNA-Seq datasets using internal criteria (clustering number accuracy) and external criteria (Adjusted rand index and Normalized mutual information) to show effectiveness of our method.

A three-stage method for batch-based incremental nonnegative matrix factorization

The main issue in incremental nonnegative matrix factorization (INMF) is how to update base matrix and coefficient matrix. The re-training scheme(RT-NMF) and the scheme proposed by Bucak and Gunsel(BG-INMF) are two common methods. However, both of them have problems in balancing root mean square error(RMSE) and time cost when incremental samples appear in a batch form. In this paper, a three-stage method(3S-INMF) is proposed to derive a good balance between RMSE and time cost. In the first stage, only the coefficient matrix of incremental samples is updated while the base matrix and the coefficient matrix of old samples are fixed. If the RMSE does not meet the required precision after this stage, the second stage, i.e. BG-INMF, is carried out. In the second stage, the base matrix and the coefficient matrix of incremental samples are updated alternatively while the coefficient matrix of old samples is fixed. If the RMSE still does not meet with the required precision after BG-INMF, the coefficient matrix of old samples will be updated in the third stage while the base matrix and the coefficient matrix of incremental samples are fixed. In the three consecutive stages, the initial values of base matrix and coefficient matrix in each stage are the corresponding output values in the previous stage. In addition, extensive experiments on the three popular datasets show that 3S-INMF obtains the best balance between RMSE and time cost compared with RT-NMF and BG-INMF. Furthermore, the 3S-INMF is extended to graph nonnegative matrix factorization(GNMF) and kernel nonnegative matrix factorization(KNMF), which also has a superior performance examined by further experiments.

Simultaneous fault detection and isolation using semi‐supervised kernel nonnegative matrix factorization

This paper presents a monitoring approach for nonlinear processes based on a new semi‐supervised kernel nonnegative matrix factorization (SKNMF). Different from the existing nonnegative matrix factorization (NMF) and kernel nonnegative matrix factorization (KNMF), SKNMF is a semi‐supervised matrix factorization algorithm, which takes advantages of both labelled and unlabelled samples to improve algorithm performance. Labelled samples refer to the samples whose memberships are already known, while unlabelled samples are a set of samples whose memberships are unknown. In fact, both NMF and KNMF are unsupervised algorithms, and they cannot make full use of labelled samples to improve algorithm performance. More importantly, we explain the reasons why labelled samples can improve algorithm performance even if the amount of labelled samples is small. Last but not least, SKNMF induces a simultaneous fault detection and isolation scheme for online processes monitoring. Case studies of a numerical example and a penicillin fermentation process (PFP) demonstrate that the proposed process monitoring approaches outperform the existing process monitoring approaches.

Block kernel nonnegative matrix factorization for face recognition

Nonnegative matrix factorization (NMF) is a linear approach for extracting localized feature of facial image. However, NMF may fail to process the data points that are nonlinearly separable. The kernel extension of NMF, named kernel NMF (KNMF), can model the nonlinear relationship among data points and extract nonlinear features of facial images. KNMF is an unsupervised method, thus it does not utilize the supervision information. Moreover, the extracted features by KNMF are not sparse enough. To overcome these limitations, this paper proposes a supervised KNMF called block kernel NMF (BKNMF). A novel objective function is established by incorporating the intra-class information. The algorithm is derived by making use of the block strategy and kernel theory. Our BKNMF has some merits for face recognition, such as highly sparse features and orthogonal features from different classes. We theoretically analyze the convergence of the proposed BKNMF. Compared with some state-of-the-art methods, our BKNMF achieves superior performance in face recognition.

Network Embedding Using Semi-Supervised Kernel Nonnegative Matrix Factorization

Network embedding, aiming to learn low-dimensional representations of nodes in networks, is very useful for many vector-based machine learning algorithms and has become a hot research topic in network analysis. Although many methods for network embedding have been proposed before, most of them are unsupervised, which ignores the role of prior information available in the network. In this paper, we propose a novel method for network embedding using semi-supervised kernel nonnegative matrix factorization (SSKNMF), which can incorporate prior information and thus to learn more useful features from the network through introducing kernel methodology. Besides, it can improve robustness against noises by using the objective function based on L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ,1 norm. Efficient iterative update rules are derived to resolve the network embedding model using the SSKNMF, and the convergence of these rules are strictly proved from the perspective of mathematics. The results from extensive experiments on several real-world networks show that our proposed algorithm is effective and has better performance than the existing representative methods.

Face recognition using nonnegative matrix factorization with fractional power inner product kernel

Kernel nonnegative matrix factorization (KNMF) algorithms have been widely used to extract features for face recognition. The choice of kernel function is vital to facial feature extraction. The polynomial kernel function has been commonly used in KNMF. The power of the polynomial kernel is required to be a positive integer, thereby ensuring that the kernel generates a positive semi-definite matrix. In this paper, we investigate a new type of inner-product kernel that has a fractional power. The new kernel offers us flexibility in data representation as the power can be any positive real number. Based on the fractional power inner-product kernel, we present a novel KNMF algorithm called fractional power inner-product KNMF (FPKNMF). The FPKNMF algorithm is theoretically and experimentally validated to be convergent. The experimental results confirm that our algorithm exhibits a performance superior to the state-of-the-art methods in terms of facial representation and recognition accuracy.

Hyperspectral Unmixing Based on Incremental Kernel Nonnegative Matrix Factorization

Kernel nonnegative matrix factorization (KNMF) is an extension of NMF designed to capture nonlinear dependence features in data matrix through kernel functions. In KNMF, the size of the kernel matrices is closely associated with the input data matrix, of which the calculation consumes a large amount of memory and computing resource. When applied on large-scale hyperspectral data, KNMF often meets the bottleneck of memory and may cause the overflow of memory. And when dealing with dynamically acquired data, KNMF requires recomputation of the whole data set when newly acquired data arrived, which produces huge memory and computing resource requirements. To reduce the usage of memory and improve the computational efficiency when applying KNMF on large scale and dynamic hyperspectral data, we extend KNMF by introducing partition matrix theory and considering the relationships among dividing blocks. The decomposition results of hyperspectral data are derived from much smaller scale matrices containing the formerly achieved results and the newly data blocks incrementally. In this paper, we propose an incremental KNMF (IKNMF) to reduce the computing requirements for large-scale data in hyperspectral unmixing. An improved IKNMF (IIKNMF) is also proposed to further improve the abundance results of IKNMF. Experiments are conducted on both synthetic and real hyperspectral data sets. The experimental results demonstrate that the proposed methods can effectively save memory resources without degrading the unmixing performance and the proposed IIKNMF can achieve better abundance results than IKNMF and KNMF.

Speaker conversion using kernel non-negative matrix factorization

Voice conversion (VC) based on Gaussian mixture model (GMM) is the most classic and common method which converts the source spectrum to target spectrum. However this method is prone to over-fitting because of its frame-by-frame conversion. The VC with non-negative matrix factorization (NMF) is presented in this paper, which can keep spectrum from over-fitting by adjusting the size of basis vector (dictionary). In order to realize the non-linear mapping better, kernel NMF (KNMF) is adopted to achieve spectrum mapping. In addition, to increase the accuracy of conversion, KNMF combined with GMM (GKNMF) is also introduced into VC. In the end, KNMF, GKNMF, GMM, principal component regression (PCR), PCR combined with GMM (GPCR), partial least square regression (PLSR), NMF correlation-based frequency warping (NMF-CFW) and deep neural network (DNN) methods are compared with each other. The proposed GKNMF gets better performance in both objective evaluation and subjective evaluation.

Nonlinear process monitoring using kernel nonnegative matrix factorization

This paper focuses on developing an advanced nonlinear process monitoring technique involving fault detection and identification methods. The new monitoring methods are proposed based on two nonlinear matrix factorization algorithms. Both factorizations use the kernel method to replace lower‐dimensional nonlinearity using higher‐dimensional linearity by nonlinearly mapping the data onto a high‐dimensional linear space. In the high‐dimensional linear space, also known as feature space, the first factorization decomposes the data matrix into two low‐rank matrix products, in which the first matrix factor is restricted to being orthogonal and non‐negative leading to a good performance in the subspace approximation of the original data. In the second factorization, a matrix consisting of all types of fault samples is decomposed into two low‐rank matrix products, in which the second matrix factor is restricted to being orthogonal and non‐negative providing a clear K‐means clustering interpretation. On the basis of the above two factorizations, the corresponding fault detection and identification methods are developed. Finally, the proposed approaches are used to monitor the penicillin fermentation process (PFP), and encouraging experimental results are achieved.

Online kernel nonnegative matrix factorization

Nonnegative matrix factorization (NMF) has become a prominent signal processing and data analysis technique. To address streaming data, online methods for NMF have been introduced recently, mainly restricted to the linear model. In this paper, we propose a framework for online nonlinear NMF, where the factorization is conducted in a kernel-induced feature space. By exploring recent advances in the stochastic gradient descent and the mini-batch strategies, the proposed algorithms have a controlled computational complexity. We derive several general update rules, in additive and multiplicative strategies, and detail the case of the Gaussian kernel. The performance of the proposed framework is validated on unmixing synthetic and real hyperspectral images, comparing to state-of-the-art techniques.

EFFICIENT IMAGE RETRIEVAL IN BIG DATABASE USING MULTI FEATURE DESCRIPTOR

Image processing is processing of images by using any form of signal processing using mathematical operations for which an image or a video is input. The image processing output may be an image or a set of parameters based on the image. An image retrieval system for a large database of digital images is a computer system for searching, retrieving and browsing image and obtaining most similar images. From the retrieved image multi features like Color Histogram, HOG Transform, Local Binary feature and Gist. Each of these features need to transfer by applying Heat Kernel function to generate range with each of this feature mentioned to a range form a multiview hash and put it in hash table. Calculate hash for the image and this hash can be map for different images also. The existing method for image retrieval is Hashing technology. Hashing is used to index and retrieve items in a database because it is faster to find the items using the shorter hashed key than to find it using the original value. For most hashing strategies, the performance of retrieval vigorously relies upon the decision of the high-dimensional component descriptor. The disadvantage is Curse of dimensionality and does not search faster. The proposed methods are Multi Feature Descriptor and Multiview alignment Hashing. Multiview alignment hashing approach based on regularized kernel nonnegative matrix factorization(NMF), which can find a compact representation uncovering the hidden semantics and simultaneously respecting the joint probability distribution of data and it can easily searches the similar images in less interval of time and it is effective.. Hash within Hash technology is the future enhancement, the hash size is large partitioning the hash and obtaining the efficient result.

Supervised kernel nonnegative matrix factorization for face recognition

Nonnegative matrix factorization (NMF) is a promising algorithm for dimensionality reduction and local feature extraction. However, NMF is a linear and unsupervised method. The performance of NMF would be degraded when dealing with the complicated nonlinear distributed data, such as face images with variations of pose, illumination and facial expression. Also, the available labels could potentially improve the discriminant power of NMF. To overcome the aforementioned limitations of NMF, this paper proposes a novel supervised and nonlinear approach to enhance the classification power of NMF. By mapping the input data into a reproducing kernel Hilbert space (RKHS), we can discover the nonlinear relations between the data. This is known as the kernel methods. At the same time, we make use of discriminant analysis to force the within-class scatter small and between-class scatter large in the RKHS. It theoretically shows that the proposed approach can guarantee the non-negativity of the decomposed components and the objective function is non-increasing under the update rules. The proposed method is applied to face recognition. Compared with some state-of-the-art algorithms, experimental results demonstrate the superior performance of our method.