Kernel Clustering Research Articles

Multiple Kernel Clustering With Adaptive Multi-Scale Partition Selection

Multiple Kernel Clustering With Adaptive Multi-Scale Partition Selection

On the Consistency and Large-Scale Extension of Multiple Kernel Clustering.

Existing multiple kernel clustering (MKC) algorithms have two ubiquitous problems. From the theoretical perspective, most MKC algorithms lack sufficient theoretical analysis, especially the consistency of learned parameters, such as the kernel weights. From the practical perspective, the high complexity makes MKC unable to handle large-scale datasets. This paper tries to address the above two issues. We first make a consistency analysis of an influential MKC method named Simple Multiple Kernel k-Means (SimpleMKKM). Specifically, suppose that ∧γn are the kernel weights learned by SimpleMKKM from the training samples. We also define the expected version of SimpleMKKM and denote its solution as γ*. We establish an upper bound of ||∧γn-γ*||∞ in the order of ~O(1/√n), where n is the sample number. Based on this result, we also derive its excess clustering risk calculated by a standard clustering loss function. For the large-scale extension, we replace the eigen decomposition of SimpleMKKM with singular value decomposition (SVD). Consequently, the complexity can be decreased to O(n) such that SimpleMKKM can be implemented on large-scale datasets. We then deduce several theoretical results to verify the approximation ability of the proposed SVD-based method. The results of comprehensive experiments demonstrate the superiority of the proposed method.

TFMKC: Tuning-Free Multiple Kernel Clustering Coupled With Diverse Partition Fusion.

Clustering is a popular research pipeline in unsupervised learning to find potential groupings. As a representative paradigm in multiple kernel clustering (MKC), late fusion-based models learn a consistent partition across multiple base kernels. Despite their promising performance, a common concern is the limited representation capacity caused by the inflexible fusion mechanism. Concretely, the representations are constrained by truncated- k Eigen-decomposition (EVD) without fully exploiting potential information. An intuitive idea to alleviate this concern is to generate a set of augmented partitions and then select the optimal partition by fine-tuning. However, this is overlimited by: 1) introducing undesired hyperparameters and dataset-related consequences; 2) neglecting rich information across diverse partitions; and 3) expensive parameter-tuning costs. To address these problems, we propose transforming the challenging problem of directly determining the optimal partition (optimal parameter) into a diverse partition fusion (parameter ensemble) problem. We design a novel flexible fusion mechanism called tuning-free multiple kernel clustering coupled with diverse partition fusion (TFMKC) by reweighting diverse partitions through optimization, achieving an optimal consensus partition by integrating diverse and complementary information rather than traditional fine-tuning, and distinguishing our work from existing methods. Extensive experiments verify that TFMKC achieves competitive effectiveness and efficiency over comparison baselines. The code can be accessed at https://github.com/ZJP/TFMKC.

One-Stage Shifted Laplacian Refining for Multiple Kernel Clustering.

Graph learning can effectively characterize the similarity structure of sample pairs, hence multiple kernel clustering based on graph learning (MKC-GL) achieves promising results on nonlinear clustering tasks. However, previous methods confine to a "three-stage" scheme, that is, affinity graph learning, Laplacian construction, and clustering indicator extracting, which results in the information distortion in the step alternating. Meanwhile, the energy of Laplacian reconstruction and the necessary cluster information cannot be preserved simultaneously. To address these problems, we propose a one-stage shifted Laplacian refining (OSLR) method for multiple kernel clustering (MKC), where using the "one-stage" scheme focuses on Laplacian learning rather than traditional graph learning. Concretely, our method treats each kernel matrix as an affinity graph rather than ordinary data and constructs its corresponding Laplacian matrix in advance. Compared to the traditional Laplacian methods, we transform each Laplacian to an approximately shifted Laplacian (ASL) for refining a consensus Laplacian. Then, we project the consensus Laplacian onto a Fantope space to ensure that reconstruction information and clustering information concentrate on larger eigenvalues. Theoretically, our OSLR reduces the memory complexity and computation complexity to O(n) and O(n2) , respectively. Moreover, experimental results have shown that it outperforms state-of-the-art MKC methods on multiple benchmark datasets.

A Study of Stock Price Prediction Models Based on Kernel Clustering Localized Sliced Inverse Regression and Bayesian Model Averaging

A Study of Stock Price Prediction Models Based on Kernel Clustering Localized Sliced Inverse Regression and Bayesian Model Averaging

Join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering

AbstractImage clustering has received significant attention due to the growing importance of image recognition. Researchers have explored Riemannian manifold clustering, which is capable of capturing the non‐linear shapes found in real‐world datasets. However, the complexity of image data poses substantial challenges for modelling and feature extraction. Traditional methods such as covariance matrices and linear subspace have shown promise in image modelling, and they are still in their early stages and suffer from certain limitations. However, these include the uncertainty of representing data using only one Riemannian manifold, limited feature extraction capacity of single kernel functions, and resulting incomplete data representation and redundancy. To overcome these limitations, the authors propose a novel approach called join multiple Riemannian manifold representation and multi‐kernel non‐redundancy for image clustering (MRMNR‐MKC). It combines covariance matrices with linear subspace to represent data and applies multiple kernel functions to map the non‐linear structural data into a reproducing kernel Hilbert space, enabling linear model analysis for image clustering. Additionally, the authors use matrix‐induced regularisation to improve the clustering kernel selection process by reducing redundancy and assigning lower weights to identical kernels. Finally, the authors also conducted numerous experiments to evaluate the performance of our approach, confirming its superiority to state‐of‐the‐art methods on three benchmark datasets.

Low-Rank Kernel Tensor Learning for Incomplete Multi-View Clustering

Incomplete Multiple Kernel Clustering algorithms, which aim to learn a common latent representation from pre-constructed incomplete multiple kernels from the original data, followed by k-means for clustering. They have attracted intensive attention due to their high computational efficiency. However, our observation reveals that the imputation of these approaches for each kernel ignores the influence of other incomplete kernels. In light of this, we present a novel method called Low-Rank Kernel Tensor Learning for Incomplete Multiple Views Clustering (LRKT-IMVC) to address the above issue. Specifically, LRKT-IMVC first introduces the concept of kernel tensor to explore the inter-view correlations, and then the low-rank kernel tensor constraint is used to further capture the consistency information to impute missing kernel elements, thereby improving the quality of clustering. Moreover, we carefully design an alternative optimization method with promising convergence to solve the resulting optimization problem. The proposed method is compared with recent advances in experiments with different missing ratios on seven well-known datasets, demonstrating its effectiveness and the advantages of the proposed interpolation method.

Incomplete Multiple Kernel Alignment Maximization for Clustering.

Multiple kernel alignment (MKA) maximization criterion has been widely applied into multiple kernel clustering (MKC) and many variants have been recently developed. Though demonstrating superior clustering performance in various applications, it is observed that none of them can effectively handle incomplete MKC, where parts or all of the pre-specified base kernel matrices are incomplete. To address this issue, we propose to integrate the imputation of incomplete kernel matrices and MKA maximization for clustering into a unified learning framework. The clustering of MKA maximization guides the imputation of incomplete kernel elements, and the completed kernel matrices are in turn combined to conduct the subsequent MKC. These two procedures are alternately performed until convergence. By this way, the imputation and MKC processes are seamlessly connected, with the aim to achieve better clustering performance. Besides theoretically analyzing the clustering generalization error bound, we empirically evaluate the clustering performance on several multiple kernel learning (MKL) benchmark datasets, and the results indicate the superiority of our algorithm over existing state-of-the-art counterparts. Our codes and data are publicly available at https://xinwangliu.github.io/.

Local kernels based graph learning for multiple kernel clustering

Multiple kernel clustering (MKC) has been extensively studied in recent years. The focus of MKC is how to explore the information of base kernels. Although existing methods have promising leaning abilities, they ignore the intrinsic local structure contained in base kernels, which may negatively affect their performances. To address the above problem, a novel method, termed as consensus graph learning based on local kernels (CGLLK), is introduced. CGLLK is based on the partitions extracted by kernel k-means. Specifically, we first design a simple yet effective scheme to construct the local kernels of base kernels and then a consensus graph is applied to capture the complementary information contained in the extracted partitions of local kernels. CGLLK also considers the prior knowledge existing in base kernels. Since the partitions of local kernels and the learning stage of the consensus graph contain useful information for each other, the two processes are optimized jointly. Extensive experiments on some popular datasets are carried out to verify the effectiveness of the proposed method. Experimental results illustrate that CGLLK is much more competitive than the state-of-the-art algorithms.

Local Sample-Weighted Multiple Kernel Clustering With Consensus Discriminative Graph.

Multiple kernel clustering (MKC) is committed to achieving optimal information fusion from a set of base kernels. Constructing precise and local kernel matrices is proven to be of vital significance in applications since the unreliable distant-distance similarity estimation would degrade clustering performance. Although existing localized MKC algorithms exhibit improved performance compared with globally designed competitors, most of them widely adopt the KNN mechanism to localize kernel matrix by accounting for τ -nearest neighbors. However, such a coarse manner follows an unreasonable strategy that the ranking importance of different neighbors is equal, which is impractical in applications. To alleviate such problems, this article proposes a novel local sample-weighted MKC (LSWMKC) model. We first construct a consensus discriminative affinity graph in kernel space, revealing the latent local structures. Furthermore, an optimal neighborhood kernel for the learned affinity graph is output with naturally sparse property and clear block diagonal structure. Moreover, LSWMKC implicitly optimizes adaptive weights on different neighbors with corresponding samples. Experimental results demonstrate that our LSWMKC possesses better local manifold representation and outperforms existing kernel or graph-based clustering algorithms. The source code of LSWMKC can be publicly accessed from https://github.com/liliangnudt/LSWMKC.

Regularized Simple Multiple Kernel k -Means With Kernel Average Alignment.

Multiple kernel clustering (MKC) aims to learn an optimal kernel to better serve for clustering from several precomputed basic kernels. Most MKC algorithms adhere to a common assumption that an optimal kernel is linearly combined by basic kernels. Based on a min-max framework, a newly proposed MKC method termed simple multiple kernel k -means (SimpleMKKM) can acquire a high-quality unified kernel. Although SimpleMKKM has achieved promising clustering performance, we observe that it cannot benefit from any prior knowledge. This would cause the learned partition matrix may seriously deviate from the expected one, especially in clustering tasks where the ground truth is absent during the learning course. To tackle this issue, we propose a novel algorithm termed regularized simple multiple kernel k -means with kernel average alignment (R-SMKKM-KAA). According to the experimental results of existing MKC algorithms, the average partition is a strong baseline to reflect true clustering. To gain knowledge from the average partition, we add the average alignment as a regularization term to prevent the learned unified partition from being far from the average partition. After that, we have designed an efficient solving algorithm to optimize the new resulting problem. In this way, both the incorporated prior knowledge and the combination of basic kernels are helpful to learn better unified partition. Consequently, the clustering performance can be significantly improved. Extensive experiments on nine common datasets have sufficiently demonstrated the effectiveness of incorporation of prior knowledge into SimpleMKKM.

Multiple kernel clustering with local kernel reconstruction and global heat diffusion

Multiple Kernel Clustering (MKC) is an effective approach for revealing nonlinear cluster structures in candidate kernels. However, existing MKC methods still face two key challenges. Firstly, the pairwise affinity in these methods is primarily determined by kernel similarity, disregarding the correlations among highly similar neighbors and resulting in redundant weight assignments and reduced clustering discriminability. Secondly, the direct utilization of affinity matrices overlooks high-order connections and introduces noise due to independent row-wise solving. To address these issues, we propose a novel local MKC method called LKRGDF. We begin by exploring affinity using the Local Kernel Reconstruction (LKR) model, reducing redundancy and enhancing clustering discriminability. Furthermore, we exploit the affinities with the Global heat kernel Diffusion (GD) procedure to capture long-range connections smoothly. The GD process acts as a low pass filter, focusing on small eigenvalues corresponding to top clusters. Finally, we integrate these smooth affinities within an auto-weighted Multiple Graph Fusion (MGF) framework to obtain a consensus graph. By assembling LKR, GD, and MGF in a sequential pipeline, our approach achieves the exploration and exploitation of local structures, gradually improving clustering performance while ensuring computational efficiency without the need for iterative steps. Extensive experiments on ten datasets demonstrate the superiority of our algorithm in terms of effectiveness and efficiency compared to state-of-the-art methods. The code for our method is publicly available at https://github.com/YanChenSCU/LKRGDF-2023.git.

Multi-cluster high performance computing method based on multimodal tensor in enterprise resource planning system

The big data representation and processing method based on multimodal tensors can achieve the fusion representation of different types of data, and perform correlation analysis on the fusion data through multimodal tensor decomposition, providing accurate data services for practical applications. However, in the process of big data analysis and processing based on multimodal tensors, there are still some urgent problems to be solved, such as repeated calculations caused by streaming data, time-consuming calculations caused by large scale, ultra-high energy consumption caused by complex calculations, and poor analysis results caused by complex relationships. This article uses big data tools Hadoop and Spark to build a data mining platform to analyze and study the data of traditional manufacturing enterprises' enterprise resource planning (ERP) systems. In order to improve the accuracy of data analysis, the Stacking fusion framework was also introduced. Furthermore, We transform the data generated by the enterprise into valuable information, making more accurate decisions for enterprise personnel. This article proposes a distributed parallel multi-mode tensor chain decomposition method. In the distributed parallel multi-mode tensor chain decomposition method, a method and theoretical proof for solving tensor chain decomposition are proposed based on the original tensor chain decomposition results and tensor chain decomposition results. The multi kernel clustering algorithm based on K-Means aims to improve the performance of kernel K-clustering by integrating a set of kernels. Deep learning model is also proposed to fuse these multilinear attributes, a clustering method based on weighted average similarity matrix and a clustering method based on optional weighted Euclidean distance were proposed. The distributed parallel multimodal method proposed in this article can effectively achieve tensor chain decomposition, thereby improving the efficiency of big data processing. © 2001 Elsevier Science. All rights reserved

Multimodal discourse analysis of English reading instruction in colleges and universities based on weighted function algorithm

Abstract This paper proposes a kernel clustering method that uses the weighted function method to address the problem of excessive similarity calculation in the kernel function clustering process. Based on function-based clustering with orthogonal basis expansion, function-based principal component analysis is used to reduce the dimensionality of function-based data and extract the first few principal components that can contain the most information of the original data. The factors of each component are assigned with characteristic weights so that the scores of the principal components replace the original functional data for clustering analysis of English reading teaching in colleges and universities. A systematic functional language framework was used to conduct multimodal discourse analysis based on clustering results. The symbolic annotation of pure language reached 14.5, and the symbolic resources of visual imagery reached 85.7 during the CB stage of English reading instruction. The number of semiotic resources was relatively small, so the complexity of the modality was lower than in other stages.

One-Step Clustering with Adaptively Local Kernels and a Neighborhood Kernel

Among the methods of multiple kernel clustering (MKC), some adopt a neighborhood kernel as the optimal kernel, and some use local base kernels to generate an optimal kernel. However, these two methods are not synthetically combined together to leverage their advantages, which affects the quality of the optimal kernel. Furthermore, most existing MKC methods require a two-step strategy to cluster, i.e., first learn an indicator matrix, then executive clustering. This does not guarantee the optimality of the final results. To overcome the above drawbacks, a one-step clustering with adaptively local kernels and a neighborhood kernel (OSC-ALK-ONK) is proposed in this paper, where the two methods are combined together to produce an optimal kernel. In particular, the neighborhood kernel improves the expression capability of the optimal kernel and enlarges its search range, and local base kernels avoid the redundancy of base kernels and promote their variety. Accordingly, the quality of the optimal kernel is enhanced. Further, a soft block diagonal (BD) regularizer is utilized to encourage the indicator matrix to be BD. It is helpful to obtain explicit clustering results directly and achieve one-step clustering, then overcome the disadvantage of the two-step strategy. In addition, extensive experiments on eight data sets and comparisons with six clustering methods show that OSC-ALK-ONK is effective.

SFTN: Fast object detection for aerial images

AbstractThe task of remote sensing image object detection in low latency scenes is of great research significance. To address the problem that the current high‐precision object detection algorithm based on a feature pyramid network is slow due to a large number of parameters and complicated computation, a fast remote sensing image object detection method based on a Single‐scale Feature Transformation Network (SFTN) is proposed. Firstly, based on the single‐scale remote sensing image features, the new channel features are quickly generated by a linear transformation of the original features and convolution kernel clustering optimization using cosine similarity; secondly, in order to obtain multi‐scale receptive fields, a parallel residual hole convolution module is designed to cover multi‐category remote sensing object scales on the feature map; finally, angle variables are introduced and optimized using angle similarity to effectively improve the object orientation accuracy. The experimental results on different datasets show that the method in this paper improves the detection speed rapidly while ensuring the accuracy of remote sensing image object detection, which is better than many remote sensing image object detection methods. The results demonstrate the reliability and robustness of the method.

Mutual structure learning for multiple kernel clustering

Multiple kernel clustering (MKC) has garnered considerable attention in recent years, aiming to obtain an optimal partition from multiple base kernels. Existing MKC methods typically focus on either learning the pairwise structure by constructing an optimal kernel from the base kernels or learning the cluster structure by integrating multiple base partitions into a consensus one for clustering. However, these previous approaches overlook the potential for mutual structure learning between the two aspects, i.e., pairwise and cluster structure, of clustering. To address this limitation, we propose a novel multiple kernel clustering method, referred to as MSL-MKC, which simultaneously learns pairwise and cluster structure to achieve an improved consensus partition matrix for clustering. Specifically, MSL-MKC extends the adaptive neighbor graph learning approach into kernel space to construct a discriminative similarity graph from multiple base kernels for pairwise structure learning. The consensus partition is formulated by aligning it with multiple base partitions for cluster structure learning. We utilize a Laplacian regularization term to preserve the pairwise structure in the consensus partition and inject the cluster structure into the discriminative similarity graph. The proposed method integrates similarity graph learning, partition fusion, and Laplacian regularization into a unified framework, optimized using an iterative algorithm. Experimental results on various benchmark datasets demonstrate the efficacy of MSL-MKC. The demo code of this work is publicly available at https://github.com/guanyuezhen/MSL-MKC.

Optimal neighborhood kernel clustering with adaptive local kernels and block diagonal property

Optimal neighborhood kernel clustering with adaptive local kernels and block diagonal property

Late Fusion Multiple Kernel Clustering With Proxy Graph Refinement.

Multiple kernel clustering (MKC) optimally utilizes a group of pre-specified base kernels to improve clustering performance. Among existing MKC algorithms, the recently proposed late fusion MKC methods demonstrate promising clustering performance in various applications and enjoy considerable computational acceleration. However, we observe that the kernel partition learning and late fusion processes are separated from each other in the existing mechanism, which may lead to suboptimal solutions and adversely affect the clustering performance. In this article, we propose a novel late fusion multiple kernel clustering with proxy graph refinement (LFMKC-PGR) framework to address these issues. First, we theoretically revisit the connection between late fusion kernel base partition and traditional spectral embedding. Based on this observation, we construct a proxy self-expressive graph from kernel base partitions. The proxy graph in return refines the individual kernel partitions and also captures partition relations in graph structure rather than simple linear transformation. We also provide theoretical connections and considerations between the proposed framework and the multiple kernel subspace clustering. An alternate algorithm with proved convergence is then developed to solve the resultant optimization problem. After that, extensive experiments are conducted on 12 multi-kernel benchmark datasets, and the results demonstrate the effectiveness of our proposed algorithm. The code of the proposed algorithm is publicly available at https://github.com/wangsiwei2010/graphlatefusion_MKC.

Simple multiple kernel k-means with kernel weight regularization

Multiple kernel clustering (MKC) aims to determine the optimal kernel from several pre-computed basic kernels. Most of MKC algorithms follow a common assumption that the optimal kernel is linearly combined by basic kernels. Based on a min–max framework, a newly proposed MKC method termed simple multiple kernel k-means can acquire a high-quality unified kernel. Although it has achieved promising clustering performance, we have observed that it cannot benefit from any regularization or prior knowledge, thus the learned kernel weight coefficients may be seriously sparse or over-selected. To tackle this issue, we have developed a new algorithm termed simple multiple kernel k-means with kernel weight regularization (SMKKM-KWR), where we introduce average coefficients to avoid too sparse kernel weights. Specially, we add the average kernel coefficients as a regularization term to prevent the learned weight coefficients being far from the average values. After that, an efficient optimization strategy is proposed to solve the new resultant problem. By this way, the unified partition can learn clustering structure from fusion information of all the kernel views, with the goal to reach better clustering performance. Extensive experiments on nine benchmark datasets and four large-scale datasets have demonstrated the effectiveness and efficiency of the proposed algorithm.