Soft Subspace Clustering Algorithm Research Articles

An Improved Soft Subspace Clustering Algorithm Based on Particle Swarm Optimization for MR Image Segmentation.

Soft subspace clustering (SSC), which analyzes high-dimensional data and applies various weights to each cluster class to assess the membership degree of each cluster to the space, has shown promising results in recent years. This method of clustering assigns distinct weights to each cluster class. By introducing spatial information, enhanced SSC algorithms improve the degree to which intraclass compactness and interclass separation are achieved. However, these algorithms are sensitive to noisy data and have a tendency to fall into local optima. In addition, the segmentation accuracy is poor because of the influence of noisy data. In this study, an SSC approach that is based on particle swarm optimization is suggested with the intention of reducing the interference caused by noisy data. The particle swarm optimization method is used to locate the best possible clustering center. Second, increasing the amount of geographical membership makes it possible to utilize the spatial information to quantify the link between different clusters in a more precise manner. In conclusion, the extended noise clustering method is implemented in order to maximize the weight. Additionally, the constraint condition of the weight is changed from the equality constraint to the boundary constraint in order to reduce the impact of noise. The methodology presented in this research works to reduce the amount of sensitivity the SSC algorithm has to noisy data. It is possible to demonstrate the efficacy of this algorithm by using photos with noise already present or by introducing noise to existing photographs. The revised SSC approach based on particle swarm optimization (PSO) is demonstrated to have superior segmentation accuracy through a number of trials; as a result, this work gives a novel method for the segmentation of noisy images.

Design of a modular neural network based on an improved soft subspace clustering algorithm

Being a commonly used way for task decomposition in modular neural network (MNN), clustering analysis is employed to decompose the complex task into several simple subtasks for learning. Recent studies mainly focus on hard clustering, but the clusters might be not sufficiently represented when the cluster boundary is ambiguous, which may degenerate the learning performance of subnetworks in MNN. To solve this problem, we design a modular neural network based on an improved soft subspace clustering (IESSC-MNN) algorithm in this study. Firstly, we propose an improved soft subspace clustering algorithm for task decomposition in MNN, which divides the original space into several interactive feature subspaces and allocates a weight item to each subspace to describe the contribution of the subtasks at the same time. Secondly, each RBF subnetwork is adaptively constructed using a structure growing strategy, and all subnetworks learning the corresponding subtask in parallel. Finally, all subnetworks’ outputs are integrated by weighted summation using the contribution weight of subnetworks. The simulation results of the proposed model on five benchmark data and a practical dataset indicate that IESSC-MNN improves the modeling accuracy and generalization performance with a simple structure when compared with other MNNs.

Fuzzy clustering algorithms with distance metric learning and entropy regularization

Clustering has been used in various fields, such as image processing, data mining, pattern recognition, and statistical analysis. Generally, clustering algorithms consider all variables equally relevant or not correlated. Nevertheless, the pattern of data samples in the multidimensional space can be geometrically complicated, e.g., clusters may exist in different subsets of features. In this regard, new soft subspace clustering algorithms have been proposed, in which the correlation and relevance of variables are considered to improve their performance. Since regularization-based methods are robust for initializations, the approaches proposed introduce an entropy regularization term for controlling the membership degree of the objects. Such regularizations are popular due to high performance in large-scale data clustering and low computational complexity. These three-step iterative algorithms provide a fuzzy partition, a representative for each cluster, and the relevance weight of the variables or their correlation by minimizing a suitable objective function. Several experiments on synthetic and real datasets, including their application to the segmentation of noisy image textures, demonstrate the usefulness of the proposed clustering methods.

Self-organizing subspace clustering for high-dimensional and multi-view data

A surge in the availability of data from multiple sources and modalities is correlated with advances in how to obtain, compress, store, transfer, and process large amounts of complex high-dimensional data. The clustering challenge increases with the growth of data dimensionality which decreases the discriminate power of the distance metrics. Subspace clustering aims to group data drawn from a union of subspaces. In such a way, there is a large number of state-of-the-art approaches and we divide them into families regarding the method used in the clustering. We introduce a soft subspace clustering algorithm, a Self-organizing Map (SOM) with a time-varying structure, to cluster data without any prior knowledge of the number of categories or of the neural network topology, both determined during the training process. The model also assigns proper relevancies (weights) to different dimensions, capturing from the learning process the influence of each dimension on uncovering clusters. We employ a number of real-world datasets to validate the model. This algorithm presents a competitive performance in a diverse range of contexts among them data mining, gene expression, multi-view, computer vision and text clustering problems which include high-dimensional data. Extensive experiments suggest that our method very often outperforms the state-of-the-art approaches in all types of problems considered.

View-collaborative fuzzy soft subspace clustering for automatic medical image segmentation

With the rapid development of medical imaging methodologies, such as magnetic resonance (MR) and positron emission tomography (PET)/MR, various types of MR images, which are acquired using inconsistent MR pulse sequences on the same patient, have been applied in medical-image-based diagnoses. A feature map extracted from an MR image describes the patient’s condition from one perspective. By effectively using all feature maps from various MR images, it is possible to completely describe the intrinsic characteristics of the patient’s condition to facilitate a diagnosis. Facing such a scenario, classic machine learning algorithms typically stack these feature maps for unified processing and do not explore the importance of each feature within a single feature map or the relationships among feature maps. To address these challenges, both multiview and subspace learning scenarios are considered in this study, and the multiview collaboration-based fuzzy soft subspace clustering (MVC-FSSC) algorithm is proposed. The MVC-FSSC algorithm not only strives to exploit the agreement of decisions across all views via collaborative learning but also strives to utilize the soft subspace-based weighting mechanism to automatically evaluate the contribution of each dimensional feature to each estimated cluster within a single view. Our experimental results indicate that the proposed MVC-FSSC algorithm can effectively explore the collaborative relations among all views and the importance of features in their respective views. Additionally, our MVC-FSSC method has substantial advantages over traditional clustering algorithms in MR image segmentation. Applying the MVC-FSSC algorithm to five patients’ MR images, the average mean absolute prediction deviation (MAPD) is 98.62 ± 8.34, which is significantly better than the score of 131.90 ± 16.03 that was obtained using the collaborative fuzzy k-means (CO-FKM) algorithm and the score of 128.87 ± 11.32 that was obtained using the quadratic weights and Gini-Simpson diversity-based fuzzy clustering (QWGSD-FC) algorithm.

Novel evolutionary multi-objective soft subspace clustering algorithm for credit risk assessment

The financial liberalization and globalization have increased the need for expert and intelligent systems to deal with credit risk management works. Clustering algorithms are widely used for knowledge acquisition in such systems to assess the credit risk from a data point of view. Conventional clustering algorithms fall short in credit risk assessment because credit datasets are typically high-dimensional, class-imbalanced, and with large sample size. To address these problems, this paper proposes a novel evolutionary multi-objective soft subspace clustering (EMOSSC) algorithm for credit risk assessment. Firstly, we develop a soft subspace clustering validity index for credit risk assessment, by which we can detect the underlying subspace for each cluster from the entire high-dimensional feature space, and we also incorporate the weight of each cluster and the between-cluster separation into the clustering validity index to obtain a comprehensive data structure in the clustering process. Secondly, we propose to optimize the clustering criteria of the new clustering validity index simultaneously by a multi-objective evolutionary algorithm without any predefined weighting coefficients, which guarantees the robustness of the algorithm. We also further provide a local search strategy which significantly accelerates the convergence of the algorithm. Thirdly, we design a GPU-based parallel computation framework for updating the weights of features in our proposed algorithm to improve the computational efficiency on credit datasets with large sample size. Finally, we conduct a comprehensive experiment, with its results demonstrate the superiority of our proposed algorithm in credit risk assessment.

Subspace Clustering of Categorical and Numerical Data With an Unknown Number of Clusters.

In clustering analysis, data attributes may have different contributions to the detection of various clusters. To solve this problem, the subspace clustering technique has been developed, which aims at grouping the data objects into clusters based on the subsets of attributes rather than the entire data space. However, the most existing subspace clustering methods are only applicable to either numerical or categorical data, but not both. This paper, therefore, studies the soft subspace clustering of data with both of the numerical and categorical attributes (also simply called mixed data for short). Specifically, an attribute-weighted clustering model based on the definition of object-cluster similarity is presented. Accordingly, a unified weighting scheme for the numerical and categorical attributes is proposed, which quantifies the attribute-to-cluster contribution by taking into account both of intercluster difference and intracluster similarity. Moreover, a rival penalized competitive learning mechanism is further introduced into the proposed soft subspace clustering algorithm so that the subspace cluster structure as well as the most appropriate number of clusters can be learned simultaneously in a single learning paradigm. In addition, an initialization-oriented method is also presented, which can effectively improve the stability and accuracy of -means-type clustering methods on numerical, categorical, and mixed data. The experimental results on different benchmark data sets show the efficacy of the proposed approach.

A survey on soft subspace clustering

Subspace clustering (SC) is a promising technology involving clusters that are identified based on their association with subspaces in high-dimensional spaces. SC can be classified into hard subspace clustering (HSC) and soft subspace clustering (SSC). While HSC algorithms have been studied extensively and are well accepted by the scientific community, SSC algorithms are relatively new. However, as they are said to be more adaptable than their HSC counterparts, SSC algorithms have been attracting more attention in recent years. A comprehensive survey of existing SSC algorithms and recent developments in the field are presented in this paper. SSC algorithms have been systematically classified into three main categories: conventional SSC (CSSC), independent SSC (ISSC), and extended SSC (XSSC). The characteristics of these algorithms are highlighted and potential future developments in the area of SSC are discussed. Through a comprehensive review of SSC, this paper aims to provide readers with a clear profile of existing SSC methods and to foster the development of more effective clustering technologies and significant research in this area.

Transfer Entropy Weighting Soft Subspace Clustering

In order to get better clustering precision, the traditional clustering algorithms usually need the support of large amount of historical data. The impact it brings about is: the previous clustering algorithm seems not effective if there exists some information losses in the current situation data collection and the division relationship between datasets is not significant. In this study, a novel clustering technique called transfer entropy weighting soft subspace clustering algorithm (T_EWSC) is proposed by employing the historical information. The properties of this algorithm are investigated and performance is evaluated experimentally using real datasets, including UCI benchmarking datasets, high dimensional gene expression datasets. The experimental results demonstrate that the proposed algorithm is able to use historical information to make up for the inadequacy of the current information and perform well.

Distance metric learning for soft subspace clustering in composite kernel space

Soft subspace clustering algorithms have been successfully used for high dimensional data in recent years. However, the existing algorithms often utilize only one distance function to evaluate the distance between data items on each feature, which cannot deal with datasets with complex inner structures. In this paper, a composite kernel space (CKS) is constructed based on a set of basis kernels and a novel framework of soft subspace clustering is proposed by integrating distance metric learning in the CKS. Two soft subspace clustering algorithms, i.e., entropy weighting fuzzy clustering in CKS for kernel space (CKS-EWFC-K) and feature space (CKS-EWFC-F) are thus developed. In both algorithms, the prototype in the feature space is mapped into the CKS by multiple simultaneous mappings, one mapping for each cluster, which is distinct from existing kernel-based clustering algorithms. By evaluating the distance on each feature in the CKS, both CKS-EWFC-K and CKS-EWFC-F learn the distance function adaptively during the clustering process. Experimental results have demonstrated that the proposed algorithms in general outperform classical clustering algorithms and are immune to ineffective kernels and irrelevant features in soft subspace.

Active learning for improving a soft subspace clustering algorithm

In this paper a new soft subspace clustering algorithm is proposed. It is an iterative algorithm based on the minimization of a new objective function. The classification approach is developed by acting at three essential points. The first one is related to an initialization step; we suggest to use a multi-class support vector machine (SVM) for improving the initial classification parameters. The second point is based on the new objective function. It is formed by a separation term and compactness ones. The density of clusters is introduced in the last term to yield different cluster shapes. The third and the most important point consists in an active learning with SVM incorporated in the classification process. It allows a good estimation of the centers and the membership degrees and a speed convergence of the proposed algorithm. The developed approach has been tested to classify different synthetic datasets and real images databases. Several indices of performance have been used to demonstrate the superiority of the proposed method. Experimental results have corroborated the effectiveness of the proposed method in terms of good quality and optimized runtime.

A soft subspace clustering algorithm with log-transformed distances

Entropy weighting used in some soft subspace clustering algorithms is sensitive to the scaling parameter. In this paper, we propose a novel soft subspace clustering algorithm by using log-transformed distances in the objective function. The proposed algorithm allows users to choose a value of the scaling parameter easily because the entropy weighting in the proposed algorithm is less sensitive to the scaling parameter. In addition, the proposed algorithm is less sensitive to noises because a point far away from its cluster center is given a small weight in the cluster center calculation. Experiments on both synthetic datasets and real datasets are used to demonstrate the performance of the proposed algorithm.

A novel soft subspace clustering algorithm with noise detection for high dimensional datasets

When dealing with high dimensional data, clustering faces the curse of dimensionality problem. In such data sets, clusters of objects exist in subspaces rather than in whole feature space. Subspace clustering algorithms have already been introduced to tackle this problem. However, noisy data points present in this type of data can have great impact on the clustering results. Therefore, to overcome these problems simultaneously, the fuzzy soft subspace clustering with noise detection (FSSC-ND) is proposed. The presented algorithm is based on the entropy weighting soft subspace clustering and noise clustering. The FSSC-ND algorithm uses a new objective function and update rules to achieve the mentioned goals and present more interpretable clustering results. Several experiments have been conducted on artificial and UCI benchmark datasets to assess the performance of the proposed algorithm. In addition, a number of cancer gene expression datasets are used to evaluate the performance of the proposed algorithm when dealing with high dimensional data. The results of these experiments demonstrate the superiority of the FSSC-ND algorithm in comparison with the state of the art clustering algorithms developed in earlier research.

A Robust k-Means Type Algorithm for Soft Subspace Clustering and Its Application to Text Clustering

Soft subspace clustering are effective clustering techniques for high dimensional datasets. In this work, a novel soft subspace clustering algorithm RSSKM are proposed. It is based on the incorporation of the alternative distance metric into the framework of k -means type algorithm for soft subspace clustering and can automatically calculates the feature weights of each cluster in the clustering process. The properties of RSSKM are also investigated. Experiments on real world text datasets are conducted and the results show that RSSKM outperformed some popular clustering algorithms for text mining, while still maintaining efficiency of the k -means clustering process.

A Robust k-Means Type Algorithm for Soft Subspace Clustering and Its Application to Text Clustering

Soft subspace clustering are effective clustering techniques for high dimensional datasets. Although several soft subspace clustering algorithms have been developed in recently years, its robustness should be further improved. In this work, a novel soft subspace clustering algorithm RSSKM are proposed. It is based on the incorporation of the alternative distance metric into the framework of k-means type algorithm for soft subspace clustering and can automatically calculates the feature weights of each cluster in the clustering process. The properties of RSSKM are also investigated. Experiments on real world text datasets are conducted and the results show that RSSKM outperformed some popular clustering algorithms for text mining, while still maintaining efficiency of the k-means clustering process.

Non-Mode Clustering of Categorical Data with Attributes Weighting

类属型数据广泛分布于生物信息学等许多应用领域,其离散取值的特点使得类属数据聚类成为统计机器学习领域一项困难的任务.当前的主流方法依赖于类属属性的模进行聚类优化和相关属性的权重计算.提出一种非模的类属型数据统计聚类方法.首先,基于新定义的相异度度量,推导了属性加权的类属数据聚类目标函数.该函数以对象与簇之间的平均距离为基础,从而避免了现有方法以模为中心导致的问题.其次,定义了一种类属型数据的软子空间聚类算法.该算法在聚类过程中根据属性取值的总体分布,而不仅限于属性的模,赋予每个属性衡量其与簇类相关程度的权重,实现自动的特征选择.在合成数据和实际应用数据集上的实验结果表明,与现有的基于模的聚类算法和基于蒙特卡罗优化的其他非模算法相比,该算法有效地提高了聚类结果的质量.;While categorical data are widely used in many applications such as Bioinformatics, clustering categorical data is a difficult task in the filed of statistical machine learning due to the characteristic of the data which can only take discrete values. Typically, the mainstream methods are dependent on the mode of the categorical attributes in order to optimize the clusters and weight the relevant attributes. A non-mode approach is proposed for statistically clustering of categorical data in this paper. First, based on a newly defined dissimilarity measure, an objective function with attributes weighting is derived for categorical data clustering. The objective function is defined based on the average distance between the objects and the clusters, therefore overcomes the problems in the existing methods based on the mode category. Then, a soft-subspace clustering algorithm is proposed for clustering categorical data. In this algorithm, each attribute is assigned with weights measuring its degree of relevance to the clusters in terms of the overall distribution of categories instead of the mode category, enabling automatic feature selection during the clustering process. Experimental results carried out on some synthetic datasets and real-world datasets demonstrate that the proposed method significantly improves clustering quality.

Soft Subspace Clustering Algorithm for Streaming Data

PDF HTML阅读 XML下载 导出引用 引用提醒 一种基于数据流的软子空间聚类算法 DOI: 10.3724/SP.J.1001.2013.04469 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(61273258,61272437,61073189);上海市自然科学基金(13ZR1417500);上海市教育委员会科研创新项目(14YZ131) Soft Subspace Clustering Algorithm for Streaming Data Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:针对高维数据的聚类研究表明,样本在不同数据簇往往与某些特定的数据特征子集相对应.因此,子空间聚类技术越来越受到关注.然而,现有的软子空间聚类算法都是基于批处理技术的聚类算法,不能很好地应用于高维数据流或大规模数据的聚类研究中.为此,利用模糊可扩展聚类框架,与熵加权软子空间聚类算法相结合,提出了一种有效的熵加权流数据软子空间聚类算法——EWSSC(entropy-weighting streaming subspace clustering).该算法不仅保留了传统软子空间聚类算法的特性,而且利用了模糊可扩展聚类策略,将软子空间聚类算法应用于流数据的聚类分析中.实验结果表明,EWSSC 算法对于高维数据流可以得到与批处理软子空间聚类方法近似一致的实验结果. Abstract:A key challenge to most conventional clustering algorithms in handling many real life problems is that data points in different clusters are often correlated with different subsets of features. To address this problem, subspace clustering has attracted increasing attention in recent years. However, the existing subspace clustering methods cannot be effectively applied to large-scale high dimensional data and data streams. In this study, the scalable clustering technique to subspace clustering is extend to form soft subspace clustering for streaming data. An entropy-weighting streaming subspace clustering algorithm, EWSSC is proposed. This method leverages on the effectiveness of fuzzy scalable clustering method for streaming data by revealing the important local subspace characteristics of high dimensional data. Substantial experimental results on both artificial and real-world datasets demonstrate that EWSSC is generally effective in clustering high dimensional streaming data. 参考文献 相似文献 引证文献

Alternative Soft Subspace Clustering Algorithm

While the Euclidean distance metric is commonly utilized in most soft subspace clustering algorithms, some robust distance metrics can also be considered for soft subspace clustering. In this study, a novel soft subspace clustering algorithm called Alternative Soft Subspace Clustering (ASSC) is proposed by incorporating the alternative distance metric into the framework of entropy weighting subspace clustering. The properties of ASSC are investigated and a theoretical analysis on robustness is established accordingly. Experiments on UCI data are conducted and the experimental results show the effectiveness of the proposed algorithm.

Fuzzy partition based soft subspace clustering and its applications in high dimensional data

As one of the most popular clustering techniques for high dimensional data, soft subspace clustering (SSC) algorithms have been receiving a great deal of attention in recent years. Unfortunately, most existing works do not cluster high dimensional sparse data and noisy data in an effective manner. In this study, a novel soft subspace clustering algorithm called PI-SSC is proposed. By introducing a partition index (PI) into the objective function, a novel soft subspace clustering algorithm that combines the concepts of hard and fuzzy clustering is proposed. Furthermore, the robust property of PI-SSC is analyzed from the viewpoint of ε-insensitive distance. A convergence theorem for PI-SSC is also established by applying Zangwill’s convergence theorem. The results of the experiment demonstrate the effectiveness of the proposed algorithm in high dimensional sparse text data and noisy texture data.

Evolving soft subspace clustering

A key challenge to most conventional clustering algorithms in handling many real world problems is that, data points in different clusters are often correlated with different subsets of features. To address this problem, subspace clustering has attracted increasing attention in recent years. In practical data mining applications, data points may arrive in continuous streams with chunks of samples being collected at different time points. In addition, huge amounts of data often cannot be kept in the main memory due to memory restriction. Accordingly, a range of evolving clustering algorithms has been proposed, however, traditional evolving clustering methods cannot be effectively applied to large-scale high dimensional data and data streams. In this study, we extend the online learning strategy and scalable clustering technique to soft subspace clustering to form evolving soft subspace clustering. We propose two online soft subspace clustering algorithms, OFWSC and OEWSC, and two streaming soft subspace clustering algorithms, SSSC_F and SSSC_E. The proposed evolving soft subspace clustering leverages on the effectiveness of online learning scheme and scalable clustering methods for streaming data by revealing the important local subspace characteristics of high dimensional data. Substantial experimental results on both artificial and real-world datasets demonstrate that our proposed methods are generally effective in evolving clustering and achieve superior performance over existing soft subspace clustering techniques.