Subspace Clustering Technique Research Articles

A Proposed Clustering Algorithm for Efficient Clustering of High-Dimensional Data

To partition transaction data values, clustering algorithms are used. To analyse the relationships between transactions, similarity measures are utilized. Similarity models based on vectors perform well with low-dimensional data. High-dimensional data values are clustered using subspace clustering techniques. Clustering high-dimensional data is difficult due to the curse of dimensionality. Projective clustering seeks out projected clusters in subsets of a data space's dimensions. In high-dimensional data space, a probability model represents predicted clusters. A model-based fuzzy projection clustering method to find clusters with overlapping boundaries in different projection subspaces. The system employs the Model Based Projective Clustering (MPC) method. To cluster high-dimensional data, projective clustering algorithms are used. A subspace clustering technique is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been optimized. A subspace clustering approach is the model-based projective clustering algorithm. Similarity analysis use non-axis-subspaces. Anomaly transactions are segmented using projected clusters. The suggested system is intended to cluster objects in high-dimensional spaces. The similarity analysis includes non-access subspaces. The clustering procedure validates anomaly data values with similarity. The subspace selection procedure has been improved.

Interpretable Recognition for Dementia Using Brain Images

Machine learning-based models are widely used for neuroimage-based dementia recognition and achieve great success. However, most models omit the interpretability that is a very important factor regarding the confidence of a model. Takagi–Sugeno–Kang (TSK) fuzzy classifiers as the high interpretability and promising classification performance have widely used in many scenarios. TSK fuzzy classifier can generate interpretable fuzzy rules showing the reasoning process. However, when facing high-dimensional data, the antecedent become complex which may reduce the interpretability. In this study, to keep the antecedent of fuzzy rule concise, we introduce the subspace clustering technique and use it for antecedent learning. Experimental results show that the used model can generate promising recognition performance as well as concise fuzzy rules.

Building Extraction from Airborne LiDAR Data Based on Min-Cut and Improved Post-Processing

Building extraction from LiDAR data has been an active research area, but it is difficult to discriminate between buildings and vegetation in complex urban scenes. A building extraction method from LiDAR data based on minimum cut (min-cut) and improved post-processing is proposed. To discriminate building points on the intersecting roof planes from vegetation, a point feature based on the variance of normal vectors estimated via low-rank subspace clustering (LRSC) technique is proposed, and non-ground points are separated into two subsets based on min-cut after filtering. Then, the results of building extraction are refined via improved post-processing using restricted region growing and the constraints of height, the maximum intersection angle and consistency. The maximum intersection angle constraint removes large non-building point clusters with narrow width, such as greenbelt along streets. Contextual information and consistency constraint are both used to eliminate inhomogeneity. Experiments of seven datasets, including five datasets provided by the International Society for Photogrammetry and Remote Sensing (ISPRS), one dataset with high-density point data and one dataset with dense buildings, verify that most buildings, even with curved roofs, are successfully extracted by the proposed method, with over 94.1% completeness and a minimum 89.8% correctness at the per-area level. In addition, the proposed point feature significantly outperforms the comparison alternative and is less sensitive to feature threshold in complex scenes. Hence, the extracted building points can be used in various applications.

Improved subspace clustering algorithm using multi-objective framework and subspace optimization

Subspace clustering technique divides the data set into different groups or clusters where each cluster comprises of objects that share some similar properties. Again, the feature sets or the subspace features that are used to represent clusters are different for different clusters. Moreover, in subspace clustering, the grouping of similar objects and the subspace feature set representing that group are identified simultaneously. In evolutionary-based machine learning problems, two critical measures to determine the quality of the generated clusters are compactness within and separation between the clusters. However, the distance-based separation between two clusters may not be useful in the context of subspace clustering, as the clusters may belong to two different subspaces. Again, in the case of subspace clustering, the selection of relevant subspace features plays a primary role in generating good quality subspace clusters. Therefore, the proposed approach optimizes the subspace features by considering two new objective functions, feature non-redundancy (FNR) and feature per cluster (FPC) represented in the form of PSM-index. Another objective function, intra-cluster compactness (ICC-index), is modified and used to optimize the compactness among objects within the cluster. Finally, an evolutionary-based multi-objective subspace clustering technique is developed in this paper optimizing these validity indices. A new mutation operator, namely duplication and deletion along with the modified version of the exogenous genetic material uptake, are developed to explore the search space effectively. The developed algorithm is tested on sixteen synthetic data sets and seven standard real-life data sets for identifying different subspace clusters. Again, to show the effectiveness of using multiple objectives, the algorithm is also tested on three big data sets and a MNIST data set. Also, an application of the proposed method is shown in bi-clustering the gene expression data. The results obtained by the proposed algorithm are compared against some state-of-the-art methods. Experimentation reveals that the proposed algorithm can take advantage of its evolvable genomic structure and the newly defined objective functions on the multi-objective based framework.

Estimating distance threshold for greedy subspace clustering

Many approaches have been proposed to recognize clusters in subspaces. However, their performance is highly sensitive to input parameter values. The purpose and expected ranges of these parameters may not available to a non-expert user. The parameter setting producing optimal results can only be known after repeated execution of the clustering process every time with a different set, which is very time consuming. Most of the existing algorithms show high runtimes due to excessive data scans. In this work, we propose a subspace clustering technique that estimates the distance threshold parameter automatically from the data for each attribute and works on the basis of single linkage clustering, in bottom up, greedy fashion. The experimental results show that, the algorithm produces optimal results without accepting any input from the user, achieves up to 10 times better runtime and improved accuracy in a single run without requiring any tuning of parameter values.

Fusion of evolvable genome structure and multi-objective optimization for subspace clustering

Subspace clustering techniques become paramount in pattern recognition for detecting local variations from high dimensional data. Several techniques exist in the recent literature for subspace clustering, majority of which optimize implicitly or explicitly a single cluster quality measure. Inspired by the success of multi-objective optimization in solving clustering problem, we developed a multi-objective based subspace clustering technique in this paper. The proposed technique simultaneously optimizes two subspace cluster quality measures, capable of capturing different cluster shapes/properties. Two existing cluster quality measures, XB-index and PBM-index, are modified to develop subspace cluster validity indices, and then those are used as optimization criteria. These cluster validity indices measure the appropriateness of generated subspace clusters in terms of intra-subspace cluster similarity and separation between subspace clusters. The proposed approach utilizes a new evolvable genome structure which stores the information about subspaces in its phenotype and genotype and evolves this genome structure with the help of different genetic operators. The developed algorithm is applied on ten standard real-life data sets and sixteen synthetic datasets for identifying different subspace clusters. The results obtained by this algorithm are compared against some state-of-the-art techniques with respect to different performance metrics. Experimentation reveals that the proposed algorithm is able to take advantages of its evolvable genomic structure and multi-objective based framework and it can be applied to any data set. In a part of the paper, the efficacy of the proposed technique is also shown for bi-clustering of gene-expression data sets.

Semisupervised Sparse Subspace Clustering Method With a Joint Sparsity Constraint for Hyperspectral Remote Sensing Images

Sparse subspace clustering (SSC), as an effective subspace clustering technique, has been widely applied in the remote sensing community, demonstrating a superior performance over the traditional methods such as k-means. In this paper, we propose a unified framework for hyperspectral image (HSI) clustering, which incorporates spatial information and label information in an SSC model, aiming at generating a more precise similarity matrix. The spatial information is included through a joint sparsity constraint on the coefficient matrix of each local region. Pixels within a local region are encouraged to select a common set of samples in the subspace-sparse representation, which greatly promotes the connectivity of the similarity matrix. We incorporate the available label information effectively within the same framework, by zeroing the entries of the sparse coefficient matrix, which correspond to the data points from different classes. An optimization algorithm is derived based on the alternating direction method of multipliers for the resulting model. Experimental results on real HSIs demonstrate a superior performance over the related state-of-the-art methods.

Subspace clustering for the finite mixture of generalized hyperbolic distributions

The finite mixture of generalized hyperbolic distributions is a flexible model for clustering, but its large number of parameters for estimation, especially in high dimensions, can make it computationally expensive to work with. In light of this issue, we provide an extension of the subspace clustering technique developed for finite Gaussian mixtures to that of generalized hyperbolic distribution. The methodology will be demonstrated with numerical experiments.

Estimating animal acoustic diversity in tropical environments using unsupervised multiresolution analysis

Ecoacoustic monitoring has proved to be a viable approach to capture ecological data related to animal communities. While experts can manually annotate audio samples, the analysis of large datasets can be significantly facilitated by automatic pattern recognition methods. Unsupervised learning methods, which do not require labelled data, are particularly well suited to analyse poorly documented habitats, such as tropical environments. Here we propose a new method, named Multiresolution Analysis of Acoustic Diversity (MAAD), to automate the detection of relevant structure in audio data. MAAD was designed to decompose the acoustic community into few elementary components (soundtypes) based on their time–frequency attributes. First, we used the short-time Fourier transform to detect regions of interest (ROIs) in the time–frequency domain. Then, we characterised these ROIs by (1) estimating the median frequency and (2) by running a 2D wavelet analysis at multiple scales and angles. Finally, we grouped the ROIs using a model-based subspace clustering technique so that ROIs were automatically annotated and clustered into soundtypes. To test the performance of the automatic method, we applied MAAD to two distinct tropical environments in French Guiana, a lowland high rainforest and a rock savanna, and we compared manual and automatic annotations using the adjusted Rand index. The similarity between the manual and automated partitions was high and consistent, indicating that the clusters found are intelligible and can be used for further analysis. Moreover, the weight of the features estimated by the clustering process revealed important information about the structure of the acoustic communities. In particular, the median frequency had the strongest effect on modelling the clusters and on classification performance, suggesting a role in community organisation. The number of clusters found in MAAD can be regarded as an estimation of the soundtype richness in a given environment. MAAD is a comprehensive and promising method to automatically analyse passive acoustic recordings. Combining MAAD and manual analysis would maximally exploit the strengths of both human reasoning and computer algorithms. Thereby, the composition of the acoustic community could be estimated accurately, quickly and at large scale.

A rough set based subspace clustering technique for high dimensional data

Subspace clustering aims at identifying subspaces for cluster formation so that the data is categorized in different perspectives. The conventional subspace clustering algorithms explore dense clusters in all the possible subspaces. These algorithms suffer from the curse of dimensionality. That is, with the increase in the number of dimensions, the possible number of subspaces to be explored as well as the number of subspace clusters increase exponentially. This makes analysis of clustering result difficult due to high probability of redundant clustering information presented in various subspaces. To handle this problem, a new algorithm called Interesting Subspace Clustering (ISC) is proposed which makes use of attribute dependency measure, γ from Rough Set theory, to identify interesting subspaces. Anti-monotonicity based on Apriori property is used to efficiently prune the subspaces in the process of identifying interesting subspaces. A density based clustering method is used so as to mine arbitrary shaped dense regions as clusters in interesting subspaces. The proposed algorithm identifies non-redundant and interesting subspace clusters of better quality. The size of the clustering result is reduced as well as the mean dimensionality needed to describe the clustering solution compared to existing algorithms, SUBCLU and SCHISM on different datasets.

Subspace Clustering of Categorical and Numerical Data With an Unknown Number of Clusters.

In clustering analysis, data attributes may have different contributions to the detection of various clusters. To solve this problem, the subspace clustering technique has been developed, which aims at grouping the data objects into clusters based on the subsets of attributes rather than the entire data space. However, the most existing subspace clustering methods are only applicable to either numerical or categorical data, but not both. This paper, therefore, studies the soft subspace clustering of data with both of the numerical and categorical attributes (also simply called mixed data for short). Specifically, an attribute-weighted clustering model based on the definition of object-cluster similarity is presented. Accordingly, a unified weighting scheme for the numerical and categorical attributes is proposed, which quantifies the attribute-to-cluster contribution by taking into account both of intercluster difference and intracluster similarity. Moreover, a rival penalized competitive learning mechanism is further introduced into the proposed soft subspace clustering algorithm so that the subspace cluster structure as well as the most appropriate number of clusters can be learned simultaneously in a single learning paradigm. In addition, an initialization-oriented method is also presented, which can effectively improve the stability and accuracy of -means-type clustering methods on numerical, categorical, and mixed data. The experimental results on different benchmark data sets show the efficacy of the proposed approach.

Point cloud normal estimation via low-rank subspace clustering

In this paper, we present a robust normal estimation algorithm based on the low-rank subspace clustering technique. The main idea is based on the observation that compared with the points around sharp features, it is relatively easier to obtain accurate normals for the points within smooth regions. The points around sharp features and smooth regions are identified by covariance analysis of their neighborhoods. The neighborhood of a point in a smooth region can be well approximated by a plane. For a point around sharp features, some of its neighbors may be in smooth regions. These neighbor points’ normals are estimated by principal component analysis, and used as prior knowledge to carry out neighborhood clustering. An unsupervised learning process is designed to represent the prior knowledge as a guiding matrix. Then we segment the anisotropic neighborhood into several isotropic neighborhoods by low-rank subspace clustering with the guiding matrix, and identify a consistent subneighborhood for the current point. Hence the normal of the current point near sharp features is estimated as the normal of a plane fitting the consistent subneighborhood. Our method is capable of estimating normals accurately even in the presence of noise and anisotropic samplings, while preserving sharp features within the original point data. We demonstrate the effectiveness and robustness of the proposed method on a variety of examples.

Integration of unsupervised and supervised neural networks to predict dissolved oxygen concentration in canals

The main focus of this paper was to devise a method to accurately predict the amount of dissolved oxygen (DO) in Bangkok canals at the present month based on the following 13 water quality parameters collected the previous month: temperature, pH value (pH), hydrogen sulfide (H2S) content, DO, biochemical oxygen demand (BOD), chemical oxygen demand (COD), suspended solids (SS), total kjeldahl nitrogen (TKN), ammonia nitrogen (NH3N), nitrite nitrogen (NO2N), nitrate nitrogen (NO3N), total phosphorous (T-P), and total coliform (TC). Accurately predicting the amount of DO in a canal via scientific deduction is an important step in efficient water management and health care planning. We proposed a new technique that enhances the prediction accuracy by constructing a set of sub-manifolds of the predicting function by deploying unsupervised and supervised neural networks. The data were obtained from the Bangkok Metropolitan Administration Department of Drainage and Sewerage during the years 2007–2011. Comparisons between our proposed technique and other techniques using the correlation coefficient (R), the mean absolute error (MAE), and the mean square error (MSE) showed that our proposed approach with the sub-space clustering technique yielded higher accuracy than did other approaches without the sub-space clustering technique.

Feature interaction in subspace clustering using the Choquet integral

Subspace clustering has recently emerged as a popular approach to removing irrelevant and redundant features during the clustering process. However, most subspace clustering methods do not consider the interaction between the features. This unawareness limits the analysis performance in many pattern recognition problems. In this paper, we propose a novel subspace clustering technique by introducing the feature interaction using the concepts of fuzzy measures and the Choquet integral. This new framework of subspace clustering can provide optimal subsets of interacted features chosen for each cluster, and hence can improve clustering-based pattern recognition tasks. Various experimental results illustrate the effective performance of the proposed method.

Active learning and subspace clustering for anomaly detection

Today, anomaly detection is a highly valuable application in the analysis of current huge datasets. Insurance companies, banks and many manufacturing industries need systems to help humans to detect anomalies in their daily information. In general, anomalies are a very small fraction of the data, therefore their detection is not an easy task. Usually real sources of an anomaly are given by specific values expressed on selective dimensions of datasets, furthermore, many anomalies are not really interesting for humans, due to the fact that interestingness of anomalies is categorized subjectively by the human user. In this paper we propose a new semi-supervised algorithm that actively learns to detect relevant anomalies by interacting with an expert user in order to obtain semantic information about user preferences. Our approach is based on 3 main steps. First, a Bayes network identifies an initial set of candidate anomalies. Afterwards, a subspace clustering technique identifies relevant subsets of dimensions. Finally, a probabilistic active learning scheme, based on properties of Dirichlet distribution, uses the feedback from an expert user to efficiently search for relevant anomalies. Our results, using synthetic and real datasets, indicate that, under noisy data and anomalies presenting regular patterns, our approach correctly identifies relevant anomalies.