Spectral Clustering Technique Research Articles

Spectral clustering with more than K eigenvectors

Spectral clustering techniques are heuristic algorithms aiming to find approximate solutions to difficult graph-cutting problems, usually NP-complete, which are useful to clustering. A fundamental working hypothesis of these techniques is that the optimal partition of K classes can be obtained from the first K eigenvectors of the graph normalized Laplacian matrix L N if the gap between the K-th and the K+1-th eigenvalue of L N is sufficiently large. If the gap is small a perturbation may swap the corresponding eigenvectors and the results can be very different from the optimal ones. In this paper we suggest a weaker working hypothesis: the optimal partition of K classes can be obtained from a K-dimensional subspace of the first M > K eigenvectors, where M is a parameter chosen by the user. We show that the validity of this hypothesis can be confirmed by the gap size between the K-th and the M+1-th eigenvalue of L N . Finally we present and analyse a simple probabilistic algorithm that generalizes current spectral techniques in this extended framework. This algorithm gives results on real world graphs that are close to the state of the art by selecting correct K-dimensional subspaces of the linear span of the first M eigenvectors, robust to small changes of the eigenvalues.

Spectral Clustering and Visualization: A novel Clustering of Fisher's Iris Data Set

Clustering is the act of partitioning a set of elements into subsets, or clusters, so that elements in the same cluster are, in some sense, similar. Determining an appropriate number of clusters in a particular data set is an important issue in data mining and cluster analysis. Another important issue is visualizing the strength, or connectivity, of clusters. We begin by creating a consensus matrix using multiple runs of the clustering algorithm k-means. This consensus matrix can be interpreted as a graph, which we cluster using two spectral clustering methods: the Fiedler Method and the MinMaxCut Method. To determine if increasing the number of clusters from k to k + 1 is appropriate, we check whether an existing cluster can be split. Finally, we visualize the strength of clusters by using the consensus matrix and the clustering obtained through one of the aforementioned spectral clustering techniques. Using these methods, we then investigate Fisher’s Iris data set. Our methods support the existence of four clusters, instead of the generally accepted three clusters in this data.

Use of a fuzzy granulation–degranulation criterion for assessing cluster validity

The identification of a suitable clustering algorithm to partition data and assessment of the validity of the resultant partitioning are ongoing quests in unsupervised learning. In this study, a fuzzy granulation–degranulation criterion is proposed to evaluate the goodness of a fuzzy partitioning of the data. This, in turn, is used to determine the appropriate clustering algorithm suitable for a particular data set. In general, the quality of a partitioning is measured by computing the variance within it, which is a measure of compactness of the obtained partitioning. Here a new error function, which reflects how well the computed cluster centers represent the whole data set, is used as the goodness measure of the obtained partitioning. Thus a clustering algorithm, providing a good set of cluster centers which approximate well the whole data set, is considered to be the most suited. Thereafter this new fuzzy granulation–degranulation criterion is used to develop six new cluster validity indices. These indices mimic the definitions of the existing and well-known cluster validity indices, such as PBM-index, XB-index, PS-index, FS-index, K-index and SV-index, but use the new fuzzy granulation–degranulation based error function instead of cluster compactness. In order to evaluate the effectiveness of the proposed error function in correctly identifying the appropriate clustering algorithm for a particular data set, eight well-known clustering algorithms, K-means, Fuzzy C-means, GAK-means (genetic algorithm based K-means algorithm), a newly developed genetic point symmetry based clustering technique (GAPS-clustering), Average Linkage clustering algorithm, Expectation Maximization (EM) clustering algorithm, Self-Organizing Map (SOM) and Spectral clustering technique are evaluated on a set of six artificially generated and six real-life data sets. Results show that GAK-means is the most appropriate for most of the data sets used for the experiments. Thereafter the effectiveness of the proposed cluster validity indices in identifying the appropriate number of clusters automatically from different data sets are shown for above mentioned 12 data sets. For the purpose of comparison, results obtained with the original versions of the proposed cluster validity indices and results obtained by a density based clustering technique are also presented.

Non-local spatial spectral clustering for image segmentation

As one of widely used clustering algorithms, spectral clustering clusters data using the eigenvectors of the Laplacian matrix derived from a dataset and has been successfully applied to image segmentation. However, spectral clustering algorithms are sensitive to noise and other imaging artifacts because of not taking into account the spatial information of the pixels in the image. In this paper, a novel non-local spatial spectral clustering algorithm for image segmentation is presented. In the proposed method, the objective function of weighted kernel k-means algorithm is firstly modified by incorporating the non-local spatial constraint term. Then the equivalence between the objective functions of normalized cut and weighted kernel k-means with non-local spatial constraints is given and a novel non-local spatial matrix is constructed to replace the normalized Laplacian matrix. Finally, spectral clustering techniques are applied to this matrix to obtain the final segmentation result. The novel algorithm is performed on synthetic and real images, especially magnetic resonance (MR) images, and compared with the traditional spectral clustering algorithms and segmentation algorithms with spatial information. Experimental results demonstrate that the proposed algorithm is robust to noise in the image and obtains more effective performance than the comparison algorithms.

On principle axis based line symmetry clustering techniques

In this paper, at first a new line symmetry (LS) based distance is proposed which calculates the amount of symmetry of a point with respect to the first principal axis of a data set. The proposed distance uses a recently developed point symmetry (PS) based distance in its computation. Kd-tree based nearest neighbor search is used to reduce the complexity of computing the closest symmetric point. Thereafter an evolutionary clustering technique is described that uses this new principal axis based LS distance for assignment of points to different clusters. The proposed GA with line symmetry distance based (GALS) clustering technique is able to detect any type of clusters, irrespective of their geometrical shape, size or convexity as long as they possess the characteristics of LS. GALS is compared with the existing genetic algorithm based K-means clustering technique, GAK-means, existing genetic algorithm with PS based clustering technique, GAPS, spectral clustering technique, and average linkage clustering technique. Five artificially generated data sets having different characteristics and seven real-life data sets are used to demonstrate the superiority of the proposed GALS clustering technique. In a part of experiment, utility of the proposed genetic LS distance based clustering technique is demonstrated for segmenting the satellite image of the part of the city of Kolkata. The proposed technique is able to distinguish different landcover types in the image. In the last part of the paper genetic algorithm is used to search for the suitable line of symmetry of each cluster.

Distributed user profiling via spectral methods

User profiling is a useful primitive for constructing personalized services, such as content recommendation. In the present work we investigate the feasibility of user profiling in a distributed setting, with no central authority and only local information exchanges between users. Our main contributions are: (i)~We propose a spectral clustering technique, and prove its ability to recover unknown user profiles with only few measures of affinity between users. (ii)~We develop distributed algorithms which achieve an embedding of users into a low-dimensional space, based on spectral transformation. These involve simple message passing among users, and provably converge to the desired embedding.

SU‐GG‐J‐137: Application of Unsupervised Spectral Clustering in 3D for PET Tumor Segmentation: A Digital Phantom Study

Purpose: Spectral clustering refers to a class of techniques which rely on the eigenstructure of a similarity matrix to partition pixels into disjoint clusters. This study intended to investigate the feasibility of using the unsupervised spectral clustering method for PET tumor segmentation. Method and Materials: In this study a digital PET phantom was adopted as the source to generate PET images with heterogeneous and/or irregular shaped tumors. The simulation was carried out by feeding the Zubal phantom as anatomical input to the SimSET (the Simulation System for Emission Tomography) software, which employs Monte Carlo techniques to model the underlying physical processes and imaging system to produce simulated emission tomography data. There were a total of 30 lung tumors with a variety of heterogeneous activities and/or irregular shapes simulated. The unsupervised spectral clustering algorithm was then applied to segment the simulated images. Segmentation accuracy was assessed with two metrics: volume detection error (VDE) and volume overlap measure (VOM). Results: The spectral clustering technique partitions the image volume into an optimal number of clusters according to the pair‐wise similarity measure between voxels. Upon segmentation, one can then select the segmented tumor regions for subsequent processing and analysis. Applying the spectral clustering technique on all the 30 simulated PET images shows that VDE ranges from −5.5% to 6.2%, and VOM ranges from 0.94 to 1.Conclusion: This study shows the potential of the unsupervised 3D spectral clustering method for PET tumor segmentation. Segmentation performance in terms of VDE and VOM on stimulated PET images shows this method may have the potential for accurate and consistent segmentation of tumors. Compared with thresholding‐based PET tumor delineation techniques, it segments the PET image without requiring any threshold‐volume relationship. Studies to further evaluate this method on patient data are planned.

TH-D-213A-04: Application of Supervised Spectral Clustering for PET Tumor Delineation: A Phantom Study

Purpose: Spectral clustering is a powerful technique for image segmentation. This study investigated the feasibility of using the supervised spectral clustering technique for PET tumor delineation. Method and Materials: A phantom was constructed with six spheres filled with featuring simulated tumors of varying size ranged from 0.5ml to 20ml situated in a cylindrical container filled with -FDG. The phantom was scanned for 120 minutes in a PET/CT scanner. For every 2 minute scan, 6 images were reconstructed by using the iterative OSEM algorithm with 5mm-FWHM Gaussian smoothing filter and followed with an anisotropic filtering. The supervised spectral clustering algorithm using normalized cut was then applied to segment the filtered images. Spectral clustering uses the eigenvectors and eigenvalues of a similarity matrix (based on both proximity and intensity) to partition pixels into clusters. Its segmentation performance was evaluated with two metrics: area detection error and area overlap metric. Results: The supervised spectral clustering technique partitions the image to a designated number of regions according the measure of the pair-wise affinities between pixels. The spectral clustering has a rather steady segmentation performance of greater than 85% area overlap for images with S/B ratios ⩾ 5. The detected area is enclosed in the true area until the S/B ratio drops to the lowest level of 2. Conclusion: The preliminary results demonstrate the potential of the supervised spectral clustering method for PET tumor segmentation. Compared with other PET tumor delineation techniques, it segments the PET image with minimal human intervention and is instrument independent.

Using K-means method and spectral clustering technique in an outfitter’s value analysis

This study applies K-means method and spectral clustering technique in the customer data analysis of an outfitter in Taipei City, Taiwan. The data set contains transaction records of 551 customers from April 2004 to March 2006. The differences between the two clustering techniques mentioned here are significant. K-means method is more capable of dealing with linear separable input, while spectral clustering technique might have the advantage in non-linear separable input. Thus, it would be of interest to know which clustering technique performs better in a real-world case of evaluating customer value when the type of input space is unknown. By using cluster quality assessment, this study found that spectral clustering technique performs better than K-means method. To summarize the analysis, this study also suggests marketing strategies for each cluster based on the results generated by spectral clustering technique.

Fast analysis of scalable video for adaptive browsing interfaces

Driven by a high demand for adaptive video interfaces and recent advances in scalable video coding technology, this work introduces a framework for video summarisation and browsing by utilising inherently hierarchical compressed-domain features of scalable video and efficient dynamic video summarisation. This approach enables instant adaptability of generated video summaries to available channel bandwidth as well as display resources. By utilising compressed domain features an efficient hierarchical analysis of motion activity at different layers of complexity is achieved. Exploiting a contour evolution algorithm, a scale space of temporal video descriptors is generated, enabling rapid video summarisation. Given the spatial resources of the terminal display and generated video summary, the final browsing layout is generated utilising an unsupervised robust spectral clustering technique and a fast discrete optimisation algorithm. Results show excellent scalability of the video summaries and good algorithm efficiency.

Detecting the community structure in complex networks based on quantum mechanics

In this paper, we develop a novel method to detect the community structure in complex networks. This approach is based on the combination of kernel-based clustering using quantum mechanics, the spectral clustering technique and the concept of the Bayesian information criterion. We test the proposed algorithm on Zachary’s karate club network and the world of American college football. Experimental results indicate that our algorithm is efficient and effective at finding both the optimal number of clusters, and the best clustering of community structures.

Compact Visualisation of Video Summaries

This paper presents a system for compact and intuitive video summarisation aimed at both high-end professional production environments and small-screen portable devices. To represent large amounts of information in the form of a video key-frame summary, this paper studies the narrative grammar of comics, and using its universal and intuitive rules, lays out visual summaries in an efficient and user-centered way. In addition, the system exploits visual attention modelling and rapid serial visual presentation to generate highly compact summaries on mobile devices. A robust real-time algorithm for key-frame extraction is presented. The system ranks importance of key-frame sizes in the final layout by balancing the dominant visual representability and discovery of unanticipated content utilising a specific cost function and an unsupervised robust spectral clustering technique. A final layout is created using an optimisation algorithm based on dynamic programming. Algorithm efficiency and robustness are demonstrated by comparing the results with a manually labelled ground truth and with optimal panelling solutions.

Efficient Layout of Comic-Like Video Summaries

In order to represent large amounts of information in the form of a video key-frame summary, this paper studies narrative grammar of comics, and using its universal and intuitive rules, lays out visual summaries in an efficient and user centered way. The system ranks importance of key-frame sizes in the final layout by balancing the dominant visual representability and discovery of unanticipated content utilizing a specific cost function and an unsupervised robust spectral clustering technique. A final layout is created using an optimization algorithm based on dynamic programming. Algorithm efficiency and robustness are demonstrated by comparing the results with the optimal panelling solutions.

A spectral clustering approach to underdetermined postnonlinear blind source separation of sparse sources

This letter proposes a clustering-based approach for solving the underdetermined (i.e., fewer mixtures than sources) postnonlinear blind source separation (PNL BSS) problem when the sources are sparse. Although various algorithms exist for the underdetermined BSS problem for sparse sources, as well as for the PNL BSS problem with as many mixtures as sources, the nonlinear problem in an underdetermined scenario has not been satisfactorily solved yet. The method proposed in this letter aims at inverting the different nonlinearities, thus reducing the problem to linear underdetermined BSS. To this end, first a spectral clustering technique is applied that clusters the mixture samples into different sets corresponding to the different sources. Then, the inverse nonlinearities are estimated using a set of multilayer perceptrons (MLPs) that are trained by minimizing a specifically designed cost function. Finally, transforming each mixture by its corresponding inverse nonlinearity results in a linear underdetermined BSS problem, which can be solved using any of the existing methods.

Ultrasound image segmentation using spectral clustering

Segmentation of ultrasound images is necessary in a variety of clinical applications, but the development of automatic techniques is still an open problem. Spectral clustering techniques have recently become popular for data and image analysis. In particular, image segmentation has been proposed via the normalized cut (NCut) criterion. This article describes an initial investigation to determine the suitability of such segmentation techniques for ultrasound images. The adaptation of the NCut technique to ultrasound is described first. Segmentation is then performed on simulated ultrasound images. Tests are also performed on abdominal and fetal images with the segmentation results compared to manual segmentation. The success of the segmentation on these test cases warrants further research into NCut-based segmentation of ultrasound images. E-mail: ( narchip@bwh.harvard.edu).

Combined morphological-spectral unsupervised image segmentation

The goal of segmentation is to partition an image into disjoint regions, in a manner consistent with human perception of the content. For unsupervised segmentation of general images, however, there is the competing requirement not to make prior assumptions about the scene. Here, a two-stage method for general image segmentation is proposed, which is capable of processing both textured and nontextured objects in a meaningful fashion. The first stage extracts texture features from the subbands of the dual-tree complex wavelet transform. Oriented median filtering is employed, to circumvent the problem of texture feature response at step edges in the image. From the processed feature images, a perceptual gradient function is synthesised, whose watershed transform provides an initial segmentation. The second stage of the algorithm groups together these primitive regions into meaningful objects. To achieve this, a novel spectral clustering technique is proposed, which introduces the weighted mean cut cost function for graph partitioning. The ability of the proposed algorithm to generalize across a variety of image types is demonstrated.

Analysis of Vowels in Sung Queries for a Music Information Retrieval System

A method for analyzing and categorizing the vowels of a sung query is described and analyzed. This query system uses a combination of spectral analysis and parametric clustering techniques to divide a single query into different vowel regions. The method is applied separately to each query, so no training or repeated measures are necessary. The vowel regions are then transformed into strings and string search methods are used to compare the results from various songs. We apply this method to a small pilot study consisting of 40 sung queries from each of 7 songs. Approximately 60% of the queries are correctly identified with their corresponding song, using only the vowel stream as the identifier.