Spectral Rotation Research Articles

One-Step Adaptive Spectral Clustering Networks

Deep spectral clustering is a popular and efficient algorithm in unsupervised learning. However, deep spectral clustering methods are organized into three separate steps: affinity matrix learning, spectral embedding learning, and K-means clustering on spectral embedding. In this case, although each step can achieve its own performance, it is still difficult to obtain robust clustering results. In this letter, we propose a one-step adaptive spectral clustering network to overcome the aforementioned shortcomings. The network embeds the three parts of affinity matrix learning, spectral embedding learning, and indicator learning into a unified framework. The affinity matrix is adaptively adjusted by spectral embedding in a deep subspace. We introduce spectral rotation to discretize spectral embedding, which makes the spectral embedding and indicator be learned simultaneously to improve clustering quality. Each part of the model can be iteratively updated based on other parts to optimize the clustering results. Experimental results on four real datasets show the effectiveness of our method on the ACC and NMI clustering evaluation metrics. In particular, our method achieves an NMI of 0.932 and an ACC of 0.973 on the MNIST dataset, a decent performance boost compared to the best baseline.

Fast Multi-view Clustering via Prototype Graph

Multi-view clustering attracts considerable attention due to its effectiveness in unsupervised learning. However, previous multi-view spectral clustering methods include two separated steps: 1) Obtaining a spectral embedding; 2) Performing classical clustering methods. Although these methods have achieved promising performance, there is still some limitations. First, in computing spectral embedding, multi-view spectral clustering approaches exist high computational complexity since they usually need eigenvalue decomposition on laplacian matrix L. Its computational complexity is O(n³) where n is the number of samples; Second, in constructing similarity matrices, previous methods need to compute similarity between any two samples; Third, the two-stage approach only can obtain sub-optimal solution; Fourth, treating equally all views is unreasonable. To address these issues, we propose a Fast Multi-view Clustering via Prototype Graph (FMVPG) method. Specifically, the prototype graph is firstly constructed, and then simultaneously perform spectral embedding to obtain the real matrix and spectral rotation to get the indicator matrix. In addition, the Alternating Direction Method of Multipliers (ADMM) is used to solve the joint optimization problem. Further, we conduct extensive experiments to evaluate the proposed FMVPG approach. These experimental results show the comparable or even better clustering performance than the state-of-the-art approaches.

Bipartite graph based spectral rotation with fuzzy anchors

Spectral clustering usually obtains impressive performance owing to the usage of the manifold structure of data. However, it is infeasible to widely apply conventional spectral clustering methods due to the high computational complexity. To address this challenging problem, a series of extensions like anchor points based models have been developed. Nevertheless, the existing approaches separate anchor points selection and the construction of the bipartite graph, which hinders the performance of the anchor-based methods. In this paper, by revealing the connection between the existing anchor-based spectral clustering and fuzzy k-means, the performance of related methods is found to be limited by the early termination. Accordingly, we proposed a complete learning approach to generate high-quality bipartite graphs. Furthermore, the produced bipartite graph can be directly utilized to perform spectral clustering, and the improved spectral rotation is employed to further boost the performance. Eventually, extensive experiments illustrate that the proposed method obtains good performance especially when the anchor points are limited.

One-step spectral rotation clustering with balanced constrains

One-step spectral rotation clustering with balanced constrains

Multi-view clustering by joint spectral embedding and spectral rotation

Discrete Spectral clustering is an effective tool for directly getting discrete labels. However, existing spectral clustering carry out spectral embedding and spectral rotation separately, which may limit their clustering performance. Seamlessly connecting above two processes is challenging. In this paper, we propose a new multi-view clustering framework, namely Multi-view Clustering by Joint Spectral Embedding and Spectral Rotation. In the framework, the differences of Laplacian matrices from different views are learned adaptively. Moreover, the real-valued cluster indicator matrix is approximated by continuous orthogonalization of the discrete clustering index matrix. By doing so, our method has better convergence, which is also strictly mathematically proven. Extensive experiments indicate that our method is superior to several state-of-the-art methods.

Fuzzy graph clustering

Spectral clustering is a group of graph-based clustering methods in which the columns of the scaled cluster indicator matrix can be obtained by stacking the eigenvectors of the Laplacian matrix corresponding to the top c smallest eigenvalues ( c is the number of clusters). This leads to the possible existence of negative values in the scaled indicator matrix and therefore a post-processing step such as K means clustering or spectral rotation is necessary to get the discrete cluster assignments. Moreover, such obtained results lack of the interpretability for data points in the boundary area of multiple clusters. To simultaneously address both limitations, we propose a two-stage clustering model, termed FGC (fuzzy graph clustering) in this paper. In FGC, we first construct a doubly stochastic graph affinity matrix which is then approximated by the scaled product of the fuzzy cluster indicator matrices. The newly designed fuzzy cluster indicator matrix has two desirable properties of non-negativity and row normalization, which can bring us two benefits. On one hand, we can directly get the cluster assignment of a certain data point by checking the largest value in the corresponding row of the fuzzy cluster indicator matrix; and on the other hand, we can obtain the membership of each data point to different clusters. An iterative method under the alternative optimization framework is proposed to solve the objective function of FGC. We conduct data clustering experiments on both synthetic and benchmark data sets and the results demonstrate the effectiveness of our proposed FGC model.

A new Kmeans clustering model and its generalization achieved by joint spectral embedding and rotation.

The Kmeans clustering and spectral clustering are two popular clustering methods for grouping similar data points together according to their similarities. However, the performance of Kmeans clustering might be quite unstable due to the random initialization of the cluster centroids. Generally, spectral clustering methods employ a two-step strategy of spectral embedding and discretization postprocessing to obtain the cluster assignment, which easily lead to far deviation from true discrete solution during the postprocessing process. In this paper, based on the connection between the Kmeans clustering and spectral clustering, we propose a new Kmeans formulation by joint spectral embedding and spectral rotation which is an effective postprocessing approach to perform the discretization, termed KMSR. Further, instead of directly using the dot-product data similarity measure, we make generalization on KMSR by incorporating more advanced data similarity measures and call this generalized model as KMSR-G. An efficient optimization method is derived to solve the KMSR (KMSR-G) model objective whose complexity and convergence are provided. We conduct experiments on extensive benchmark datasets to validate the performance of our proposed models and the experimental results demonstrate that our models perform better than the related methods in most cases.

One-step multi-view spectral clustering by learning common and specific nonnegative embeddings

Multi-view spectral clustering is a hot research area which has attracted increasing attention. Most existing multi-view spectral clustering methods utilize a two-step strategy. The first step obtains a common embedding by fusing spectral embeddings of different views, and the second step conducts hard clustering, such as K-means or spectral rotation, on the common embedding. Because the goal of the first step is not obtaining optimal clustering result, and the requirement to post-processing makes the final clustering result uncertain. In this paper, we propose a novel one-step multi-view spectral clustering method, in which the spectral embedding and nonnegative embedding are unified into one framework. Therefore, our method can avoid the uncertainty brought by post-processing and obtain optimal clustering result. Moreover, the nonnegative embedding is divided into two parts. The common nonnegative embedding indicates the shared cluster structure, and the specific nonnegative embedding indicates the exclusive cluster structure of each view. Hence, our method can well tackle with noises and outliers of different views. Furthermore, an alternating iterative algorithm is used to solve the joint optimization problem. Extensive experimental results on four real-world datasets have demonstrated the effectiveness of the proposed method.

Fast optimization of spectral embedding and improved spectral rotation

Spectral clustering is a vital clustering method and has been widely applied for data analysis and pattern reorganization. A routine of solving spectral clustering problem consists of two successive stages: (1) solving a relaxed continuous optimization problem to obtain a real-valued indicator solution (2) transform the real-valued indicator into a 0-1 discrete one as the final clustering result. However, we may lose the optimal solution with such a two-stage process. Besides, the spectral clustering has a high time complexity which limits the analysis of large-scale data. To alleviate these problems, this paper proposes an efficient spectral clustering framework that computes spectral embedding and improved spectral rotation simultaneously (SE-ISR). In addition, we also provide a parameter-free method (SE-ISR-PF) to automatically choose the trade-off parameter. Furthermore, with an anchor-based similarity matrix construction, it is scalable to large-scale data. An effective algorithm with a strict convergence proof is provided to solve the corresponding optimization problem. Experimental results on several benchmark datasets demonstrate that the proposed algorithm outperforms the state-of-art methods.

Adaptive Spectral Rotation via Joint Cluster and Pairwise Structure

Density structure and pairwise structure serve as two different but complementary perspectives for clustering. Either side of road is frequently visited and explored by multiple clustering methods. However, there are seldom approaches, which could mutually exploit both structures for clustering. To address this problem, in this paper, we develop a novel adaptive joint clustering algorithm, which combines unsupervised discrete orthogonal least squares discriminant analysis (DOLSDA) and discrete spectral clustering (DSC) with adaptive neighbors and side information into a unified model. Firstly, we extend supervised OLSDA to a discrete kernel clustering problem. To further achieve a clear pairwise structure, a new similarity with adaptive neighbors is then derived to establish sparse Laplacian matrix. In addition, side information could be incorporated to formulate clearer graph by modifying the proposed similarity. Based on the constructed graph, DSC is embedded with the discrete kernel OLSDA (DKOLSDA) clustering to exploit both cluster and pairwise data structures. Equipped with the proposed framework regarding quadratic weighted optimization, adaptive weight can be obtained automatically to leverage both unsupervised DKOLSDA and DSC. Since the unified problem is still discrete, we develop an increment scheme to achieve the optimal spectral rotation for the approximate solution to the predicted indicator.

Design and Evaluation of a Spectral Phase Rotation Algorithm for Upmixing to 3D Audio

Design and Evaluation of a Spectral Phase Rotation Algorithm for Upmixing to 3D Audio

One-step spectral rotation clustering for imbalanced high-dimensional data

The class distribution of imbalanced data sets is skewed in practical application. As traditional clustering methods mainly are designed for improving the overall learning performance, the majority class usually tends to be clustered and the minority class which is more valuable maybe ignored. Moreover, existing clustering methods can be limited for the performance of imbalanced and high-dimensional domains. In this paper, we present one-step spectral rotation clustering for imbalanced high-dimensional data (OSRCIH) by integrating self-paced learning and spectral rotation clustering in a unified learning framework, where sample selection and dimensionality reduction are simultaneously considered with mutual and iterative update. Specifically, the imbalance problem is considered by selecting the same number of training samples from each intrinsic group of the training data, where the sample-weight vector is obtained by self-paced learning. Moreover, dimensionality reduction is conducted by combining subspace learning and feature selection. Experimental analysis on synthetic datasets and real datasets showed that OSRCIH could recognize and enhance the weight of important samples and features so as to avoid the clustering method in favor of the majority class and to improve effectively the clustering performance.

Ultrashort Vortex Pulses with Controlled Spectral Gouy Rotation

Recently, the spatio-spectral propagation dynamic of ultrashort-pulsed vortex beams was demonstrated by 2D mapping of spectral moments. The rotation of characteristic anomalies, so-called “spectral eyes”, was explained by wavelength-dependent Gouy phase shift. Controlling of this spectral rotation is essential for specific applications, e.g., communication and processing. Here, we report on advanced concepts for spectral rotational control and related first-proof-of-principle experiments. The speed of rotation of spectral eyes during propagation is shown to be essentially determined by angular and spectral parameters. The performance of fixed diffractive optical elements (DOE) and programmable liquid-crystal-on silicon spatial light modulators (LCoS-SLMs) that act as spiral phase gratings (SPG) or spiral phase plates (SPP) is compared. The approach is extended to radially chirped SPGs inducing axially variable angular velocity. The generation of time-dependent orbital angular momentum (self-torque) by superimposing multiple vortex pulses is proposed.

Large Graph Clustering With Simultaneous Spectral Embedding and Discretization.

Spectral clustering methods are gaining more and more interests and successfully applied in many fields because of their superior performance. However, there still exist two main problems to be solved: 1) spectral clustering methods consist of two successive optimization stages-spectral embedding and spectral rotation, which may not lead to globally optimal solutions, 2) and it is known that spectral methods are time-consuming with very high computational complexity. There are methods proposed to reduce the complexity for data vectors but not for graphs that only have information about similarity matrices. In this paper, we propose a new method to solve these two challenging problems for graph clustering. In the new method, a new framework is established to perform spectral embedding and spectral rotation simultaneously. The newly designed objective function consists of both terms of embedding and rotation, and we use an improved spectral rotation method to make it mathematically rigorous for the optimization. To further accelerate the algorithm, we derive a low-dimensional representation matrix from a graph by using label propagation, with which, in return, we can reconstruct a double-stochastic and positive semidefinite similarity matrix. Experimental results demonstrate that our method has excellent performance in time cost and accuracy.

A novel data representation framework based on nonnegative manifold regularisation

ABSTRACT Representation learning techniques have been frequently applied in multimedia content analysis and retrieval. In this study, an efficient multimedia data clustering method is presented, which consists of two independent algorithms. First, we propose a new representation framework by incorporating sparse coding and manifold regularisation in an optimisation objective function, the cluster indicator matrix is estimated by introducing sparsity norm coarsely. Second, we refine the estimated cluster indicator matrix by performing spectral rotation such that an optimal assignment for clustering can be learned. Compared with existing methods, we have the following merits: our method takes into account the global matrix reconstruction information and locality manifold information simultaneously. Therefore, global and locality information both are respected. Additionally, theoretical justification about the novel representation method is presented in this study. Comprehensive experiments demonstrate the effectiveness and efficiency of our method in comparison with the state-of-the-art clustering methods on six real-world image datasets.

Intermediate frequency variation in long‐range FMCW radar for harmonics cancellation

This study presents a technique for removing the inevitable harmonic distortions and intermodulation products that primary frequency-modulated continuous wave (FMCW) radars inherently generate in the baseband. The required mixers and baseband amplifiers are non-linear devices that introduce signal distortions, depending on the positions of targets in the field of view. Target distances are directly coupled to their corresponding baseband frequencies. The mitigation strategy presented in this study involves changing the resulting baseband frequencies of targets by varying the mixer deramping frequency input. After applying a processing algorithm that involves spectral rotation and combination, harmonic contributions can be suppressed to a high degree, while target-related signals remain the same. After only four iterations, a gain of 30 dB in spurious free dynamic range can be achieved that can even be improved with more time on target. This behaviour is not only observed in simulations, but is also proven with measurements using an FMCW radar, working at a 3 GHz carrier frequency. The algorithm is applicable to radar systems that work in any frequency band.

Spectral representation learning for one-step spectral rotation clustering

Conventional spectral clustering is generally divided into two main steps, 1) constructing a reliable spectral representation by similarity matrix or Laplacian matrix; 2) performing K-means clustering on the spectral representation. In this paper, we propose a novel spectral clustering algorithm to improve clustering performance by separately improving these two steps in a same learning framework. Specifically, we first utilize two dimensionality reduction methods to learn the robust low-dimensional spectral representation, and a hypergraph structure to make the spectral representation keep the high-order relation of original data. Furthermore, a spectral rotation clustering method is embedded into the spectral representation learning model to conduct one-step clustering, which effectively reduces the deviation of clustering and obtains a reliable clustering performance. Besides, an effective optimization algorithm is further proposed to solve the objective problem to have a fast convergence. Experimental analysis on real data sets showed that our proposed clustering method outperformed the classical and state-of-the-art spectral clustering methods in terms of four frequently-used clustering metrics.

A Spectral Rotation Method with Triplet Periodicity Property for Planted Motif Finding Problems.

Genes are known as functional patterns in the genome and are presumed to have biological significance. They can indicate binding sites for transcription factors and they encode certain proteins. Finding genes from biological sequences is a major task in computational biology for unraveling the mechanisms of gene expression. Planted motif finding problems are a class of mathematical models abstracted from the process of detecting genes from genome, in which a specific gene with a number of mutations is planted into a randomly generated background sequence, and then gene finding algorithms can be tested to check if the planted gene can be found in feasible time. In this work, a spectral rotation method based on triplet periodicity property is proposed to solve planted motif finding problems. The proposed method gives significant tolerance of base mutations in genes. Specifically, genes having a number of substitutions can be detected from randomly generated background sequences. Experimental results on genomic data set from Saccharomyces cerevisiae reveal that genes can be visually distinguished. It is proposed that genes with about 50% mutations can be detected from randomly generated background sequences. It is found that with about 5 insertions or deletions, this method fails in finding the planted genes. For a particular case, if the deletion of bases is located at the beginning of the gene, that is, bases are not randomly deleted, then the tolerance of the method for base deletion is increased.

Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding

The application of most existing multi-view spectral clustering methods is generally limited by the following three deficiencies. First, the requirement to post-processing, such as K-means or spectral rotation. Second, the susceptibility to parameter selection. Third, the high computation cost. To this end, in this paper we develop a novel method that integrates nonnegative embedding and spectral embedding into a unified framework. Two promising advantages of proposed method include 1) the learned nonnegative embedding directly reveals the consistent clustering result, such that the uncertainty brought by post-processing can be avoided; 2) the involved model is parameter-free, which makes our method more applicable than existing algorithms that introduce many additional parameters. Furthermore, we develop an efficient inexact Majorization-Minimization method to solve the involved model which is non-convex and non-smooth. Experiments on multiple benchmark datasets demonstrate that our method achieves state-of-the-art performance.

LABIN: Balanced Min Cut for Large-Scale Data.

Although many spectral clustering algorithms have been proposed during the past decades, they are not scalable to large-scale data due to their high computational complexities. In this paper, we propose a novel spectral clustering method for large-scale data, namely, large-scale balanced min cut (LABIN). A new model is proposed to extend the self-balanced min-cut (SBMC) model with the anchor-based strategy and a fast spectral rotation with linear time complexity is proposed to solve the new model. Extensive experimental results show the superior performance of our proposed method in comparison with the state-of-the-art methods including SBMC.