Deep Fuzzy K-Means With Adaptive Loss and Entropy Regularization

Abstract

Neural network based clustering methods usually have better performance compared to the conventional approaches due to more efficient feature extraction. Most of existing deep clustering techniques either exploit graph information as prior to extract pivotal deep structure from the raw data and simply utilizes stochastic gradient descent (SGD). However, they often suffer from separating the learning steps regarding dimensionality reduction and clustering. To address these issues, a novel deep model named as deep fuzzy k-means (DFKM) with adaptive loss function and entropy regularization is proposed. DFKM performs deep feature extraction and fuzzy clustering simultaneously to generate a more appropriate nonlinear feature map. Additionally, DFKM incorporates FKM so that fuzzy information is utilized to represent a clear structure of deep clusters. To further promote the robustness of the model, a robust loss function is applied to the objective with adaptive weights. Moreover, an entropy regularization is employed for affinity to provide confidence of each assignment and the corresponding membership and centroid matrices are updated by close-form solutions rather than SGD. Extensive experiments show that DFKM has better performance compared to the state-of-the-art fuzzy clustering techniques under three clustering metrics.