Optimizing cluster structures with inner product induced norm based dissimilarity measures: Theoretical development and convergence analysis

Abstract

Dissimilarity measures play a key role in exploring the inherent cluster structure of the data for any partitional clustering algorithm. Commonly used dissimilarity functions for clustering purpose are so far confined to the Euclidean, exponential and Mahalanobish distances. In this article we develop generalized algorithms to solve the partitional clustering problems formulated with a general class of Inner Product Induced Norm (IPIN) based dissimilarity measures. We provide an in-depth mathematical analysis of the underlying optimization framework and analytically address the issue of existence of a solution and its uniqueness. In absence of a closed form solution, we develop a fast stochastic gradient descent algorithm and the Minimization by Incremental Surrogate Optimization (MISO) algorithm (in case of constrained optimization) with exponential convergence rate to obtain the solution. We carry out a convergence analysis of the fuzzy and k-means clustering algorithms with the IPIN based dissimilarity measures and also establish how these algorithms guarantee convergence to a stationary point. In addition, we investigate the nature of the stationary point. Novelty of the paper lies in the introduction of a generalized class of divergence measures, development of fuzzy and k-means clustering algorithms with the general class of divergence measures and a thorough convergence analysis of the developed algorithms.