Extreme physical information and objective function in fuzzy clustering

Abstract

Fuzzy clustering algorithms have been widely studied and applied in a variety of areas. They become the major techniques in cluster analysis. In this paper, we focus on objective function models whose aim is to assign the data to clusters so that a given objective function is optimized. We propose a new approach in fuzzy clustering and show how it can be used to obtain a systematic method deriving objective functions. This approach is based on a unifying principle of physics, that of extreme physical information (EPI) defined by Frieden (Physics from Fisher Information. A Unification, 1999). The information in question is the trace of the Fisher information matrix for the estimation procedure; this information is shown to be a physical measure of disorder. We use the EPI approach for finding the effective and minimal constraint terms in objective functions. With the proposed approach we justify the constraint terms defined a priori in the Fuzzy c-means (FcM) algorithm and Possibilistic and Maximum Entropy Inference approaches. Indeed, these algorithms, by contrast, offer no such systematic method of finding its constraints. Moreover, in this context, the EPI approach derives the “reason” for the extremization of objective functions. The resulting formulae have a clearer physical meaning than those obtained by means of classical algorithms. The updated equations of our algorithm are identical to those of the possibilistic, MEI and FcM with regularization approaches.