Research on Principle and Application of Maximum Entropy

Abstract

Entropy is a measure of the degree of chaos in the system which comes from physics. Then scientists proposed information entropy form a mathematical perspective. Later, it discovered the relationship between entropy and information entropy. This broke down the barriers between disciplines and derived many related conceptual principles. Among them, the principle of maximum entropy is widely used in disciplines such as finance, computer, etc. and many applications and technologies based on it was emerged. This paper introduces the principle of entropy and maximum entropy principle and reviews the application and development of the maximum entropy principle in analysis of clustering, decision and spectrum. Drawing upon our literature survey this paper presents a new application of maximum entropy principle, called elastic net of clustering based on maximum entropy (ENCM), which applies the maximum entropy principle to elastic net to change the objective function to solve clustering. Experiments verify that this method could effectively improve the result of clustering.