Introduction to Entropy and Entropy Optimization Principles

Abstract

This chapter provides a historical perspective of the concept of entropy, Shannon’s reasoning, and the axioms that justify the principles of entropy optimization, namely, the maximum entropy and minimum cross-entropy principles. The mathematical forms of various entropy optimization problems are also discussed along with references to the existing literature. The chapter consists of two sections. Section 1.1 introduces the concept of entropy, and Section 1.2 classifies different entropy optimization problems to be studied in this book.