Image Formation Theory and Experiment Based on ML Criterion and EM Algorithm

Abstract

We obtain data and communicate with others every day. Under so much data, a large number of images are formed and grasped by us. As we all know, information theory builds a foundation of communication. However, whether image formation theory could be guided by information theory needs further study. In this paper, we introduce information-theoretic image formation (ITIF) model and point out the goal of image formation and its relationship with information theory. Since data is not equal to information which is measured by entropy, inference should be used to extract information from data. Human beings communicate by making use of sound waves and electromagnetic waves which carry information. The wave is described by equation while the image is characterized by dispersion equation. The latter is related to the former by the Fourier transform. The image formation method depends on the metric of information. It is easy to derive the maximum entropy (ME) and Maximum Likelihood (ML) under the metric of Shannon entropy and Fisher likelihood information. The paper focuses on ML and its numerical computation method (i.e., Expectation-Maximization algorithm). In order to further reduce the complexity of EM and avoid complex mathematical expectation calculation in E-step of EM algorithm, we introduce Monte Carlo Expectation Maximization (MCEM) algorithm where sample mean replaces population mean in E-step. After introducing the sampling theorem and channel capacity theorem as array design principles, a double-helix array in pool experiment serves to prove the effectiveness of ML criterion, EM and MCEM algorithm we propose. Finally, in the end of the paper, we discuss about extensibility of the introduced algorithm and conclude our work.