Information Theory and Coding Fundamentals

Abstract

This chapter is devoted to classical information theory fundamentals and its application to fading channels and channels with memory. The chapter starts with definitions of entropy, joint entropy, conditional entropy, relative entropy, mutual information, and channel capacity, followed by the information capacity theorem. We discuss the channel capacity of discrete memoryless channels, continuous channels, and channels with memory. Regarding the wireless channels, we describe how to calculate the channel capacity of flat fading and frequency-selective channels. We also discuss different optimum and suboptimum strategies to achieve channel capacity including the water-filling method, multiplexed coding and decoding, channel inversion, and truncated channel inversion. We also study different strategies for channel capacity calculation depending on what is known about the channel state information. Further, we explain how to model the channel with memory and describe McMillan–Khinchin model for channel capacity evaluation. After that, the fundamentals of linear blocks codes are introduced, followed by the binary LDPC coding fundamentals.