Communications, Compression and Fundamental Limits

Abstract

This book is about Information Theory and Rate Distortion Theory specifically pointed toward the problems of communications and compression. Information theory and rate distortion theory were introduced by Shannon in his landmark 1948 paper [1, 2], wherein he presents the fundamental results concerning lossless (noiseless) source coding, channel capacity, and rate distortion theory and demonstrates the critical roles played by entropy and mutual information in establishing these fundamental limits. Shannon revisited fundamental limits on lossy source coding in his 1959 paper [3], wherein he coined the term rate distortion function, proved coding theorems, calculated R(D) for several examples, derived what we now call the Shannon lower bound on R(D), and noted the duality between a rate distortion function and a capacity cost function.