Entropy Approximation in Lossy Source Coding Problem

Marek Śmieja,Jacek Tabor

doi:10.3390/e17053400

Abstract

In this paper, we investigate a lossy source coding problem, where an upper limit on the permitted distortion is defined for every dataset element. It can be seen as an alternative approach to rate distortion theory where a bound on the allowed average error is specified. In order to find the entropy, which gives a statistical length of source code compatible with a fixed distortion bound, a corresponding optimization problem has to be solved. First, we show how to simplify this general optimization by reducing the number of coding partitions, which are irrelevant for the entropy calculation. In our main result, we present a fast and feasible for implementation greedy algorithm, which allows one to approximate the entropy within an additive error term of log2 e. The proof is based on the minimum entropy set cover problem, for which a similar bound was obtained.

Highlights

Lossy source coding transforms possibly continuously-distributed information into a finite number of codewords [1,2]
Our method is reminiscent of the procedure used to approximate the solution of the minimum entropy set cover problem (MESC) [17,18], where a similar bound was derived
The paper focused on a non-standard type of lossy source coding

Summary

Introduction

Lossy source coding transforms possibly continuously-distributed information into a finite number of codewords [1,2]. This allows one to encode data efficiently, such an operation is irreversible, and once modified, information cannot be restored accurately. One of the fundamental questions in lossy coding is the following: What is the lowest achievable statistical code length given a maximal coding error? The precise formulation of the coding error and related definition of the Entropy 2015, 17 entropy need to be given. We present how to approximate the value of the entropy in the case when every entry element has a fixed upper limit on the permitted error

Objectives

Results

Discussion

Conclusion