Quantization of FIR filters under a total integer cost constraint

Abstract

In this paper, we present computationally efficient algorithms for obtaining a particular class of optimal quantized representations of finite-impulse response (FIR) filters. We consider a scenario where each quantization level is associated with a certain integer cost and, given an FIR filter with real coefficients, our goal is to find the quantized representation that minimizes a certain error criterion under a constraint on the total cost of all quantization levels used to represent the filter coefficients. We first formulate the problem as a constrained shortest path problem and discuss how an efficient dynamic programming algorithm can be used to obtain the optimal quantized representation for arbitrary quantization sets. We then develop a greedy algorithm which has even lower computational complexity and is shown to be optimal when the quantization levels and their associated costs satisfy a certain, easily checkable criterion. For the special case of the quantization set that involves levels that are sums of signed powers-of-two and whose cost is captured by the number of powers of two used in their representation, the total integer cost relates to the cost of the very large-scale integration implementation of the given FIR filter and our analysis clarifies the optimality of previously proposed algorithms in this setting.