Coefficients Quantization for Separable Two‐Dimensional FIR Filter

Abstract

AbstractA separable two‐dimensional (2‐D) finite impulse response (FIR) filter can be implemented using an efficient parallel structure. The continuous coefficient separable 2‐D FIR filter has already been well studied. In this study, the quantization of the coefficients of a separable 2‐D FIR filter is addressed for the first time. Two (iterative) schemes [two‐step integer linear programming algorithm (2‐step‐integer‐LP) and two‐step integer linear programming neighborhood search algorithm (2‐step‐integer‐LP‐neighbor)] for quantizing the coefficients are proposed, which have the same core idea: fixing some coefficients and optimally quantizing the other coefficients. Simulation experiment is conducted to verify the effectiveness of the proposed quantization schemes. The experimental results reveal that the two schemes both outperform the direct rounding method in terms of the design error. 2‐Step‐integer‐LP performs slightly better than 2‐step‐integer‐LP‐neighbor. However, the former may not work, in some case, due to the large number of optimization variables. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.