Dealing with Collinearity in Finite Impulse Response Models Using Bayesian Shrinkage

Abstract

Finite impulse response (FIR) models are widely used in the chemical industry due to their simplicity and easily defined model structures. However, they usually require a large number of coefficients to capture the process dynamics. This large number of coefficients introduces collinearity or redundancy in the models, which usually increases the uncertainty of their estimation. This paper presents a Bayesian approach, called empirical Bayesian FIR (EBFIR) modeling, to deal with this collinearity problem. Unlike ridge regression (RR), which shrinks the FIR model parameters toward zero in order to reduce their variations, EBFIR makes better use of the data by estimating a prior density function for the FIR coefficients, which is then used within a Bayesian framework to shrink the FIR coefficients in a more rigorous fashion. The developed Bayesian approach is computationally inexpensive, as a closed form solution of the Bayesian estimate of the FIR coefficients is derived. The EBFIR approach is also more general than some of the existing methods, such as ordinary least squares regression (OLS) and RR, which are shown to be special cases of EBFIR. Finally, the developed EBFIR approach has been shown to outperform some of the commonly used existing FIR estimation methods over a wide range of noise levels.