Tuning the Parameters for Precision Matrix Estimation Using Regression Analysis

Jiangtao Xi,Philip O Ogunbona,Jun Tong,Jiayi Yang,Yanguang Yu

doi:10.1109/access.2019.2926725

Abstract

Precision matrix, i.e., inverse covariance matrix, is widely used in signal processing, and often estimated from training samples. Regularization techniques, such as banding and rank reduction, can be applied to the covariance matrix or precision matrix estimation for improving the estimation accuracy when the training samples are limited. In this paper, exploiting regression interpretations of the precision matrix, we introduce two data-driven, distribution-free methods to tune the parameter for regularized precision matrix estimation. The numerical examples are provided to demonstrate the effectiveness of the proposed methods and example applications in the design of minimum mean squared error (MMSE) channel estimators for large-scale multiple-input multiple-output (MIMO) communication systems are demonstrated.

Highlights

Covariance matrix reveals the marginal correlations between variables, while precision matrix encodes conditional correlations between pairs of variables given the remaining ones [1]
This paper deals with the estimation of precision matrix, which is required in applications such as minimum mean squared error (MMSE) estimation [5], [6], array signal processing [7], correlation analysis [8] and linear discriminant analysis (LDA) [9]
Since CV requires a form of prediction error to be used as a proxy for measuring the quality of parameter estimation, we propose to use two types of regression errors for this purpose: One uses a regression interpretation of the precision matrix itself and the other uses a similar interpretation of the modified Cholesky factor of the precision matrix [25]

Summary

INTRODUCTION

Covariance matrix reveals the marginal correlations between variables, while precision matrix (i.e., inverse covariance matrix) encodes conditional correlations between pairs of variables given the remaining ones [1]. For banding-based designs, the distribution-free resampling methods have shown to be able to select the bandwidth [14], [30] for covariance matrix estimation under several criteria. They may not yield satisfactory performance for applications where the precision matrices are required. We show that the proposed methods can select precision matrix estimators achieving good out-of-sample prediction power They can be directly used for other precision matrix estimators such as graphical lasso [28] and reduced-rank estimators, as shown by numerical results. We discuss two bandwidth selection methods based on CV and regression analysis, which may be applied to more general regularized designs

AN EXAMPLE OF BANDING-BASED PRECISION

PARAMETER SELECTION

APPLICATIONS TO GENERAL PRECISION MATRIX ESTIMATORS

PRECISION MATRIX ESTIMATION

Findings

CONCLUSION