High-dimensional partial correlation coefficients: A survey study of estimation Methods

Abstract

. The partial correlation coefficient (Pcor) measures the degree of association between two random variables after removing the effect of another set of variables, called the controlling variables. When the dimension of these controlling variables is high, estimating Pcor becomes exceedingly challenging, as conventional computational methods fail to provide efficient estimators. Although several methods have been proposed for estimating Pcor in high-dimensional data, they primarily focus on testing whether its value is zero rather than enhancing estimation efficiency. This article concentrates on improving estimation efficiency and surveys existing methods. Through extensive simulation studies on high-dimensional data, we observe that all current methods consistently underestimate the absolute values of Pcor, with the bias being more pronounced when Pcor is positive. By considering a combination of regression equations, X = Z α + ε and Y = Z β + ζ , the method proposed in this article demonstrates a significant improvement over existing approaches. Stocks data in Hong Kong is employed to demonstrate the effectiveness of these methods.