Differentiating the Method of Conjugate Gradients

Abstract

The method of conjugate gradients (CG) is widely used for the iterative solution of large sparse systems of equations $Ax=b$, where $A\in\Re^{n\times n}$ is symmetric positive definite. Let $x_k$ denote the $k$th iterate of CG. This is a nonlinear differentiable function of $b$. In this paper we obtain expressions for $J_k$, the Jacobian matrix of $x_k$ with respect to $b$. We use these expressions to obtain bounds on $\|J_k\|_2$, the spectral norm condition number of $x_k$, and discuss algorithms to compute or estimate $J_kv$ and $J_k^Tv$ for a given vector $v$.