A variable annuity contract with Guaranteed Minimum Withdrawal Benefit (GMWB) promises to return the entire initial investment through cash withdrawals during the contract plus the remaining account balance at maturity, regardless of the portfolio performance. Under the optimal (dynamic) withdrawal strategy of a policyholder, GMWB pricing becomes an optimal stochastic control problem that can be solved by backward recursion of Bellman equation. In this paper we develop a very efficient new algorithm for pricing these contracts in the case of stochastic interest rate not considered previously in the literature. Presently our method is applied to the Vasicek interest rate model, but it is generally applicable to any model when transition density or moments of the underlying asset and interest rate are known in closed form or can be evaluated efficiently. Using bond price as a numeraire the required expectations in the backward recursion are reduced to two-dimensional integrals calculated through a high order Gauss-Hermite quadrature applied on a two-dimensional cubic spline interpolation. The quadrature is applied after a rotational transformation to the variables corresponding to the principal axes of the bivariate transition density, which empirically was observed to be more accurate than the the use of Cholesky transformation.Numerical results from the new algorithm for a series of GMWB contracts for both static and optimal cases are presented. As a validation, results of the algorithm are compared with the closed form solutions for simple vanilla options, and with Monte Carlo and finite difference results for the static GMWB. The comparison demonstrates that the new algorithm is significantly faster than finite difference or Monte Carlo for all the two-dimensional problems tested so far. For dynamic GMWB pricing, we found that for positive correlation between the underlying asset and interest rate, the GMWB price under the stochastic interest rate is significantly higher compared to the case of deterministic interest rate, while for negative correlation the difference is less but still significant. In the case of static GMWB, for negative correlation, the difference in prices between stochastic and deterministic interest rate cases is not material while for positive correlation the difference is still significant. The algorithm can be easily adapted to solve similar stochastic control problems with two state variables possibly affected by control. Also applications to numerical pricing of Asian, barrier and other financial derivatives under stochastic interest rate are straightforward.