The purpose of this paper is to compare the coverage probability errors of the parametric bootstrap with that of the delta method for the covariance parameters of a regression model with auto-regressive fractionally integrated moving average (ARFIMA) errors. We consider the coverage probability errors of both confidence intervals (CIs) and tests based on the the plug-in Whittle maximum likelihood (PWML) estimators. We first show that, under some sets of conditions on the regression coefficients, the spectral density function, and the parameter values, the bounds on the coverage probability errors of the two-sided delta method and parametric bootstrap confidence intervals on the plug-in Whittle likelihood estimator or the covariance parameter are shown to be $O(n^{-1})$ and $o(n^{-3/2}\ln{n})$, respectively, where n is the sample size. Next, we show that those of the one-sided parametric bootstrap confidence intervals are shown to be $O(n^{-1/2})$ and $o(n^{-1}\ln{n})$, respectively. These results show that for both one-sided and two-sided confidence intervals and tests, the bootstrap provides a significant improvement over that of the delta method.