In this paper, we study the asymptotic behavior of solutions to the wave equation with damping depending on the space variable and growing at the spatial infinity. We prove that the solution is approximated by that of the corresponding heat equation as time tends to infinity. The proof is based on semigroup estimates for the corresponding heat equation having a degenerate diffusion at spatial infinity and weighted energy estimates for the damped wave equation. To construct a suitable weight function for the energy estimates, we study a certain elliptic problem.