In this article, we consider the hyperbolic and parabolic Anderson models in arbitrary space dimension $d$, with constant initial condition, driven by a Gaussian noise which is white in time. We consider two spatial covariance structures: (i) the Fourier transform of the spectral measure of the noise is a non-negative locally-integrable function; (ii) $d=1$ and the noise is a fractional Brownian motion in space with index $1/4<H<1/2$. In both cases, we show that there is striking similarity between the Laplace transforms of the second moment of the solutions to these two models. Building on this connection and the recent powerful results of [Ann. Inst. Henri Poincaré Probab. Stat. 53 (2017) 1305–1340] for the parabolic model, we compute the second order (upper) Lyapunov exponent for the hyperbolic model. In case (i), when the spatial covariance of the noise is given by the Riesz kernel, we present a unified method for calculating the second order Lyapunov exponents for the two models.