We address an interesting question raised by Dos Santos Ferreira, Kenig and Salo about regions ${\mathcal R}_g\subset {\mathbb C}$ for which there can be uniform $L^{\frac{2n}{n+2}}\to L^{\frac{2n}{n-2}}$ resolvent estimates for $\Delta_g+\zeta$, $\zeta \in {\mathcal R}_g$, where $\Delta_g$ is the Laplace-Beltrami operator with metric $g$ on a given compact boundaryless Riemannian manifold of dimension $n\ge3$. This is related to earlier work of Kenig, Ruiz and the third author for the Euclidean Laplacian, in which case the region is the entire complex plane minus any disc centered at the origin. Presently, we show that for the round metric on the sphere, $S^n$, the resolvent estimates in Ferreira et al, involving a much smaller region, are essentially optimal. We do this by establishing sharp bounds based on the distance from $\zeta$ to the spectrum of $\Delta_{S^n}$. In the other direction, we also show that the bounds in \cite{Kenig} can be sharpened logarithmically for manifolds with nonpositive curvature, and by powers in the case of the torus, ${\mathbb T}^n={\mathbb R}^n/{\mathbb Z}^n$, with the flat metric. The latter improves earlier bounds of Shen. Further improvements for the torus are obtained using recent techniques of the first author and his work with Guth based on the multilinear estimates of Bennett, Carbery and Tao. Our approach also allows us to give a natural necessary condition for favorable resolvent estimates that is based on a measurement of the density of the spectrum of $\sqrt{-\Delta_g}$, and, moreover, a necessary and sufficient condition based on natural improved spectral projection estimates for shrinking intervals.