We investigate the linear stability of shears near the Couette flow for a class of 2D incompressible stably stratified fluids. Our main result consists of nearly optimal decay rates for perturbations of stationary states whose velocities are monotone shear flows $(U(y),0)$ and have an exponential density profile. In the case of the Couette flow $U(y)=y$, we recover the rates predicted by Hartman in 1975, by adopting an explicit point-wise approach in frequency space. As a by-product, this implies optimal decay rates as well as Lyapunov instability in $L^2$ for the vorticity. For the previously unexplored case of more general shear flows close to Couette, the inviscid damping results follow by a weighted energy estimate. Each outcome concerning the stably stratified regime applies to the Boussinesq equations as well. Remarkably, our results hold under the celebrated Miles-Howard criterion for stratified fluids.