We present spectroscopic follow-up observations on two bright Ly$\alpha$ emitter (LAE) candidates originally found by Krug et al. (2012) at a redshift of z~7.7 using the Multi-Object Spectrometer for Infra-Red Exploration (MOSFIRE) at Keck. We rule out any line emission at the >5$\sigma$ level for both objects, putting on solid ground a previous null result for one of the objects. The limits inferred from the non-detections rule out the previous claim of no or even reversed evolution between 5.7 < z < 7.7 in the Ly$\alpha$ luminosity function (LF) and suggest a drop in the Ly$\alpha$ luminosity function consistent with that seen in Lyman Break galaxy (LBG) samples. We model the redshift evolution of the LAE LF using the LBG UV continuum LF and the observed rest-frame equivalent width distribution. From the comparison of our empirical model with the observed LAE distribution, we estimate lower limits of the neutral hydrogen fraction to be 50-70% at z~7.7. Together with this, we find a strong evolution in the Ly$\alpha$ optical depth characterized by (1+z)^(2.2 $\pm$ 0.5) beyond z=6 indicative of a strong evolution of the IGM. Finally, we extrapolate the LAE LF to z~9 using our model and show that it is unlikely that large area surveys like UltraVISTA or Euclid pick up LAEs at this redshift assuming the current depths and area.