This paper aims to understand the statistics of the electric field produced by water interfacing a non-polar solute of nanometer dimension. We study, by numerical simulations, the interface between SPC/E water and a Kihara solute, which is a hard-sphere core with a Lennard-Jones layer at its surface. The distribution of the interfacial electric field is monitored as a function of the magnitude of a point dipole placed close to the solute-water interface. The free energy surface as a function of the electric field projected on the dipole direction shows a cross-over with increasing dipole magnitude. While it is a single-well harmonic function at low dipole values, it becomes a double-well surface at intermediate dipole moment magnitudes, transforming into a single-well surface again, with a non-zero minimum position, at still higher dipoles. This transformation, reminiscent of a discontinuous phase transition in bulk materials, has a broad intermediate region where the interfacial waters fluctuate between the two minima. This region is characterized by intense field fluctuations, with non-Gaussian statistics and variance far exceeding expectations from the linear-response approximation. The excited state of the surface water is found to be lifted above the ground state by the energy required to break approximately two hydrogen bonds. This state is pulled down in energy by the external electric field of the solute dipole, making it readily accessible to thermal excitations. The excited state is a surface defect in the hydrogen-bond network, creating a stress in the nearby network, but otherwise relatively localized in the region closest to the solute dipole.