We derive analytical expressions for the change in the orbital eccentricity of a binary following a distant encounter with a third star on a hyperbolic or parabolic orbit. To establish the accuracy of these expressions, we present detailed comparisons with the results of direct numerical integrations of the equations of motion for the three bodies. We treat with particular care the difficult case of a binary with zero initial eccentricity. In this case, we show that the eccentricity $\delta e$ induced by the encounter declines in general as a power-law, $\delta e\propto (a/\rp)^{5/2}$, where $a$ is the binary semi-major axis and $\rp$ is the periastron distance of the encounter. This power-law arises from the octupole-level secular perturbation of the binary. In contrast, non-secular quadrupole-level perturbations induce an eccentricity change that declines exponentially with $\rp$. These non-secular effects can become dominant at sufficiently small $\rp$, for a sufficiently high relative velocity, or for a sufficiently massive perturber. We also derive cross sections for eccentricity change and compare our results with those of previous studies based on numerical scattering experiments. Our results have important implications for a number of astrophysical problems including, in particular, the evolution of binary millisecond pulsars in globular clusters.