In electron beam welding (EBW), the coupling dynamics between the keyhole and the weld pool play significant roles in joint quality, yet the mechanisms of these dynamics have not been well understood, since the rise of EBW in the 1960s. The present study reported a novel three-dimensional mathematical model of EBW and used it to theoretically understand the coupling behaviors between the keyhole and the weld pool during EBW of a Ti-6-Al-4V alloy. A new phenomenological heat source model, dependent on the dynamic keyhole profile and temperature, was developed. The physical effects, including the beam reflection and absorption of the keyhole wall, scattering of secondary electrons, keyhole evaporation heat loss and beam defocus, were rigorously treated in the heat source model. The transient free surface keyhole dynamics were tracked with a high-resolution level set method (LSM). On the free surface boundary, the majority of the key physical effects, such as the recoil pressure, Marangoni shear stress, the surface tension and hydrodynamic pressure of the molten liquid, were incorporated using a recently developed sharp interface model. The coupling behaviors of the three-dimensional keyhole evolutions, heat transfer, and fluid flow in the weld pool asa function of the process parameters in the EBW of Ti-6-Al-4-V alloys were simulated, of which the results reasonably agreed with independent experimental data. The dynamical keyhole violently exhibited fluctuations within the sub-millisecond characteristic time in the radial direction. The oscillations of the keyhole, which were characterized by its surface area and volume, were theoretically studied asa function of the welding speed and surface tension coefficient. The welding speed has stabilization effects on the keyhole and fully stabilized the keyhole under certain high welding speed conditions, an effect similar to that observed in keyhole mode laser welding. The surface tension governed the keyhole oscillation periods and was proposed asa major source of oscillations. In addition, the complex and violent flow patterns of weld pool were predicated and consistent with the well-accepted X-ray imaging experimental results. The EBW transient keyhole and weld pool dynamics were self-consistent. The present study is to present the successful development of EBW, thereby allowing the establishment of the self-consistent keyhole and weld pool dynamics asa function of the process parameters and material properties, to provide a promising avenue for process optimizations in industrial applications.