The effect of anionic asphalt emulsion (AE) as partial replacement of Portland cement to enhance the damping properties of concrete was examined. Five concrete mixtures were designed and prepared, increasing AE content from 0 to 40% in overall binder volume, in 10% increments. Loss factor, dynamic moduli, resonant frequencies, and compressive strength were obtained at the age of 7, 28, 90, and 180 days. Results show that the loss factor of concrete increased as the asphalt content increased, whereas the dynamic modulus and resonant frequency decreased. In addition to the viscous damping contributed by asphalt droplets, the more porous microstructure and weaker interfacial transition zones in AE modified concrete can also enhance the damping properties. However, an inverse correlation between concrete strength and damping can be observed with increasing AE content, where the substitution of AE results in a significant decrease in the compressive strength of concrete. Therefore, the anionic AE modified concrete seems to be more suitable for non-structural application where strength is not a key requirement.