The phenomenon that the ultrasonic vibration (UV) can reduce the flow stress in the metal plastic deformation has been shown in many studies, however, the mechanism of the flow stress decline when the copper-nickel alloy sheet undergoes the UV is unclear yet. In this study, a series of tensile tests are conducted on the copper-nickel alloy sheet. The results indicate that the UV induces softening effect, i.e., the significant decline of the flow stress and the tensile strength. Nevertheless, the elongation of specimens decreases obviously in the tensile tests. To investigate the mechanisms, the microstructure evolution of the copper-nickel alloy sheet in the UV assisted tensile tests are explored by the scanning electron microscope (SEM), the electron backscattering diffraction (EBSD) and the transmission electron microscope (TEM). It is found that after the UV superimposition the high-angle grain boundaries (HAGB) retain more and the reduction of the twinning decreases in the copper-nickel alloy sheet, which impairs the plasticity and the elongation. The results also show that the average grain size under the UV conditions is bigger, which leads to the decrease of the flow stress and weaken the plasticity of the metal.