Kinetic impacts on a binary asteroid will produce ejecta debris as well as causing a disturbance to the spin-orbit motion of the binary components. Understanding the complex interactions between the ejecta and the binary system can be crucial to both theoretical study and mission design. In this paper, we develop a detailed model for the dynamical evolution of ejecta within a binary asteroid system, in which we combined various perturbations on the ejecta and considered the collisional process with the asteroid. Taking an example scenario like the DART impact, we calculated the fate of ejecta from size 0.01 mm to 10 m and analyzed the distribution of their final deposition locations. The results show that ejecta with diameter less than 0.1 mm escape quickly under the influence of solar radiation pressure, while ejecta of larger size survive longer, i.e., a few cycles in the binary system for more than 400 days. We analyzed the influence of the coefficient of restitution between ejecta and asteroid, and found more damped collisions would allow the ejecta to survive longer in orbit. We also studied the effect of the tumbling rotation of the secondary on the distribution of ejecta on the surface of the secondary, the results suggest that a tumbling rotational state causes obvious changes in Dimorphos’ surface slope, which, however, are not sufficient for triggering surface landslides. In addition, the influence of the internal structure of the primary on the evolution of the ejecta is studied, and we show it has little effect on the evolution of the ejecta but does affect the distribution of ejecta on the surface of the primary. This is due to the change of the distribution of relative acceleration on the surface of the primary, which changes the regions where the ejecta can remain.