Rubble-pile asteroids may be the type of near-Earth object most likely to threaten Earth in a future collision event. Small-scale impact experiments and numerical simulations for large-scale impacts were conducted to clarify the size ratio of the boulder/projectile diameter effects on ejecta size–velocity distribution. A series of small-scale impact cratering experiments were performed on porous gypsum–basalt targets at velocities of 2.3 to 5.5 km·s−1. Three successive ejection processes were observed by high-speed and ultra-high-speed cameras. The momentum transfer coefficient and cratering size were measured. A three-dimensional numerical model reflecting the random distribution of the interior boulders of the rubble-pile structure asteroid is established. The size ratio (length to diameter) of the boulder size inside the asteroid to the projectile diameter changed from 0.25 to 1.7. We conducted a smoothed particle hydrodynamics numerical simulation in the AUTODYN software to study the boulder size effect on the ejecta size–velocity distribution. Simulation results suggest that the microscopic porosity on regolith affects the propagation of shock waves and reduces the velocity of ejecta. Experiments and numerical simulation results suggest that both excavation flow and spalling ejection mechanism can eject boulders (0.12–0.72 m) out of the rubble-pile asteroid. These experiments and simulation results help us select the potential impact site in a planetary defense scenario and reduce deflection risk. are comprised primarily of boulders of a range of sizes.