For a few decades now, molecular dynamics (MD) simulations have been extensively used for studying primary damage formation in collision cascades in materials exposed to fast particle irradiation. Because of the stochastic nature of defect production phenomena, it is vital to model collision cascades in batches with the same set of essential simulation parameters that includes primary knock-on atom (PKA) energy and target temperature in the first instance. No generally accepted practice has been established so far to determine the optimal size of the sampling set of MD simulations of collision cascades. The suggested simple a posteriori sampling set size validation method is based on reviewing the dependence of the average and median number of Frenkel pairs generated in collision cascades on the amount n of simulated cascades in a series with identical simulation parameters. As the average and median number of Frenkel pairs converges with raising n, the optimal sampling set size is determined as a function of the simulation conditions. It is shown how it depends on the PKA energy, irradiation temperature, alloy composition and the interaction of collision cascades with the free surfaces.