While the effect of ballast degradation on lateral resistance is noteworthy, limited research has delved into the specific aspect of ballast breakage in this context. This study is dedicated to assessing the influence of breakage on sleeper lateral resistance. For simplicity, it is assumed that ballast breakage has already occurred. Accordingly, nine granularity variations finer than No. 24 were chosen for simulation, with No. 24 as the assumed initial particle size distribution. Initially, a DEM model was validated for this purpose using experimental outcomes. Subsequently, employing this model, the lateral resistance of different particle size distributions was examined for a 3.5 mm displacement. The track was replaced by a reinforced concrete sleeper in the models, and no rails or rail fasteners were considered. The sleeper had a simplified model with clumps, the type of which was the so-called B70 and was applied in Western Europe. The sleeper was taken into consideration as a rigid body. The crushed stone ballast was considered as spherical grains with the addition that they were divided into fractions (sieves) in weight proportions (based on the particle distribution curve) and randomly generated in the 3D model. The complete 3D model was a 4.84 × 0.6 × 0.57 m trapezoidal prism with the sleeper at the longitudinal axis centered and at the top of the model. Compaction was performed with gravity and slope walls, with the latter being deleted before running the simulation. During the simulation, the sleeper was moved horizontally parallel to its longitudinal axis and laterally up to 3.5 mm in static load in the compacted ballast. The study successfully established a relationship between lateral resistance and ballast breakage. The current study’s findings indicate that lateral resistance decreases as ballast breakage increases. Moreover, it was observed that the rate of lateral resistance decrease becomes zero when the ballast breakage index reaches 0.6.