Currently the main method of mine solid waste disposal and utilization is to apply gangue to underground filling. For the grading characteristics of coal gangue in the hollow zone in underground filling, the state of pressure and the mechanical properties of coal gangue filling materials under different particle sizes. In this paper, through laboratory tests and numerical simulations, the bearing characteristics and deformation damage of gangue are deeply investigated and analyzed from both macroscopic and microscopic perspectives. The results of the study show that when the axial pressure reaches a certain threshold, the strain increases accordingly with the increase in grain size. Under the axial pressure condition of 15MPa, the bearing capacity of gangue with different grain sizes under the lateral pressure of steel cylinder showed obvious positive correlation. In the software PFC3D simulation, as the particle size of gangue particles increases, it has a significant effect on the natural stacking porosity of the specimen, and gangue particles are more prone to force chain instability and failure. Mixed particle size gangue can significantly reduce the natural accumulation porosity of the specimen, thus improving its overall stability. Significant displacement triangles existed in the top and lateral directions of the B1 (0–5 mm), B2 (5–10 mm), and B5 (0–20 mm) specimens under the same circumferential pressure conditions. The results of the study are of great significance for further selection of reasonable gangue gradation and determination of its mechanical properties.