Three groups of filling materials with different mix proportions were prepared. PO42.5 grade cement was selected as the binding material, mechanized granular coal gangue and fine sand were used as the filling aggregate, and then the specimens were curing at room temperature and 95% moisture. Through the uniaxial compression test, the influence of gangue–sand ratio, the mass fraction, and curing age on the mechanical properties of filling materials was analyzed. The microstructure analysis of SEM was conducted to explore the internal mechanism of the strength difference of filling materials. The results show that the gangue–sand ratio dramatically influences the uniaxial compressive strength. The strength increases first and then decreases with the gangue–sand ratio increase. The mass fraction of filling paste is positively correlated with specimen strength. When the sand–cement ratio, gangue–sand ratio, and the mass fraction remain unchanged, the longer the curing age, the greater the uniaxial compressive strength of specimens. The specimens with a sand–cement ratio of 3.5 : 1, a gangue–sand ratio of 5 : 5, and a mass fraction of 86% reach the maximum value of 11.03 MPa at a curing age of 10 days. It can be seen that when the gangue–sand ratio is 5 : 5, the filling material has the best mechanical properties, so it can be recommended as the optimal mix proportion for goaf filling. Thus, it provides a solid technical guarantee for the treatment of mine surface collapse disasters and the utilization of bulk coal gangue resources.