To utilize solid soda residue (SR) and building solid waste (BSW), this study proposes to use SR for modification and preparation of mining-used soda residue-gangue-building solid waste-based cement filling material (SGBCFM). In this study, we studied the influence of fly ash (FA) content, SR content, BSW content, and mass concentration (MC) on the early strength and later strength of SGBCFM. Through XRD and SEM tests, we analyzed the hydration products and microstructure of SGBCFM and investigated the mechanism of SR enhancing the later strength of SGBCFM. The results show that these influencing factors have a weaker impact on the early strength of SGBCFM, but a significant impact on later strength. These influencing factors can be ranked from the most significant impact to the least impact as follows: MC, SR content, FA content, and BSW content. A proper amount of SR can effectively improve the later strength of the material, but an excessive amount of SR leads to a decrease in strength; SR has combined effects such as micro-aggregate filling, alkali activation, and promoting cementation reactions. These effects can increase the original cementitious substances and generate new compounds, effectively enhancing the later strength of SGBCFM.