This study focuses on the feasibility of preparing clinker-free ultra-high performance concrete with solid waste based binder and steel slag aggregate. The influence of refining slag (RS) content, gypsum (DG) content, ground granulated blast furnace slag–hot stifle steel slag (GGBFS–HSSS) ratio and binder–sand ratio on the strength of UHPC were studied by orthogonal test, and then mainly discusses the reaction mechanism and hydration products of four solid wastes to further optimize the performance of clinker-free UHPC. The results showed the compressive and flexural strength of the Y1 samples could be increased to 108.39 MPa and 37.22 MPa respectively by optimizing RS content to 5%, DG content to 20%, GGBFS–HSSS ratio to 3:1, binder–sand ratio to 0.8. According to microanalyses (including XRD, semi-QXRD, FT-IR, TG-DSC, and SEM-EDS), the predominant hydration products of Z1 samples were ettringite and C–(A)–S–H gels, generated by the synergistic reaction of RS, GGBFS, HSSS and DG. In the first 2 days of curing, approximately 12.1% C12A7 and 1.3% gypsum was rapidly consumed, and 20.8% ettringite was simultaneously generated, which demonstrated the early strength was provided by the hydration of RS. During the 3–28 days curing, gypsum and Ca/(Si+Al) molar ratio of gels decreased by 10.9% and 0.61 respectively, and the weight loss rate was 3.85%, which indicated that gypsum reacted synergistically with HSSS and GGBFS during the process, resulting in the formation and growth of a large amount of C–(A)–S–H gels. The gels and ettringite intermingled together, making the structure more compact and the strength of UHPC increased.