The low carbonization of ultra-high performance concrete (UHPC) has become a hot topic for sustainable development in the construction industry. Calcined coal gangue was used as aggregate and binder to prepare ultra-high performance coal gangue geopolymer concrete (UHPGC) in this paper. The hybrid reinforcement effect of steel fibers, calcium carbonate whiskers (CW), and carbon nanotubes (CNTs) on UHPGC was studied in the experiment. The results indicated that when the mixing ratio of straight steel fibers and hooked-end steel fibers was 2:1, the 28 d-compressive strength of UHPGC reached 145.94 MPa. However, excessive hooked-end steel fibers deteriorated the reinforcement effect of adjacent fibers and reduced the mechanical properties of UHPGC. CW and CNTs had a more significant improvement in the flexural properties of UHPGC. The bridging effect of CNTs and CW dispersed the stress on the matrix and delayed the formation and development of micro-nano cracks. Meanwhile, CNTs and CW improved the fiber bridging energy dissipation, thereby improving the flexural properties of UHPGC. The UHPGC had the highest flexural strength and flexural deflection of 13.59 MPa and 4.12 mm, respectively. The UHPGC prepared in this paper had lower carbon emissions and economic costs compared to traditional UHPC. Overall, this study provided valuable references for the resource utilization of coal gangue and the multi-scale strengthening and toughening of UHPGC.