Recycling waste steel slag for asphalt pavement construction can reduce energy consumption and preserve nonrenewable natural resources. However, the moisture susceptibility of steel slag asphalt mixture (SSAM) in severe freeze–thaw regions should be taken into consideration. In light of this, three types of fibers (basalt fiber, polyester fiber, and lignin fiber) and rubber powder were blended into the SSAMs, and their freeze–thaw damage characteristics were investigated. First, the optimum bitumen and fiber content of SSAMs were determined based on Marshall design using response surface methodology. Then, the air voids of SSAMs were identified through X-ray computed tomography (CT) technology, while the effects of the fiber and freeze–thaw cycles on the moisture stability of SSAMs were investigated. Moreover, the enhancement effects of the fibers were analyzed using scanning electron microscopy (SEM). The results show the optimum fiber content of different fiber-reinforced SSAMs are 0.35%, 0.29%, and 0.23%, respectively, and the optimum bitumen content is 6.7% for all cases. The effect of freezing–thawing leads to an increase of the air voids in the middle part of the specimen, whereas the incorporation of fibers weakens the adverse action of freezing–thawing and effectively improves the moisture stability of the SSAMs. Furthermore, the network structure formed by basalt fibers in asphalt mastic significantly improves the integrity of the SSAM. Additionally, basalt fiber reinforced SSAM shows the best resistance to moisture damage.