Due to the significant temperature difference and high solar radiation in high-altitude areas, the asphalt pavement has been in freeze-thaw cycles and radiation service conditions for a long time, resulting in a shorter service life. To extend the service life of asphalt pavement in high-altitude areas, this study uses basalt fiber to toughen the asphalt mixture, and the service characteristics of the asphalt pavement under the coupled effects of freeze-thaw cycles and aging were evaluated. First, the freeze-thaw cycles, aging time, and basalt fiber content were selected as independent variables to conduct the Box–Behnken design (BBD) test design. Consequently, the corresponding response surface model was established. Based on the rutting test, three-point bending test, and indirect tensile test, the optimum content of basalt fiber was obtained. Finally, the fatigue properties of the asphalt mixture by the four-point bending fatigue test were compared under the optimum content. The results showed the consequences of freeze-thaw cycles will reduce the road properties of asphalt and give rise to damage to the asphalt; among them, the indirect tensile strength was decreased by 21.7% at most (the number of freeze-thaw cycles from 10 to 30). Aging will reduce the fatigue life of the asphalt mixture, but its dynamic stability, flexural tensile strength, and slitting strength will be enhanced to a certain extent; among them, the dynamic stability can be improved by up to 28.1% (aging from 4 to 12 h). Basalt fiber can increase the flexural tensile strength, dynamic stability, and spitting strength; the optimum content is 0.35% of asphalt mixtures. Furthermore, the fatigue life of basalt fiber-reinforced asphalt mixture can be significantly improved, which can reduce the influence of freeze-thaw cycles and aging. In addition, the unit price of basalt fiber-reinforced asphalt mixture is only 6.9% higher than that of SBS-modified asphalt mixture, which has significant economic benefits. Under experimental conditions, the effect of aging on the properties of the asphalt mixture is greater than that of the freeze-thaw cycle. This research shows that careful consideration of the road behavior of basalt fiber-reinforced asphalt mixture under freeze-thaw cycles and aging is essential for the pavement durability design.