Steel slag, an industrial by-product during the steelmaking process, has not been effectively utilized so far. In this paper, the feasibility of using basic oxygen furnace (BOF) steel slag aggregate in metakaolin-based geopolymer was systematically evaluated. Tests results showed the use of steel slag aggregate (SSA) as full replacement of natural limestone aggregate could yield higher volume stability, improved compressive strength, smaller drying shrinkage and enhanced anti-freeze property of geopolymer concrete. Moreover, the activator of water glass was demonstrated to be superior to NaOH because free oxides in steel slag could be consumed in water glass solution. Discussion on the mechanism behind the superiority of BOF SSA in conjunction with alkalis was elaborated from the perspective of chemical reaction. The geopolymer matrix and interfacial transition zone (ITZ) were enhanced due to the diffusion of Ca ions to form the gel and densify the microstructure. Techniques of XRD, TGA, SEM/EDX were utilized for the analysis of the improved geopolymer matrix and strengthened ITZ. This research could facilitate the recycling and upgrading of steel slag for construction.