The influence factors of the high compressive strength of fly ash based geopolymer concrete (FGPC) were studied rarely mentioned the generation of hydroxy sodalite before. In this paper, FGPC was prepared by using low-calcium fly ash and NaOH acted as the main component in the base activator. By comparing the compressive strengths of FGPC under different preparation parameters, the microstructures of cross-section and hydroxy sodalite (Na4Al3Si3O12OH) in FGPC were observed by SEM. The spectra information of FGPC were characterized by XRD and FT-IR. The compressive strength of FGPC was analyzed by studying the growth of hydroxy sodalite and the microstructure of the matrix section under different conditions. The results showed that increasing the concentration of NaOH solution, changing curing temperature and curing time in a certain range can effectively enhance the activity of fly ash and promote the generation of N-A-S-H gel and hydroxy sodalite. The compressive strength of FGPC was decided by matrix structure and the generation of hydroxy sodalite. When the ratio of Na2SiO3/NaOH is 1.7, the ratio of alkali activator/fly ash is 0.58, the compressive strength of the FGPC can reach 110.2 MPa by curing at 75 ℃ for 24 h after 28 days. The results can help to understand the mechanism of promotion of hydroxy sodalite and give a novelty reference to produce a high compressive strength FGPC.