The deterioration of different phases under sulfate attack is a critical durability problem for cementitious materials. Alkali-activated slag (AAS) is a kind of low carbon footprint cementitious material, whose hydration products are quite different from that of Portland cement (PC). The specific variations in the behavior of different hydration products under sulfate attack in PC and AAS have not been comprehensively explored. In this paper, a combination of BSE and EDS techniques was applied to investigate the change of various hydration products in PC and AAS before and after exposure to sulfate solutions. The results indicate that the unhydrated slag in AAS is more stable than the unhydrated C2S and C3S in PC paste after exposure to deionized water and sulfate solutions, and obvious substitution of Ca by Mg can be found for C2S and C3S after exposure to MgSO4 solution. In PC sample, the Ca/Si reduction of outer product after deterioration in different solutions is less significant than that of inner product, mainly due to the existing of Ca rich phases intermixed in Op. While in AAS, the change of Ca/Si in dark rims in different solutions is consistent with that of C-A-S-H presented in the matrix. After exposure to sulfate solutions, no significant sulfate rich area can be observed for AAS, which should mainly be attributed to the relatively slower formation of gypsum in AAS due to the absence of highly soluble calcium-rich phases like portlandite in PC.