The hydration reaction products of Portland cement are primarily responsible for all the characteristics of concrete. When sulfate agents attack the reaction products, their chemical and microstructural alterations are highly complex, necessitating fundamental knowledge across various length–time scales. In this study, nanomechanical properties were incorporated into microstructural and chemical analyses at varying sulfate exposure levels to investigate the sulfate-driven deterioration of Portland cement paste. The nanomechanical characteristics were observed using a nanoindentation test following cement paste was exposed to MgSO4. Microstructural and chemical analyses of the impact of sulfate exposure on the hydration products were performed. According to the statistical deconvolution of the nanoindentation data, the elastic modulus of the calcium silicate hydrate (C-S-H) gel progressively declined with increasing exposure period to MgSO4 solution according to the statistical deconvolution of the nanoindentation data. In contrast, the volume percentage of pores gradually increased. The changes in the elastic modulus were linearly related to the Ca/Si molar ratio of the C-S-H gel. New phases and microcracks due to exposure to MgSO4 were observed. The experimental research undertaken in this study suggested that one of the primary reasons for the decrease in the macroscopic characteristics of cement concretes when they encounter MgSO4 is the decalcification of C-S-H gel, which is mechanically, morphologically, and chemically described.