Orientations, roughnesses, and cavities of crystals are typical factors influencing the servicing reliability of metals in corrosive environments. A phase-field scheme for modeling stress-corrosion coupled damage (SCCD) is developed. The effects of the crystal factors on SCCD are numerically simulated using the incremental-iterative scheme of the user-defined finite elements. The impacts of orientations, roughnesses, and cavities on the corrosion rate of magnesium (Mg) in corrosive environments are discussed quantitatively. It is found that crystal textures and surface roughnesses can significantly influence the diffusion-controlled corrosion rate. Strong basal texture and a smooth surface of the crystal can significantly enhance the corrosion resistance of Mg. The cavity, as a typical crystal defect, is capable of inducing the damage path and modulating the corrosion rate. The design of crystal-scale features, such as orientations, roughnesses, and cavities, is promising for the enhancement of the resistance to SCCD.