Ultrathin Si oxynitride layers were examined by using scanning tunneling microscopy (STM) and spectroscopy (STS). These techniques revealed that a structural change from an intrinsic defect (Si–Si bond) to a damaged structure (Si cluster) takes place under conventional STM/STS conditions. Comparison of the damaged structures formed in the oxynitride with those in the oxide indicated that nitrogen atoms suppress the expansion of the damaged regions. It was also found that nitrogen incorporation enhances both the defect density and the atomic-scale roughness at the oxynitride/Si interface. We suggested that this degradation is related to a local strain produced by the N≡Si3 structures at the oxynitride/Si interface. On the contrary, a normal oxynitride structure had a higher resistance to an electrical stress than an intrinsic defect, but, when the constant electrical stress was applied, the normal oxynitride structure was also damaged. This damage proceeds in two steps: creation of charge traps, and then formation of Si cluster. From these STM/STS results, we proposed that the electrical breakdown of the conventional gate-oxide film proceeds as a four-step process: (1) formation of Si clusters by the damage of intrinsic defects, (2) creation of traps in the normal structure, (3) formation of Si clusters in the normal structure, and (4) complete local breakdown when the Si clusters become connected.