The segregation kinetics of surface-active, residual elements are investigated in an in situ study of annealing scrap-based silicon electrical steel sheet where the arsenic (As) surface segregation is highlighted. During annealing in the temperature range of 300–950 °C, different kinds of interactions between the segregated residual elements were observed. Attractive interactions between the segregands produced co-segregation, e.g., between Sn and Sb, whereas repulsive interactions resulted in site competition, e.g., between Sn and As. These competing interactions are strongly time dependent. In spite of there being twice as much Sn compared to As in the bulk material, the As prevailed in the surface enrichments of the polycrystalline silicon steel at 950 °C. The intensity of the As surface segregation in the temperature range 800–950 °C is proportional to the calculated amount of γ-austenite phase in the (α + γ) steel matrix. The detected phenomenon of the As versus Sn site competition could be valuable for the texture design and surface engineering of silicon steels with a thermodynamically stable two-phase (α + γ) region.