All-optical spin switching (AOS) represents a new frontier in magnetic storage technology -- spin manipulation without a magnetic field, -- but its underlying working principle is not well understood. Many AOS ferrimagnets such as GdFeCo are amorphous and renders the high-level first-principles study unfeasible. The crystalline half-metallic Heusler Mn$_2$RuGa presents an opportunity. Here we carry out hitherto the comprehensive density functional investigation into the material properties of Mn$_2$RuGa, and introduce two concepts - the spin anchor site and the optical active site - as two pillars for AOS in ferrimagnets. In Mn$_2$RuGa, Mn$(4a)$ serves as the spin anchor site, whose band structure is below the Fermi level and has a strong spin moment, while Mn$(4c)$ is the optical active site whose band crosses the Fermi level. Our magneto-optical Kerr spectrum and band structure calculation jointly reveal that the delicate competition between the Ru-$4d$ and Ga-$4p$ states is responsible for the creation of these two sites. These two sites found here not only present a unified picture for both Mn$_2$RuGa and GdFeCo, but also open the door for the future applications. Specifically, we propose a Mn$_2$Ru$_x$Ga-based magnetic tunnel junction where a single laser pulse can control magnetoresistance.