Stabilizing role of Zr is shown in single-phase fluorite type high entropy oxides (F-HEOs). The irreversible crystallographic symmetry lowering from fluorite to bixbyite upon high temperature calcination observed in Ce based F-HEOs is suppressed in the Zr based F-HEOs. Furthermore, replacing Zr by Sn/Tb results in biphasic compositions, another feature that portrays the dominant role of Zr. Moreover, the presence of cations like Pr/Tb in the Zr-F-HEOs allows for a substantial and fully reversible tuning of the band gap within 2 eV ↔ 4 eV. Essentially, the phase-purity of F-HEOs is retained irrespective of the variation in the opto-electronic characteristics. The absence of Pr/Tb results in a wide band gap of 5 eV. Thus, the study highlights that the roles of specific cations need to be critically considered, in addition to the synergistic or cocktail effects, to gain deeper insights into structure-property relations in chemically complex HEOs.