We advance superselection theory of pure states on a \(C^*\)-algebra \(\mathfrak {A}\) outside of the conventional (DHR) setting. First, we canonically define conjugate and second conjugate classes of such states with respect to a given reference state \(\omega _{\mathrm {vac}}\) and background \(a \in \mathrm {Aut}(\mathfrak {A})\). Next, for some subgroups \(R \varsubsetneq S \varsubsetneq G\subset \mathrm {Aut}(\mathfrak {A})\) we study the family \(\{\omega _{\mathrm {vac}}\circ s| s\in S \}\) of infrared singular states whose superselection sectors may be disjoint for different s. We show that their conjugate and second conjugate classes always coincide provided that R leaves the sector of \(\omega _{\mathrm {vac}}\) invariant and a belongs to the relative normalizer\(N_G(R,S):=\{g\in G| g\cdot S\cdot g^{-1}\subset R\}\). We study the basic properties of this apparently new group theoretic concept and show that the Kraus–Polley–Reents infravacuum automorphisms belong to the relative normalizers of the automorphism group of a suitable CCR algebra. Following up on this observation we show that the problem of velocity superselection in non-relativistic QED disappears at the level of conjugate and second conjugate classes, if they are computed with respect to an infravacuum background. We also demonstrate that for more regular backgrounds such merging effect does not occur.