Magnets with frustration often show accidental degeneracies, characterized by a large classical ground-state space (CGSS). Quantum fluctuations may ``select'' one of these ground states---a phenomenon labeled ``order by (quantum) disorder'' in literature. In this article, we examine the mechanism(s) by which such state selection takes place. We argue that a magnet, at low energies, maps to a particle moving on the CGSS. State selection corresponds to localization of the particle at a certain point on this space. We distinguish two mechanisms that can bring about localization. In the first, quantum fluctuations generate a potential on the CGSS space. If the potential has a deep enough minimum, then the particle localizes in its vicinity. We denote this as ``order by potential'' (ObP). In the second scenario, the particle localizes at a self-intersection point due to bound-state formation---a consequence of geometry and quantum interference. Following recent studies by the present authors, we denote this scenario as ``order by singularity'' (ObS). In either case, localization leads to an energy gap between the ground state(s) and higher-energy states. This pseudo-Goldstone gap behaves differently in the two mechanisms, scaling differently with the spin length. We place our discussion within the context of the one-dimensional spin-$S$ Kitaev model. We map out its CGSS which grows systematically with increasing system size. It resembles a network where the number of nodes increases exponentially. In addition, the number of wires that cross at each node also grows exponentially. This self-intersecting structure leads to ObS, with the low-energy physics determined by a small subset of the CGSS, consisting of ``Cartesian'' states. A contrasting picture emerges when an additional XY antiferromagnetic coupling is introduced. The CGSS simplifies dramatically, taking the form of a circle. Spin wave fluctuations generate a potential on this space, giving rise to state selection by ObP under certain conditions. Apart from contrasting ObS and ObP, we discuss the possibility of ObS in macroscopic magnets.