Abstract The standard stabilizer formalism provides a setting to show that quantum computations restricted to operations within the Clifford group are classically efficiently simulable: this is the content of the well-known Gottesman–Knill theorem. This work analyzes the mathematical structure behind this theorem to find possible generalizations and derivation of constraints required for constructing a non-trivial generalized Clifford group. We prove that if the closure of the stabilizing set is dense in the set of SU(d) transformations, then the associated Clifford group is trivial, consisting only of local gates and permutations of subsystems. This result demonstrates the close relationship between the density of the stabilizing set and the simplicity of the corresponding Clifford group. We apply the analysis to investigate stabilization with binary observables for qubits and find that the formalism is equivalent to the standard stabilization for a low number of qubits. Based on the observed patterns, we conjecture that a large class of generalized stabilizer states are equivalent to the standard ones. Our results provide better insights into the structure of Gottesman–Knill-type results, consequently allowing us to draw a sharper line between quantum and classical computation.