Let X be a ball Banach function space on \({\mathbb R}^{n}\). Let Ω be a Lipschitz function on the unit sphere of \({\mathbb R}^{n}\), which is homogeneous of degree zero and has mean value zero, and let TΩ be the convolutional singular integral operator with kernel Ω(⋅)/|⋅|n. In this article, under the assumption that the Hardy–Littlewood maximal operator \({\mathscr{M}}\) is bounded on both X and its associated space, the authors prove that the commutator [b, TΩ] is compact on X if and only if \(b\in \text {CMO }({\mathbb R}^{n})\). To achieve this, the authors mainly employ three key tools: some elaborate estimates, given in this article, on the norm of X about the commutators and the characteristic functions of some measurable subsets, which are implied by the assumed boundedness of \({\mathcal M}\) on X and its associated space as well as the geometry of \(\mathbb R^{n}\); the complete John–Nirenberg inequality in X obtained by Y. Sawano et al.; the generalized Fréchet–Kolmogorov theorem on X also established in this article. All these results have a wide range of applications. Particularly, even when \(X:=L^{p(\cdot )}({\mathbb R}^{n})\) (the variable Lebesgue space), \(X:=L^{\vec {p}}({\mathbb R}^{n})\) (the mixed-norm Lebesgue space), \(X:=L^{\Phi }({\mathbb R}^{n})\) (the Orlicz space), and \(X:=(E_{\Phi }^{q})_{t}({\mathbb R}^{n})\) (the Orlicz-slice space or the generalized amalgam space), all these results are new.