We describe the singular locus of the compactification of the moduli space Rg,` of curves of genus g paired with an `-torsion point in their Jacobian. Generalising previous work for ` ≤ 2, we also describe the sublocus of noncanonical singularities for any positive integer `. For g ≥ 4 and ` = 3, 4, 6, this allows us to provide a lifting result on pluricanonical forms playing an essential role in the computation of the Kodaira dimension of Rg,`: for those values of `, every pluricanonical form on the smooth locus of the moduli space extends to a desingularisation of the compactified moduli space. The modular curve X1(`) := H/Γ1(`) classifying elliptic curves together with an `-torsion point in their Jacobian is among the most studied objects in arithmetic geometry. In a series of recent papers, the birational geometry of its higher genus generalisations and their variants (e.g. theta characteristics) has been systematically studied and proved to be, in many cases such as ` = 2, better understandable than that of the underlying moduli space of curves Mg. As an example, we refer to the complete computation of the Kodaira dimension of all components of the moduli of theta characteristics (L⊗2 ∼= ω), see [23, 14, 16, 17]. In this paper, for g ≥ 2 and for all positive levels `, we consider the moduli space Rg,` parametrizing level-` curves, i.e. triples (C,L, φ) where C is a smooth curve equipped with a line bundle L and a trivialisation φ : L⊗` ∼ −→ O. The Kodaira dimension of Rg,` is defined as the Kodaira dimension of an arbitrary resolution of singularities of a completion; therefore, as a first step toward the birational classification of Rg,`, we consider a natural compactification Rg,` and study the singular locus Sing(Rg,`). More precisely one needs to determine the sublocus Singnc(Rg,`) ⊆ Sing(Rg,`) of noncanonical singularities. For ` = 2, this analysis has been carried out by the second author and Ludwig in [15] using Cornalba’s compactification in terms of quasistable curves [11] of Rg,2. Clearly, we can leave out the case ` = 1, which coincides with Deligne and Mumford’s functor of stable curves Mg = Rg,1. The passage to all higher levels presents a new feature from Abramovich and Vistoli’s theory of stable maps to stacks: the points of the compactification cannot be interpreted in terms of `-torsion line bundles on a scheme-theoretic curve, but rather on a stack-theoretic curve. Instead of the above triples (C,L ∈ Pic(C), φ : L⊗` ∼ −→ O), we simply consider their stack-theoretic analogues (C, L ∈ Pic(C), φ : L⊗` ∼ −→ O) ∈ Rg,`, where C is a one-dimensional stack, whose nodes may have nontrivial stabilisers μr ⊆ μ`, and where L→ C is a line bundle whose fibres are faithful representations, see Definition 1.5. This yields a compactification which is represented by a smooth Deligne–Mumford stack. Research of the first author partially supported by the ANR grant TheorieGW. Research of the second author partially supported by the Sonderforschungsbereich 647 Raum-Zeit-Materie.