We study nonlinear Drude weights (NLDWs) for the spin-1/2 XXZ chain in the critical regime at zero temperature. The NLDWs are generalizations of the linear Drude weight. Via the nonlinear extension of the Kohn formula, they can be read off from higher-order finite-size corrections to the ground-state energy in the presence of a $U(1)$ magnetic flux. The analysis of the ground-state energy based on the Bethe ansatz reveals that the NLDWs exhibit convergence, power-law, and logarithmic divergence, depending on the anisotropy parameter $\Delta$. We determine the convergent and power-law divergent regions, which depend on the order of the response $n$. Then, we examine the behavior of the NLDWs at the boundary between the two regions and find that they converge for $n=0, 1, 2$ $({\rm mod}~4)$, while they show logarithmic divergence for $n=3$ $({\rm mod}~4)$. Furthermore, we identify particular anisotropies $\Delta=\cos(\pi r/(r+1))$ ($r=1,2, 3,\ldots$) at which the NLDW at any order $n$ converges to a finite value.