We study the infinite-U Hubbard model on ladders of 2, 4 and 6 legs with nearest (t) and next-nearest (t') neighbor hoppings by means of the density-matrix renormalization group algorithm. In particular, we analyze the stability of the Nagaoka state for several values of t' when we vary the electron density $(\rho)$ from half-filling to the low-density limit. We build the two-dimensional phase diagram, where the fully spin-polarized and paramagnetic states prevail. We find that the inclusion of a non-frustrating next nearest neighbor hopping stabilizes the fully spin-polarized phase up until |t'/t|=0.5. Surprisingly, for this value of t', the ground state is fully spin-polarized for almost any electron density 1 $\gtrsim \rho \gtrsim$ 0, connecting the Nagaoka state to itinerant ferromagnetism at low density. Also, we find that the previously found checkerboard insulator phase at t'=0 and $\rho$=0.75 is unstable against t'.