Using finite-temperature strong-coupling expansions for the SU($N$) Hubbard model, we calculate the thermodynamic properties of the model in the infinite-$U$ limit for arbitrary density $0\ensuremath{\le}\ensuremath{\rho}\ensuremath{\le}1$ and all $N$. We express the ferromagnetic susceptibility of the model as a Curie term plus $\mathrm{\ensuremath{\Delta}}\ensuremath{\chi}$, an excess susceptibility above the Curie behavior. We show that, on a bipartite lattice, graph by graph the contributions to $\mathrm{\ensuremath{\Delta}}\ensuremath{\chi}$ are non-negative in the limit that the hole density $\ensuremath{\delta}=1\ensuremath{-}\ensuremath{\rho}$ goes to zero. By summing the contributions from all graphs consisting of closed loops we find that the low-hole-density ferromagnetic susceptibility diverges exponentially as $exp\mathrm{\ensuremath{\Delta}}/T$ as $T\ensuremath{\rightarrow}0$ in two and higher dimensions. This demonstrates that the Nagaoka-Thouless ferromagnetic state exists as a thermodynamic state of matter at low enough density of holes and sufficiently low temperatures. The constant $\mathrm{\ensuremath{\Delta}}$ scales with the SU($N$) parameter $N$ as $1/N$ implying that ferromagnetism is gradually weakened with increasing $N$ as the characteristic temperature scale for ferromagnetic order goes down.