Recent studies (arXiv:1610.07916, arXiv:1711.07921, arXiv:1807.00186) of six-dimensional supersymmetric gauge theories that are engineered by a class of toric Calabi-Yau threefolds $X_{N,M}$, have uncovered a vast web of dualities. In this paper we analyse consequences of these dualities from the perspective of the partition functions $\mathcal{Z}_{N,M}$ (or the free energy $\mathcal{F}_{N,M}$) of these theories. Focusing on the case $M=1$, we find that the latter is invariant under the group $\mathbb{G}(N)\times S_N$: here $S_N$ corresponds to the Weyl group of the largest gauge group that can be engineered from $X_{N,1}$ and $\mathbb{G}(N)$ is a dihedral group, which acts in an intrinsically non-perturbative fashion and which is of infinite order for $N\geq 4$. We give an explicit representation of $\mathbb{G}(N)$ as a matrix group that is freely generated by two elements which act naturally on a specific basis of the K\"ahler moduli space of $X_{N,1}$. While we show the invariance of $\mathcal{Z}_{N,1}$ under $\mathbb{G}(N)\times S_N$ in full generality, we provide explicit checks by series expansions of $\mathcal{F}_{N,1}$ for a large number of examples. We also comment on the relation of $\mathbb{G}(N)$ to the modular group that arises due to the geometry of $X_{N,1}$ as a double elliptic fibration, as well as T-duality of Little String Theories that are constructed from $X_{N,1}$.