We present the magnetic structure of $\mathrm{TmMn_3O_6}$, solved via neutron powder diffraction - the first such study of any $R\mathrm{Mn_3O_6}$ A-site columnar-ordered quadruple perovskite to be reported. We demonstrate that long range magnetic order develops below 74 K, and at 28 K a spin-flop transition occurs driven by $f$-$d$ exchange and rare earth single ion anisotropy. In both magnetic phases the magnetic structure may be described as a collinear ferrimagnet, contrary to conventional theories of magnetic order in the manganite perovskites. Instead, we show that these magnetic structures can be understood to arise due to ferro-orbital order, the A, A$'$ and A$''$ site point symmetry, $mm2$, and the dominance of A-B exchange over both A-A and B-B exchange, which together are unique to the $R\mathrm{Mn_3O_6}$ perovskites.