We construct the effective field theory for time-reversal symmetry breaking multi-Weyl semimetals (mWSMs), composed of a single pair of Weyl nodes of (anti-)monopole charge $n$, with $n=1,2,3$ in crystalline environment. From both the continuum and lattice models, we show that a mWSM with $n>1$ can be constructed by placing $n$ flavors of linearly dispersing simple Weyl fermions (with $n=1$) in a bath of an $SU(2)$ non-Abelian static background gauge field. Such an $SU(2)$ field preserves certain crystalline symmetry (four-fold rotational or $C_4$ in our construction), but breaks the Lorentz symmetry, resulting in nonlinear band spectra (namely, $E \sim (p^2_x + p^2_y)^{n/2}$, but $E \sim |p_z|$, for example, where momenta ${\bf p}$ is measured from the Weyl nodes). Consequently, the effective field theory displays $U(1) \times SU(2)$ non-Abelian anomaly, yielding anomalous Hall effect, its non-Abelian generalization, and various chiral conductivities. The anomalous violation of conservation laws is determined by the monopole charge $n$ and a specific algebraic property of the $SU(2)$ Lie group, which we further substantiate by numerically computing the regular and "isospin" densities from the lattice models of mWSMs. These predictions are also supported from a strongly coupled (holographic) description of mWSMs. Altogether our findings unify the field theoretic descriptions of mWSMs of arbitrary monopole charge $n$ (featuring $n$ copies of the Fermi arc surface states), predict signatures of non-Abelian anomaly in table-top experiments, and pave the route to explore anomaly structures for multi-fold fermions, transforming under arbitrary half-integer or integer spin representations.