Most iron-based superconductors undergo a transition to a magnetically ordered state characterized by staggered stripes of parallel spins. With ordering vectors $(\pi,0)$ or $(0,\pi)$, this magnetic state breaks the high-temperature tetragonal symmetry of the system, which is manifested by a splitting of the lattice Bragg peaks. Remarkably, recent experiments in hole-doped iron arsenides reported an ordered state that displays magnetic Bragg peaks at $(\pi,0)$ and $(0,\pi)$ but remains tetragonal. Despite being inconsistent with a magnetic stripe configuration, this unusual magnetic phase can be described in terms of a double-$\mathbf{Q}$ magnetic structure consisting of an equal-weight superposition of the ordering vectors $(\pi,0)$ and $(0,\pi)$. Here we show that a non-collinear double-$\mathbf{Q}$ magnetic configuration, dubbed \emph{orthomagnetic}, arises naturally within an itinerant three-band microscopic model for the iron pnictides. In particular, we find that strong deviations from perfect nesting and residual interactions between the electron pockets favor the orthomagnetic over the stripe magnetic state. Using an effective low-energy model, we also calculate the spin-wave spectrum of the orthomagnetic state. In contrast to the stripe state, there are three Goldstone modes, manifested in all diagonal and one off-diagonal component of the spin-spin correlation function. The total magnetic structure factor displays two anisotropic spin-wave branches emerging from both $(\pi,0)$ and $(0,\pi)$ momenta, in contrast to the case of domains of stripe order, where only one spin-wave branch emerges from each momentum. We propose that these unique features of the orthomagnetic state can be used to unambiguously distinguish it from the stripe state via neutron scattering experiments, and discuss the implications of its existence to the nature of the magnetism of the iron arsenides.