We analyze orbits of stars and dark matter out to three effective radii for 42 galaxies formed in cosmological zoom simulations. Box orbits always dominate at the centers and $z$-tubes become important at larger radii. We connect the orbital structure to the formation histories and specific features (e.g. disk, counter-rotating core, minor axis rotation) in two-dimensional kinematic maps. Globally, fast rotating galaxies with significant recent in situ star formation are dominated by $z$-tubes. Slow rotators with recent mergers have significant box orbit and $x$-tube components. Rotation, quantified by the $\lambda_R$-parameter often originates from streaming motion of stars on $z$-tubes but sometimes from figure rotation. The observed anti-correlation of $h_3$ and $V_0 / \sigma$ in rotating galaxies can be connected to a dissipative formation history leading to high $z$-tube fractions. For galaxies with recent mergers in situ formed stars, accreted stars and dark matter particles populate similar orbits. Dark matter particles have isotropic velocity dispersions. Accreted stars are typically radially biased ($\beta \approx 0.2 - 0.4$). In situ stars become tangentially biased (as low as $\beta \approx -1.0$) if dissipation was relevant during the late assembly of the galaxy. We discuss the relevance of our analysis for integral field surveys and for constraining galaxy formation models.