A nonlinear time-varying dynamic model for right-angle gear pair systems, considering both backlash and asymmetric mesh effects, is formulated. The mesh parameters that are characteristically time-varying and asymmetric include mesh stiffness, directional rotation radius and mesh damping. The period-one dynamic motions are obtained by solving the dimensionless equation of gear motion using an enhanced multi-term harmonic balance method (HBM) with a modified discrete Fourier Transform process and the numerical continuation method. The accuracy of the enhanced HBM solution is verified by comparison of its results to the more computational intensive, direct numerical integration calculations. Also, the Floquet theory is applied to determine the stability of the steady-state harmonic balance solutions. Finally, a set of parametric studies are performed to determine quantitatively the effects of the variation and asymmetry in mesh stiffness and directional rotation radius on the gear dynamic responses.