The research deals with a novel modeling and compensation approach for the transmission error in harmonic drive gearings. This paper, as the first report, presents a mathematical modeling framework for nonlinear components in the gearing, where nonlinear friction, nonlinear elastic characteristic, and angular transmission errors are handled to provide a precise numerical simulator. In the modeling of transmission errors, physical phenomena due to nonlinear elastic deformations in micro-displacement region are especially dealt with, as well as the synchronous component which has been already discussed in a variety of conventional studies. Based on the analyses of the phenomena, the nonlinear elastic component can be mathematically formulated by applying a modeling approach for the rolling friction with hysteresis attributes. Numerical simulations and experimental results using a prototype show the effectiveness of the proposed modeling.