In recent years, there has been a growing demand in the field of engineering systems for research on the vibration response of motor-gear systems under non-steady state conditions. However, conventional modeling and simulation techniques are primarily restricted to systems within a single energy domain. When dealing with intricate electromechanical coupling systems, the modeling process becomes increasingly intricate. Consequently, this study adopts bond graph theory as the foundation for analyzing the vibration characteristics of motor-gear coupling systems. In addressing the vibration issue of the motor-gear transmission system, a nonlinear bond graph model is established that takes into account the time-varying mesh stiffness, static errors, and tooth surface friction, among other nonlinear factors. Then, according to the coupling relationship between motor mechanical systems and electrical systems, the electromechanical coupling dynamic model of the motor-gear system is established using the method of bond graph. In addition, the state equation of the system is derived based on the bond graph model. Finally, the system is simulated using 20-sim software, and the electromechanical coupled vibration characteristics of the simulation results are analyzed. This article lays the foundation for the design and analysis of motor-gear coupling systems and multi-energy domain complex systems.