Nonlinear integral positive position feedback (NIPPF) and adaptive nonlinear integral positive position feedback (ANIPPF) controllers are used in a shearer's semi-direct drive cutting transmission system, which is powered by a low speed and high torque permanent magnet synchronous motor (PMSM). The transmission system's torsional vibration characteristics were investigated with internal parameter and external cutting load excitation changes caused by harmonic force. In the case of primary and 1:1 internal resonance, an analytic solution is obtained by employing the averaging technique to solve the nonlinear differential equations and modeling the system with NIPPF and ANIPPF controllers. The Poincare portraits were created to imagine how the PMSM would behave around their fixed points. A good match is found when the approximate solutions are compared to the numerical simulations via the Runge–Kutta method fourth-order (RK4). The linearized stability strategy is used to ensure stability near fixed points in the autonomous system. Before and after control, the steady-state amplitude and stability of nonlinear systems were investigated and analyzed. Optimal system operation conditions and frequency response curves (FRCs) were examined at various controller and system parameter values. Numerous system characteristics were found to have an effect after being combined with NIPPF and ANIPPF. The numerical and analytical solutions at time-history and FRCs were compared using the MATLAB program to confirm their comparability. This study's findings were compared to those found in the literature. We conclude from this analysis that the ANIPPF controller provides the best system control. The calculations also revealed that ANIPPF is the most effective controller. Finally, the controls suppress the system's vibration amplitude and chaos motion. The outcomes of this study were used to develop a theoretical foundation for the shearer semi-direct drive cutting transmission system's design and construction.