In the field of beam pointing and optoelectronic imaging, there is a growing demand for control mechanisms that offer large stroke, and high bandwidth. These are precisely the characteristics of hybrid reluctance actuator, which therefore is increasingly being used in XY stage to replace piezo-actuator and voice coil motor. This study presents a hollow XY stage utilizing hybrid resistance actuators, aiming to adapt to optical applications with inherent nonlinearity and biaxial coupling, which are further modeled and compensated to mitigate their impacts. Firstly, the basic composition and working principle of the hollow XY stage are described, and the impact of its nonlinearity and biaxial coupling characteristics is elucidated. Secondly, the sources of nonlinearity and biaxial coupling are analyzed, and an analytical model of hybrid reluctance actuators is established. The magnetic leakage coefficient of the model is determined, and the accuracy of the model is verified by Finite Element Method (FEM) and experiment, demonstrating consistency with the analytical model's results. Furthermore, based on the nonlinear and biaxial coupling model, a compensator based on feedback linearization method is established, and verified through experiment. The results show that this method has a significant suppression effect on nonlinear and biaxial coupling, and the tracking error and cross-talk is obviously reduced, which proves the effectiveness of this compensation method.