There is a great influence of the stability of a rotor system filled with fluid on the performance of the rotor. Wave resonance theory and the model established by Wolf are applied to study the stability boundaries. First, the resonant frequencies of the radial–circular waves on the nonviscous, incompressible fluid are obtained in a rotor with radial baffles. Based on the Navier–Stokes equations of the fluid, a simple form of the Bessel equation is derived by the perturbation method. Then, the relationship of the radial–circular wave frequencies and the rotation frequencies is obtained. Furthermore, the unstable regions under varying modes are predicted, and the effects of the fluid-fill ratio on the unstable regions are analyzed. In order to verify the accuracy of this model, a comparison is made with the model by Wolf. The results show that two lower boundaries of the unstable regions are in good agreement, while the upper boundaries do not coincide with the internal resonance when the baffle is equal to 2. The mechanism of the stability of a rotor filled with fluid is revealed in the case of the chamber without baffles.