In this paper, nonlinear dynamic analysis and experimental verification of a flexible rotor supported on the active magnetic bearings (AMB) are studied. The model of the system is formulated by eight degrees of freedom. This model takes in to account the gyroscopic moments of the disk, geometric coupling of the magnetic actuators and contact forces of the auxiliary bearings. The equations of motion are solved using the Rung–Kutta method. The effects of auxiliary bearings stiffness and rotational speed on the dynamic behavior of the system are investigated by the bifurcation diagrams, dynamic trajectories, power spectra analysis, Poincare´ maps and maximum Lyapunov exponent. In the experimental test rig, two special flexible supports are constructed that can adjust the required stiffness of the auxiliary bearings. The results indicate that the auxiliary bearings stiffness and rotational speed have significant effects on the dynamic responses and can be used as effective control parameters for the system. Good correlation between the experimental and theoretical results is found. Very rich forms of periodic, quasi-periodic, period −4 and chaotic vibrations are observed. The present study can be useful in designing and selection of suitable operating parameters. As a result, the system can avoid the undesirable behavior