The nonlinear vibration of a flexible hoisting rope with time-varying length is investigated. The governing equations of the flexible hoisting rope are developed based on Hamilton’s principle. Experiments performed evaluated the theoretical model and found that the experimental data agree well with the theoretical prediction, which validates the mathematical model of the flexible hoisting system. The results of the simulations and experiments show that the flexible hoisting system dissipates energy during downward movement (thus is stabilized) and gains energy during upward movement (thus is unstabilized). In addition, a passage through resonance in the hoisting system with periodic external excitation is analyzed. Due to the time-varying length of the hoisting rope the natural frequencies of the system vary slowly, and transient resonance may occur when one of the frequencies coincides with the frequency of an external excitation.