This paper presents a series of systematic multi-passage unsteady RANS simulations on a transonic axial flow compressor rotor (Rotor 35). The objective is to have a better understanding of the underlying flow mechanism which connects the phenomena of the tip leakage vortex (TLV)'s breakdown to the appearance of flow unsteadiness. It has been revealed that both bubble-type and spiral-type breakdown of the TLV can result in a self-sustained flow unsteadiness at high-loading flow conditions. The origin of such unsteadiness lies in that the vorticity region redistributed by the vortex breakdown is capable of affecting the pressure distribution on the pressure side of a passage. Once this threshold event is met, the swirl intensity of the TLV and the strength of shock wave, which are key factors controlling the vortex breakdown, varies accordingly, thus leading to an instantaneous rather than a stationary vortex breakdown occurring in the confined rotor–passage system. As compared to the bubble-type breakdown of TLV, the spiral-type breakdown of TLV exerts more severe impact on the pressure distribution on the PS of the passage. As a result, a significant change in the scale of the breakdown region occurs. This gives rise to not only an appearance of a new vortex structure but also a blockage transfer across the passage against the rotor turning direction. The new vortex structure, termed as tip secondary vortex (TSV), is essentially a vortex segment arising from the spiral-type breakdown of TLV. The necessary condition for the inception of the rotating wave like RI is the blockage transfer induced by the spiral-type breakdown of TLV and its resultant interaction with the tip leakage in the adjacent passage.