As the inflow velocity increases, rocket-based combined cycle(RBCC) engine typically require a ramjet/scramjet mode transition within the flight range Ma∞ = 5.5 to 6.0. However, during the mode transition process, the reduction of the fuel mass flow rate usually causes the combustion intensity in the engine to drop sharply in a very short time, which will further lead to severe thrust fluctuations. So some technical methods need to be applied in this process to realize smooth mode transition. In this paper, different throttling strategies (kerosene mass flow rate dominant and rocket mass flow rate dominant) were designed and employed to study the dynamic mode transition process in the integrated RBCC engine flow path through three-dimensional numerical simulation for flight Ma∞ = 5.5 and Ma∞ = 6.0, respectively. The results show that when RBCC engine operates under the condition of large mass flow rate of fuel, the wall flow separation zone caused by oblique shock wave and combustion cannot be effectively improved by the bleed block and the inlet is still in the limit working state. Under flight Ma∞ = 5.5 condition, with the reduction of fuel equivalence ratio, RBCC engine successfully transited from ramjet mode to scramjet mode, and the flow separation zone near the bleed block disappeared. At this time, the high temperature gas of the rocket has no obvious effect on the thrust gain of the engine. By adjusting fuel mass flow rate in a small range and rocket mass flow rate in a large range, stable mode transition can be achieved, with relatively small time consumption and minimum thrust fluctuation during mode transition. With the increase of flight altitude, under flight Ma∞ = 6.0, the incoming flow speed increases, but the air capture ability of the inlet decreases, which is not conducive to the mixing of fuel and air combustion. When Ma∞ = 5.5, the mode transition by reducing the fuel equivalence ratio will cause the RBCC engine to produce huge thrust fluctuations, and the range of the high-pressure zone in the combustor will be greatly reduced. In order to ensure the stable operation of the RBCC engine, the rocket needs to operate at a larger mass flow rate.