This study aims to investigate the intricate dynamic characteristics of the high-speed duct during the over-under Turbine-Based Combined Cycle TBCC inlet mode transition process while operating in an off-design state under throttled conditions. A typical over-under TBCC inlet, designed for a working Mach number range of 0–6 with a transition Mach number of 3.5, is examined through experimental studies in a supersonic wind tunnel with a freestream Mach number of 2.9. The investigation focuses on the complex oscillatory flow and unique hysteresis observed in the mode transition process of the high-speed duct under the mildly throttled condition, utilizing high-speed Schlieren and dynamic pressure acquisition system. The findings reveal that the high-speed duct undergoes four distinct oscillation stages akin to those in a higher throttled state during the mode transition, albeit with smaller dominant frequency and energy. Moreover, an irregular alternating “big/little buzz” mode is observed in the early stage of the large oscillation stage. Notably, the mildly throttled state exhibits three intriguing hysteresis properties compared to the unthrottled and higher throttled states. Firstly, hysteresis is observed in the shock train motion stage in the duct before unstart, along with the corresponding inverse process. Subsequently, hysteresis is noted in the unstart and restart of the high-speed duct, with a smaller hysteresis interval than in the unthrottled state. Finally, the hysteresis characteristics of oscillation mode switching and the corresponding inverse process are explored. Based on the analysis, the first two hysteresis phenomena are associated with the formation and dissipation of the separation bubble. The significant adverse pressure gradient constrains the cross-sectional capacity of the channel, rendering the disappearance of the separation bubble more challenging. The hysteresis in oscillation mode switching is linked to not only the channel cross-sectional capacity but also the state of the incoming boundary layer.