In this paper, it is found that the systems alternate between oscillatory and steady states in studying the stability of a class of stochastic dynamical systems with fast time-varying periodic delays. We propose a new definition of bifurcation, State Flip (SF) bifurcation, by using the abrupt change in the dwell time ratio of two states. Taking the amplitude of the time-varying delay function δ as the bifurcation parameter, we investigate the dynamics of the evolution of a time-varying delay system driven by Gaussian white noise. Firstly, we convert systems with fast time-varying periodic delays into comparative systems with distributed delays, which in turn are converted into stochastic systems with constant delays for discussion. In order to reduce the stochastic delay equation to the average Itô equation, we adopted the stochastic averaging method and the generalized central manifold theory. It is used to capture the one-dimensional slow variables of a system near the critical state of the transition between two steady states during the evolution of the system. We then find that as the amplitude increases from weak to strong, the stochastic time-varying delay system changes from a two-period oscillation to alternating between oscillation and steady, and finally to a steady state. Lastly, we perform numerical simulations of the original stochastic time-varying delay system and the transformed two-dimensional stochastic constant time-delay system respectively to verify that the ideas and methods in this paper are reasonable and effective.