In this paper, the decomposition and retrieval of the chirp modes with dynamic cross, appearance, and/or disappearance in a signal are formulated as a process of tracking time-frequency (TF) trajectories. Analyses show that the amplitude and frequency evolutions of each chirp mode in the signal can be described by polynomial prediction models. When all these evolutions are viewed as state equations, it is illustrated that the random appearance and/or disappearance of these modes will make the state equations into a state random finite set (RFS). If the amplitudes and frequencies obtained by the short-time spectrum of the signal are taken as the measurements of the state equations, a measurement RFS model in the presence of data association and detection uncertainties is coined when the statistical properties of the existing and newborn chirp modes as well as the noise/clutter are taken into consideration. So, a RFS TF model for tracking the TF trajectories in a signal is established. It is then demonstrated that a modified probability hypothesis density filter can be used to infer the amplitude and frequency posterior distributions of the chirp modes in the signal when the intensity of the RFS states is modelled as a Gaussian mixture. Numerical simulation experiments verify the effectiveness of our model and analytical results. Meanwhile, a superior performance is achieved especially for the high readability, extractability, and decomposability of instantaneous frequency trajectories with dynamic cross, appearance, and disappearance in a signal.