In real application, when rotary machinery frequently involves variable-speed, unsteady load and defect, it will produce non-stationary vibration signal. Such signal can be characterized by mono- or multi-component frequency modulation (FM) and its internal instantaneous patterns are closely related to operation condition of the rotary machinery. For example, instantaneous frequency (IF) and instantaneous amplitude (IA) of a non-stationary signal are two important time–frequency features to be inspected. For vibration signal analysis of the rotary machinery, time–frequency analysis (TFA), known for analyzing the signal in the time and frequency domain simultaneously, has been accepted as a key signal processing tool. Particularly, parameterized TFA, among various TFAs, has shown great potential to investigate time–frequency features of non-stationary signals. It attracts more attention for improving time–frequency representation (TFR) with signal-dependent transform parameters. However, the parameter estimation and component separation are two problems to tackle with while using the parameterized TFA to extract time–frequency features from non-stationary vibration signal of varying-speed rotary machinery. In this paper, we propose a procedure for the parameterized TFA to analyze the non-stationary vibration signal of varying-speed rotary machinery. It basically includes four steps: initialization, estimation of transform parameter, component separation and parameterized TFA, as well as feature extraction. To demonstrate the effectiveness of the proposed method in analyzing mono- and multi-component signals, it is first used to analyze the vibration response of a laboratory rotor during a speed-up and run-down process, and then extract the instantaneous time–frequency signatures of a hydro-turbine rotor in a hydroelectric power station during a shut-down stage. In addition, the results are compared with several traditional TFAs and the proposed method outperforms others in accurate feature extraction, which is promising in applications of fault detection, system condition monitoring, parameter identification, etc.