Based on the chirplet path pursuit and the sparse signal decomposition method, a new sparse signal decomposition method based on multi-scale chirplet is proposed and applied to the decomposition of vibration signals from gearboxes in fault diagnosis. An over-complete dictionary with multi-scale chirplets as its atoms is constructed using the method. Because of the multi-scale character, this method is superior to the traditional sparse signal decomposition method wherein only a single scale is adopted, and is more applicable to the decomposition of non-stationary signals with multi-components whose frequencies are time-varying. When there are faults in a gearbox, the vibration signals collected are usually AM–FM signals with multiple components whose frequencies vary with the rotational speed of the shaft. The meshing frequency and modulating frequency, which vary with time, can be derived by the proposed method and can be used in gearbox fault diagnosis under time-varying shaft-rotation speed conditions, where the traditional signal processing methods are always blocked. Both simulations and experiments validate the effectiveness of the proposed method.