Sparse coefficient solutions have received continuous attention as one of the core steps of sparse representation. The computational complexity of almost all sparse coefficient solution algorithms is exponentially increased with a signal dimension. In this paper, impact signals caused by localized faults are sparsely represented by a shift-invariant dictionary, which can be Fourier diagonalized based on its circulant structure. Then, the matrix–matrix and matrix–vector multiplications involved in obtaining sparse coefficients are converted into calculation forms that mainly contain the Fourier transform and its inverse to reduce computational complexity. Based on this result, fast solutions of three common sparse coefficient solution algorithms are deduced in detail. Simulation analysis shows that the deduced fast algorithms can significantly fasten running time while maintaining same accuracy as corresponding original algorithms. Localized rolling bearing faults related experiments further verify the effectiveness of the proposed sparse coefficient fast solution algorithms for machine fault diagnosis.