Sparse fault transient extraction is the primary step in rotating machine fault detection. In the present paper, periodical convolutional sparse representation (PCSR) is proposed for reliable separation of fault transients imbedded in raw vibration signals. Specifically, a sparse optimization problem of PCSR is constructed, in which periodical fault transients and harmonic components are sparsely represented by a learned dictionary and Fourier dictionary, and the periodicity and group sparsity of sparse coefficients related to sparse fault transients are also incorporated. Meanwhile, to further promote the sparsity of the sparse coefficients, a non-convex function is also introduced into the optimization problem. In addition, an iterative algorithm is developed to resolve the constructed sparse optimization problem, and the parameter selection method is also investigated to ensure the fault transient extraction ability of PCSR. The performance of the proposed PCSR is assessed via a synthetic and actual vibration signal. The results illustrate that the proposed PCSR has an excellent ability in fault transient extraction and machine fault detection.