In this paper, we proposed an improved recovery algorithm, with efficient pilot placement, for Compressed Sensing (CS)-based sparse channel estimation based on Sparsity Adaptive Matching Pursuit (SAMP) in Orthogonal Frequency Division Multiplexing (OFDM) communication systems. The proposed algorithm does not require a priori knowledge of the sparsity, and to approach the true sparsity, adjusts the step size adaptively. Furthermore, estimation accuracy can be affected by different measurement matrices in CS, due to different pilot arrangements. It is known that, the Cyclic Difference Set (CDS) is the optimal setoff pilot locations when the signal is sparse on the unitary Discrete Fourier Transform (DFT) matrix by minimizing the mutual coherence of the measurement matrix. Based on this, an efficient pilot insertion scheme is introduced in cases where Cyclic Different Set does not exist. Simulation results in the paper show that the channel estimation algorithm, with the new pilot placement scheme, which offers a better trade-off between the performances in terms of Bit Error Rate (BER), Mean Squared Error (MSE) and the complexity, when compared to previous estimation algorithms.