This paper clarifies that adequate decimation of high-rate sampled input-output data can give much improvement on the MSE of the least squares estimate of an impulse response. In order to reconstruct the impulse response estimate with the original sampling rate, we employ an interpolation scheme with the same factor of the decimation. We will show that there exists an optimal sampling rate for the least squares estimation, which minimizes the MSE that depends on the impulse response itself, the input power spectrum or the distribution of the eigenvalues of input correlation matrix, the noise variance and the total number of data. Furthermore, we give an effective data-based scheme for deciding the decimation and interpolation rate using the accessible input-output data.