Block-matching motion estimation (ME) is a principal process for most existing video compression standards, but it has a significant impact on computation overhead. An efficient block-matching algorithm is proposed for ME. The proposed algorithm consists of three effective steps: (1) apply the full search (FS) algorithm to construct a polynomial interpolation model from the group with four skipping frames to determine near-optimal global motion vector (MV) for the initial search point, (2) perform an adaptive search pattern in the interpolation step to refine the MV and (3) use the half-way stop technique to reduce search points. Experimental results show that the proposed algorithm achieves a maximum speed-up factor of 366.47 with only 0.52% peak signal-to-noise ratio (PSNR) degradation in comparison with the FS algorithm.