Video sequence matching aims to locate a query video clip in a video database. It plays an important role in reducing storage redundancy and detecting video copies for copyright protection. In this paper, we propose an effective method for video sequence matching based on the invariance of color correlation. The proposed method first splits each key-frame into nonoverlapping blocks. For each block, we sort the red, green, and blue color components according to their average intensities, and use the percentage of the color correlation to generate a frame feature with a small size. Finally, the resulting video feature is made up of the consecutive frame features, which is demonstrated to be robust against most typical video content-preserving operations, including geometric distortion, blurring, noise contamination, contrast enhancement, and strong re-encoding. The experimental results show that the proposed method outperforms the existing methods in the literature, as well as the method based on the traditional color histogram. Furthermore, the time and space complexity of our algorithm are both satisfactory, which are very important for many real-time applications.