Feature matching is the foundation and key task of remote sensing image registration, which is to establish a reliable point corresponding relationship between the feature points of two images. In this article, a simple and effective local consensus method for rigid and nonrigid feature matching is proposed and applied to solve the problem of high outliers ratio caused by nonrigid transformation, nonlinear radiation difference, and speckle noise in the remote sensing image registration task. We first establish the putative feature correspondences according to the similarity between local descriptors and then use local consensus constraints (including neighborhood consensus and motion vector consensus) to remove outliers. The specific steps are given as follows. First, we use the neighborhood consensus constraint of feature points to carry out preliminary filtering to remove outliers with obvious errors and retain a large number of inliers, so as to obtain a clean reliable set. Then, the reliable set space is grided into several nonoverlapping cells, and the estimated motion vector is calculated for each cell. By taking the comprehensive deviation between the ordinary motion vectors and estimated motion vectors, we transform the matching problem into a mathematical optimization model and derive a closed-form solution with linear time and linear space complexities. In this way, our method can also significantly increase the speed of operation without sacrificing accuracy. A large number of feature matching experiments on remote sensing prove that our method is superior to existing methods and also has good results in the general scene.