Based on characteristic of High Spatial Resolution(HSR) image, an adaptive local geometric invariant feature extraction method based on Gabor transform and Tensor Voting(TV) is proposed and applied to building area extraction from HSR remote sensing images.First, image geometric features are analyzed. Second, the feasibility of extracting built-up area using local feature points and probability density estimation are determined. This study provides specific methods and steps. First, considering the abundant geometric features of high spatial resolution remote sensing images, multi-scale and multi-direction Gabor filter banks are adopted in detecting the singularity of images contain building area. In order to extract edge information of buildings, only real part Gabor coefficients are used. Meanwhile, we also measured the influence of Gabor kernels of different size on geometric feature extraction experiments, and the optimal scale parameter intervals for geometric feature extraction of remote sensing images with different spatial resolutions are thus determined. Second, it is a fact that the response of Gabor filter at each pixel is a measure for orientation certainty, thus, we introduce the orientation tensor which represents an ideal direction in the direction of the unit vector perpendicular to frequency. By weighting the orientation tensor with Gabor coefficients ans summing over it to complete the information fusion, the resulting tensor givens an estimate for local orientation and orientation uncertainty at image position. The tensors are then used as an initial estimate for global context refinement using tensor voting and the points are classified based on the their likelihoods of being part of a feature type, non-maximal suppression is used to extract geometric features. A key advantage of combining the Gabor filtering and tensor voting is that it eliminates the need for any thresholds therefore removing any data dependencies. In order to achieve a reliable extraction of local invariant feature such as corners from built-up area, three criterions are further proposed to refine the first stage result. Finally, each local building corner indicates a building be detected in image. However, only one of them is not sufficient alone to detecting a building. In fact, the more local points a buildings has, the more probable its detection becomes. Based on that fact, a probability density estimate method is generated to describe the probability that the pixel belongs to the building area, and the Otsu method is used to automatically extract the polygon area of the residential area. Experiments were carried out using image data sets such as Google and GF-2 with resolution higher than 1 meter, results showing that the proposed method can achieve higher accuracy in building area detection compared with the state-of-the-art corner detection algorithm such as Harris corner and High-speed corner detect method.