Under fixed imaging conditions, the landmark selection method based feature traversal analysis has high computational complexity. The hierarchical statistical significance detection method uses global statistical information for feature analysis to overcome the computational complexity problem caused by feature traversal analysis. The frequency domain de-correlation method can remove repeat mode in the image by adaptive Gaussian filtering on the amplitude-frequency characteristics. In this paper, combined the hierarchical statistical saliency detection method with the frequency domain de-correlation method, a fast landmark selection algorithm based on saliency analysis is proposed. Based on the proposed algorithm, the automatic landmark selection architecture for terrain matching navigation was constructed. The selection of landmark points was carried out in the Qin-ling Mountains and the Guangdong and Guangxi hills. The results show that compared with the feature or pixel-based landmark selection method, the landmark selection efficiency of the proposed method is improved by 2 to 3 orders of magnitude. The correct matching rate of candidate landmarks selected in the Qinling Mountains and the Guangdong and Guangxi hills are 73.9% and 88.3% respectively.