Ontology provides a formal representation of knowledge as a set of concepts within a domain and the relationships between them. Ontology Matching (OM) identifies similar entities across different ontologies. Similarity Features (SFs) characterize the similarity between two entities from different perspectives, which serves as the foundation of OM method. However, noisy and redundant SFs can obstruct the relevance of useful ones, reducing the quality of matching results. To offer a practical and efficient SF selection, this work proposes a new automatic SF selection framework for OM, which consists of two new components. First, a semantic sampling method is proposed to automatically construct a balanced and representative training dataset, without the need for expert intervention. This method first generates a basic sample set that considers both the representativeness of the concepts and the heterogeneous characteristics of the two ontologies, and then over-samples the minority samples based on their semantic context. Second, a wrapper-based SF selection method is designed, which includes an Adaptive Compact Genetic Algorithm (ACGA) and a Relaxed Naive Bayes (RNB) classifier. The ACGA is able to efficiently identify the SF subset, reducing the computational cost by adaptively maintaining a population probability model based on the diversity and confidence of the SFs. Moreover, the RNB can effectively evaluate the quality of selected SFs by modifying the feature independence assumption with recombination and weighting strategies. Experimental results on the ontology alignment evaluation initiative’s Benchmark demonstrate that our method can efficiently find accurate OM results, and significantly outperform the state-of-the-art matching techniques.