In this letter, we propose a novel method to solve the problem of detecting geospatial objects present in high-resolution remote sensing images automatically. Each image is represented as a segmentation tree by applying a multiscale segmentation algorithm at first, and all of the tree nodes are described as coherent groups instead of binary classified values. The trees are matched to select the maximally matched subtrees, denoted as common subcategories. Then, we organize these subcategories to learn the embedded taxonomic semantics of objects categories, which allow categories to be defined recursively, and express both explicit and implicit spatial configuration of categories. Detection, recognition, and segmentation of the geospatial objects in a new image can be simultaneously conducted by using the learned taxonomic semantics. This procedure also provides a meaningful explanation for image understanding. Experiments for complex and compound objects demonstrate the precision, robustness, and effectiveness of the proposed method.