Pairwise orthogonal planes are important semantic structures in indoor scenes. The method of Manhattan world orientation calculation and constrained planes selection can solve most segmentation problems of point cloud, but it is inefficient when it comes to small areas on beams, columns or other small architectural primitives. In this paper, we propose a novel method to extract pairwise orthogonal planes (abbr. POP) from point cloud building model. We firstly define the orthogonal structure, and then utilize it as a primitive shape to extract the pairwise orthogonal planes using RANSAC. Our method can segment small regions of beams and columns. Moreover, it can also extract non-manifold structures with broken point cloud data. Experimental results show that our proposed method can extract the POP model efficiently and accurately.