Building point cloud contour is widely used in urban planning and building identification. The existing method cannot work well in the extraction of UAV building point cloud contour points because the low point cloud accuracy of UAV, and effect of noise. In this paper we propose extraction method of UAV building point cloud contour based on the adjacent points distribution. Firstly, we construct an information entropy model for determining optimal neighborhood points, and then use the eigenvectors corresponding to the maximum and minimum eigenvalues obtained by PCA to construct initial projection plane Π1 and Π2. Secondly, we propose the fine-tuning model of the projection plane Π2 by using the aggregation characteristics of neighborhood points, and construct the extraction model of fold points with three parameter constraints. Finally, we propose the information entropy model of points probabilities in different quadrants on the plane Π1 to extract boundary points. Real UAV point cloud data is used to test the performance and parameters of the proposed method, experiment results show that the proposed method is superior to OPAHT, 2D line detection and region clustering segmentation methods in the performance of UAV building point cloud contour extraction. The proposed method can accurately extract the point cloud contour points of UAV buildings.