To tackle problems such as the destruction of topological structures and the loss of detailed features in the simplification of 3D building models, we propose a 3D building model simplification algorithm that considers detailed features and topological structures. Based on the edge collapse algorithm, the method defines the region formed by the first-order neighboring triangles of the endpoints of the edge to be collapsed as the simplification unit. It incorporates the centroid displacement of the simplification unit, significance level, and approximate curvature of the edge as influencing factors for the collapse cost to control the edge collapse sequence and preserve model details. Additionally, considering the unique properties of 3D building models, boundary edge detection and face overlay are added as constraints to maintain the model’s topological structure. The experimental results show that the algorithm is superior to the classic QEM algorithm in terms of preserving the topological structure and detailed features of the model. Compared to the QEM algorithm and the other two comparison algorithms selected in this paper, the simplified model resulting from this algorithm exhibit a reduction in Hausdorff distance, mean error, and mean square error to varying degrees. Moreover, the advantages of this algorithm become more pronounced as the simplification rate increases. The research findings can be applied to the simplification of 3D building models.