Investigating the mathematical and geometric principles embedded in ancient classic architecture is a significant tradition in the history of architectural development. Drawing inspiration from the modular design and creative ideology based on the geometric proportions of squareness and roundness in ancient Chinese architecture, we propose a new mode of squareness and roundness composition based on scale proportion specifically for the design of multi-story buildings. Taking Yingxian Wooden Pagoda as the case study, we not only re-evaluate the modular system and proportional rules followed in the design of the entire pagoda, but also reveal the technical approaches and geometric rules for effectively controlling the form of multi-story buildings. In particular, the mode of squareness and roundness composition based on scale proportion, utilizing a modular grid combined with squareness and roundness drawings as decision-making tools, can control the scale and proportion of buildings across different design dimensions and organically coordinate the design of multi-story buildings’ plans and elevations. Thus, it can achieve an effective balance of multi-story architectural forms. This study has potential applications in the creation of traditional multi-story buildings and heritage restoration projects, and offers valuable insights for future research on ancient multi-story buildings.