Seismic fragility analysis is a crucial tool for assessing the seismic performance of buildings. In areas with dense clusters of tall buildings, the significant site-city interaction (SCI) effect alters wave propagation mechanisms, influencing the seismic fragility of structures. However, a significant increase in computational workload results from the need for detailed modeling of sites and building clusters for the SCI analysis. To address this challenge, this work first investigates the minimum number of earthquake waves required to characterize SCI-induced response changes. The Central Business District of Shanghai is analyzed. A table for the recommended minimum number for a given accuracy requirement and prediction reliability is provided. Moreover, a seismic fragility analysis method considering the SCI effect is proposed for low-rise buildings. The case study indicates that, buildings with similar height will exhibit various fragility changes after considering SCI. For the complete damage state, the mean intensity value of the fragility curve can be 14.4 % smaller than that without SCI. In addition, this approach provides significant computational workload reduction. For the case study, the computational workload of the proposed method is roughly 1/50 of that using traditional IDA method.