In this paper, a story damage index was developed to evaluate the damage condition of a torsionally coupled building based on its dominant modal frequencies and mode shapes. This index has an analytical formula with a calculated value ranging from 0 (undamaged) to 1.0 (collapsed) to indicate the reduction of story lateral stiffness. The involved computation is simple once the modal parameters of any three modes are obtained through system identification techniques from few floor acceleration measurements. The damage region within a story can also be identified through tracking the change of eccentricity of center of rigidity. This index was verified by numerical simulations and a data analysis of the ASCE benchmark model. In addition, it was also applied to the damage assessment of a four-story reinforced concrete building in Taiwan, which experienced severe damage during the 2006 Taitung Beinan earthquake ( M = 6.2). The results agree fairly well with the visual inspection and show the applicability of the proposed damage assessment technique.