This paper deals with the applicability of system realization using the information matrix(SRIM) system identification technique combined with moving time windows (calledWSRIM) in order to evaluate the time-varying dynamic properties and damage of abuilding based on its real earthquake records considering the soil–structure interactioneffect. Successive short segments of inputs and their corresponding output signals areselected and structural properties exhibited during that segment of time are identified.Both numerical simulation and real earthquake response measurements from several floorsand foundations are utilized to verify the accuracy of the proposed time-varying systemidentification technique. In addition, the seven-storey reinforced concrete building of theCivil and Environmental Engineering Department at the National Chung Hsing Universityexperienced moderate structural damage during the 1999 Taiwan Chi-Chi earthquake. Thispaper evaluated the time-varying modal parameters of this building on the basis of recordsof its acceleration response to the Chi-Chi earthquake main shock and its numerousafter-shocks. It is proved that the proposed system identification technique is usefulfor real-time dynamic property estimation and damage assessment of buildings.