The high structural deficient rate poses serious risks to the operation of many bridges and buildings. To prevent critical damage and structural collapse, a quick structural health diagnosis tool is needed during normal operation or immediately after extreme events. In structural health monitoring (SHM), many existing methods will have limited usefulness in the quick damage identification process because (1) the damage event needs to be identified quickly, and (2) postdamage information is usually unavailable. To address these drawbacks, we propose a new damage detection and localization approach based on stochastic time series analysis. Specifically, damage sensitive features, which are extracted from vibration signals, follow different distributions before and after a damage event. Hence, we use optimal change-point detection theory to find the time of damage occurrence. Because existing change-point detectors require the postdamage feature distribution, which is unavailable in SHM, we propose a maximum likelihood method for learning the distribution parameters from the time-series data. The proposed damage detection using estimated parameters achieves optimal performance. Also, we utilize the detection results to find damage location without any further computation. Validation results show highly accurate damage identification in American Society of Civil Engineers benchmark structures and two shake table experiments.