When traffic flows pass through existing bridges, the dynamic effect of the vehicle loads increases due to the local surface damage rather than the entire roughness level of the bridge deck. This study presents a method for evaluating a dynamic impact factor (IM) from a bridge dynamic response under a traffic flow considering the local surface damage. The random axle loads of traffic vehicles under the deteriorated roughness conditions are presented using an input load time-history applied to each discretized deck node of a finite element bridge model. The amplification ratios of the root mean square of the simulated acceleration under nominal road surface conditions with respect to the specified IMs in the design stage are calculated. The relationship between the ratios and IMs is determined to evaluate the dynamic impact factors under any surface roughness condition. The method is validated by applying it to two existing bridges with different road surface damage and traffic conditions. The results show that the IMs depend on both the traffic loading and bridge surface roughness conditions. Under the same traffic flow conditions, the values increase when the road surface damage increases. However, the increases in the IMs vary according to the location of the surface roughness deterioration in both of the target bridges.