Structure from Motion (SfM) method can reconstruct the story drift ratio, but the camera motion produced by earthquakes reduces the measuring accuracy. This paper improves the applicability of the traditional method by correcting camera motion according to the identification of translation and rotation of camera. Experiments at different levels are designed to prove the proposed method. First, one set of experiments proves the motion correction method by the Single-degree-of-freedom (SDOF) system.The error of maximum response is 4.5% for the case with less rotation. As for the camera motion with larger rotation, the average error increases to 7.9%, which still meets the practical utilization. After that, the accuracy of using SfM method is confirmed by the Multi-degree-of-freedom (MDOF) system with the average error of 4.9%. This paper is expected to extend approaches for the application of the SfM method in the case of huge earthquakes.