This paper proposes a dynamic deformation measurement method based on temporal speckle pattern interferometry, tailored for on-site applications. This technique utilizes an improved dynamic phase extraction algorithm to obtain phases associated with both ground vibrations and object deformations. Deformations are discerned from vibrations by employing a synchronized measurement system that integrates laser Doppler vibrometers and a high-speed camera. The effectiveness of the method for dynamic deformation measurements under micron-level vibrations is verified by experiments.