Video of a vibrating structural component provides dense continuous spatial information of motion compared to contact based sensors which are more localized. In order to harness such quantitative information, various Computer Vision based methods have been studied in the past few years. Computer vision algorithms that implement sparse optical flow, tracks the displacement of few selected texture rich patches of pixels corresponding to salient points of the structure, in subsequent frames of the video starting from the initial frame. The obtained displacement response of individual, spatially discrete, points of the structure denotes the Lagrangian representation of motion. Such vision based measurement already proved its potential in replacing the need for a contact based vibration measuring sensors. However, full-field vibration response of a structural component, in its operating condition, is vital for identifying heterogeneous mechanical properties as well as localize damages. In order to obtain full-field, spatially dense, vibration response, a large number of sensors are needed to be attached throughout the specimen’s surface, making it impractical. In such scenarios, the technique of Digital Image Correlation (DIC) is used to measure Lagrangian representation of motion using Computer Vision algorithms which implements dense optical flow. In DIC, the presence of much required rich optical textures is ensured by adding speckle paint on the surface of the specimen which can be easily tracked over subsequent frames of the video. But its application is limited to laboratory setups that cannot be implemented for full-scale real-world structural components that vibrate in its operating condition. Continuous edge of a moving object is a rich optical feature whose motion perpendicular to its orientation can be tracked in Lagrangian coordinates using optical flow. In this paper, a novel method of measuring full-field displacement response of a vibrating continuous edge of a structural component is proposed from its video. The proposed method computes both horizontal and vertical components of the Lagrangian displacement of each pixel of the edge for every frame in the video with sub-pixel accuracy. The method consists of applying d’Alembertian of Gaussian filter on the spatiotemporal video signal to convert the edge information into zero crossing signal which is tracked over time using the optical flow of edge. Intermittent and sparse noise in the obtained full-field displacement signal is removed using Robust Principal Component Analysis. Further, experimental validation of the proposed method is presented for two kinds of structures, one a cantilever beam, another a two-story steel frame, both undergoing harmonic as well as random base excitation. The results obtained using the proposed method is validated with the displacement measured using Laser Doppler Vibrometer at a particular point of the edge. Further, the analytical derivations of the proposed method along with the effect of the filter parameters on the edge signal are presented in the Appendix. In order to validate the efficacy of the method on real-world structures, part of the vibrating cable of Tacoma Narrows bridge is tracked from its video, moments before its collapse. The results show high correspondence between the actual motion of the cable and traced motion of its edge, over time. The proposed method reveals the absolute as well as relative displacement response of the cable along with the dominating frequencies of the cable’s vibration.