An advanced video deflectometer using off-axis digital image correlation is proposed for real-time, non-contact and targetless measurement of vertical deflection of bridges. To achieve real-time displacement tracking with sub-pixel accuracy, an efficient inverse compositional Gauss–Newton algorithm is employed. The detected image displacements in pixels are converted to physical displacements in millimeters using an easy-to-implement yet accurate calibration technique with the aid of a laser rangefinder. Real translation tests with precisely controlled motions were performed to examine the accuracy of the proposed technique. Real-time deflection monitoring of a railway bridge subjected to train pass-by is also demonstrated to show the practicality, accuracy and application potential of the proposed technique.