The deflection of immersed tunnel element is an important indicator during floating transportation and installation phases. However, there is no strictly stable platform to install camera sensors while the element is in the process of transportation and installation, which poses a challenge to the application of vision-based methods. In this study, a novel vision measurement system based on the series camera network (SCN) with displacement-relay characteristics is proposed. The SCN is composed of multiple camera stations and multiple measuring points in series through the optical path. Lateral and vertical displacement-relay measurement methods are proposed to compensate for the self-motion of the camera station. For field testing, three camera stations and ten measuring points were installed at the E15 element of Shenzhen-Zhongshan Link, China. The results show that the measurement accuracy of the SCN can reach sub-millimeter level after compensating for the self-motion of the camera station. Subsequently, the SCN has been successfully applied to the deflection monitoring of E15 immersed tunnel element, and the deflection curves of the element during floating transportation and installation have been recorded for the first time. The proposed novel vision measurement system improves the automation of the immersed tunnel during construction.