Full-field measurements of structural vibrations have been achieved by incoherent optical methods with video cameras such as digital image correlation and optical flow. In contrast to coherent optical methods such as scanning laser vibrometers, incoherent optical methods are low-cost and easy to set up. However, the sensitivity (minimum measurable displacement) of incoherent optical methods is generally lower than that of coherent optical methods. Typically, the sensitivity of incoherent optical methods is essentially limited by the finite bit depth of the digital camera due to the quantization with round-off errors. Quantitatively, this theoretical sensitivity limit is determined by the bit depth B as δp=1/(2B−1) [pixel] which corresponds to a displacement causing an intensity change of one gray level. Fortunately, natural dithering may be leveraged to overcome the quantization and exceed the sensitivity limit, achieving super-sensitivity.In this work, we first study the mechanism of the sensitivity limit induced by the quantization and the critical limitation of existing super-sensitivity methods in the full-field measurement of structural vibration. Addressing such a limitation, we present a general adaptive weighted averaging method with dithering, achieving super-sensitivity incoherent optical measurement of full-field deformational vibration of flexible structures. Specifically, both spatial and temporal samplings of the pixels are leveraged simultaneously to adaptively identify the motion (deformation) shape of the structure by principal component analysis (PCA) of spatiotemporal pixel intensities. Then, the identified deformation shape is used to perform a deformation-shape-weighted averaging over spatial pixels with dithering, revealing the sub-sensitivity displacement while retaining spatial resolution at full field. Numerical simulations and laboratory experiments are performed for principle explanation and method validation. Particularly, we derive a mathematical model of the minimum measurable vibration amplitude A∗ for the developed super-sensitivity method, which is found to be determined by the number of spatial pixels for averaging Ns and the noise level σn in the imaging system. Thus, this work provides, for the first time, a rigorous quantification of the sensitivity of incoherent optical measurement for structural vibration, and a new quantitative method to achieve super-sensitivity full-field measurement of structural vibration.