Vision-based Displacement Measurement Research Articles

Thermography measurement for bridge displacement in the darkness using power-free target

Vision-based displacement measurement has become increasingly popular in bridge health assessment. However, this technology faces challenge that a complicated power-consuming supply system must be installed in the darkness, making it unable to meet the needs of low energy consumption and low cost. This paper proposes a bridge displacement measurement method using power-free target and thermography to address it. First, a thermography measurement system for bridge displacement in the darkness is developed. The power-free target with a hollowed-out circular feature is adopted to form the contrast to the background, and graphene is coated on the target surface to enhance the infrared emissivity and contrast further. Second, a sub-pixel elliptic center positioning algorithm based on thermography enhancement and data optimization is proposed to improve the accuracy and the ability to resist occlusion. The feasibility of the method is verified by laboratory tests and bridge field tests. When the measurement distance is within 15 m, the results meet the engineering error requirements of 5%, with the prospect of popularization in small and medium-sized bridges displacement measurement.

Vibration displacement measurement method based on vision Gaussian fitting and edge optimisation for rotating shafts

The vibrations of rotating machinery contain crucial information for monitoring the operational status of machines and diagnosing potential faults. Measuring vibration displacement using machine vision is currently a promising approach for machine monitoring in industries. However, the existing vision-based vibration displacement measurement methods cannot satisfy the high-accuracy requirements for detecting micrometre-level displacements generated by low-amplitude vibrations of rotating machinery. Therefore, this study proposes an improved noncontact-type vibration displacement measurement method based on Gaussian fitting and edge optimisation for rotating shafts. The real number-type edge positions of the rotating shafts within the selected region of interest (ROI) are accurately extracted via Gaussian edge fitting. Subsequently, edge-linking schemes, double-threshold detection and edge fitting are preformed to optimise the real number-type position of the edges. Finally, continuous displacement extraction of rotating shafts is achieved by monitoring the edge positions in the ROI. The coefficient of determination is used to evaluate a series of simulated and actual experiments quantitatively. The experimental results verify the feasibility, effectiveness, and robustness of the proposed method. An eddy current sensor is used in the actual experiments to verify the accuracy of the proposed method. The result shows that the coefficient of determination reaches 0.9792 when the amplitude of the measured displacement signals is only 22.7 μm.

Variational mode decomposition-based multirate data-fusion framework for estimating structural dynamic displacement by integrating vision- and acceleration-based measurements

Dynamic displacement is a crucial parameter in structural health monitoring (SHM) for assessing the safety, dependability, and suitability of structures under various types of excitations. Computer vision-based methods for dynamic displacement estimation have attracted much interest owing to their cost-effectiveness and convenience. However, these methods are limited by their low sampling rates and high data sensitivity. To compensate for these limitations, methods for combining data obtained from other sensors have been proposed. In this study, an experimental data-fusion framework for displacement estimation based on variational mode decomposition (VMD) was developed to leverage the advantages of vision- and acceleration-based measurements. The measurements were decomposed into ensembles of modes and recomposed to reconstruct the displacement with a higher accuracy and over a wider frequency range. An optimal mode recomposition method was proposed to achieve optimal mode combinations. Furthermore, this study introduced an improved vision-based displacement measurement method and a VMD-based indirect acceleration measurement method. The proposed framework was validated through four-story RC structure tests, which demonstrated that the method could enhance the accuracy of displacement estimation and extend the feasible frequency range compared with single-source displacement measurements. The method provides a promising solution for more effective health monitoring of modern structures subjected to a wide variety of dynamic loads.

Development of an iterative Otsu method for vision-based structural displacement measurement under low-light conditions

Vision-based structural displacement measurement method presented great potential for structural health monitoring application due to its distinguished advantages, and LED targets were always employed to deal with the issue of poor visibility under low-light conditions. However, the ambient light and unfavorable weather seriously influence the identification performance even with LED target. In this paper, an iterative Otsu method was developed to help to realize the correct segmentation of LED spot image by iteratively calculating the image threshold, so as to obtain the structural displacement under low-light conditions even with environmental interference. Furthermore, a simply supported beam model and a cable-stayed bridge model were experimentally investigated under low-light conditions with strong-light and fog interference, and the final results demonstrated that comparing to existing Otsu method and template matching method, the developed method presents an excellent anti-disturbance ability against the environmental interference to realize the correct identification and guarantee the computational efficiency.

Self-calibrating technique for 3D displacement measurement using monocular vision and planar marker

Although the vision-based displacement measurement of structures has been widely studied for decades, it still poses challenges for practical applications, including the issues of calibration and automatic long-term monitoring. This paper introduces a monocular-vision based self-calibrating technique for accurately measuring the 3D structural displacement. The technique utilizes a tailor-made planar marker with unique graph patterns, enabling precise image recognition and PnP resolution. The proposed technique automatically estimates the 3D position and 3D orientation of the marker. The limitations of marker-free techniques in terms of long-term monitoring have also been discussed. The accuracy and robustness of the proposed technique are systematically examined by Monte-Carlo simulation, and then verified by experiments. Results show that accuracy of the proposed technique for 3D displacement measurement is 0.049 mm, which indicates the effectiveness of the technique for automatically measuring and calibrating stereo displacement of structures using monocular vision.

A Feasibility Study on Extension of Measurement Distance in Vision Sensor Using Super-Resolution for Dynamic Response Measurement.

The current civil infrastructure conditions can be assessed through the measurement of displacement using conventional contact-type sensors. To address the disadvantages of traditional sensors, vision-based sensor measurement systems have been derived in numerous studies and proven as an alternative to traditional sensors. Despite the benefits of the vision sensor, it is well known that the accuracy of the vision-based displacement measurement is largely dependent on the camera extrinsic or intrinsic parameters. In this study, the feasibility study of a deep learning-based single image super-resolution (SISR) technique in a vision-based sensor system is conducted to alleviate the low spatial resolution of image frames at long measurement distance ranges. Additionally, its robustness is evaluated using shaking table tests. As a result, it is confirmed that the SISR can reconstruct definite images of natural targets resulting in an extension of the measurement distance range. Additionally, it is determined that the SISR mitigates displacement measurement error in the vision sensor-based measurement system. Based on this fundamental study of SISR in the feature point-based measurement system, further analysis such as modal analysis, damage detection, and so forth should be continued in order to explore the functionality of SR images by applying low-resolution displacement measurement footage.

Spatial displacement tracking of vibrating structure using multiple feature points assisted binocular visual reconstruction

Current vision-based displacement measurement methods have limitations of requiring manual target and parameter adjustment, along with significant user involvement to achieve the desired results. This paper proposes an advanced and comprehensive vision system based on ZED cameras for full-field three-dimensional (3D) vibration displacement measurement of civil structures. Compared to existing binocular systems consisting of multiple monoculars, ZED binoculars offer greater flexibility on instrument setup and data acquisition. A novel keypoint detection and matching algorithm based on deep learning is used to achieve target-free measurement, which is capable of capturing feature points with high precision. The performance of the proposed vision-based approach is evaluated through experimental studies on a six-story scaled frame structure. Statistical analysis of the comparative results in sinusoidal cases demonstrates that the displacement responses obtained from the proposed vision-based approach show consistent results in compliance with displacement sensor measurements. System identification of the target structure is achieved through a seismic case and the error analysis is also concluded in detail.

Automated region-of-interest selection for computer-vision-based displacement estimation of civil structures

A recent trend in vision-based displacement measurement is to place a camera at the measurement point and capture the images of the surrounding areas. In this scheme, a proper region of interest (ROI) should be selected from the captured images. This paper proposes an automated ROI selection technique to improve displacement estimation accuracy. The image frames that capture larger movements of the surrounding areas were selected, and the features in the selected frames were grouped using clustering algorithms. The feature group with consistent movement and high density was finally selected as the optimum ROI. The proposed technique was validated through laboratory and field tests. A displacements estimation technique previously proposed by the authors were used to compared the optimum ROI and four intuitively selected ROIs. In all the tests, the displacement estimates from the optimum ROI showed a smaller RMSE (less than 2 mm) than those from other ROIs.

Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter

Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter

Reliability Assessment of a Vision-Based Dynamic Displacement Measurement System Using an Unmanned Aerial Vehicle.

In recent years, many studies have been conducted on the vision-based displacement measurement system using an unmanned aerial vehicle, which has been used in actual structure measurements. In this study, the dynamic measurement reliability of a vision-based displacement measurement system using an unmanned aerial vehicle was examined by measuring various vibrations with a frequency of 0 to 3 Hz and a displacement of 0 to 100 mm. Furthermore, free vibration was applied to model structures with one and two stories, and the response was measured to examine the accuracy of identifying structural dynamic characteristics. The vibration measurement results demonstrated that the vision-based displacement measurement system using an unmanned aerial vehicle has an average root mean square percentage error of 0.662% compared with the laser distance sensor in all experiments. However, the errors were relatively large in the displacement measurement of 10 mm or less regardless of the frequency. In the structure measurements, all sensors demonstrated the same mode frequency based on the accelerometer, and the damping ratios were extremely similar, except for the laser distance sensor measurement value of the two-story structure. Mode shape estimation was obtained and compared using the modal assurance criterion value compared with the accelerometer, and the values for the vision-based displacement measurement system using an unmanned aerial vehicle were close to 1. According to these results, the vision-based displacement measurement system using an unmanned aerial vehicle demonstrated results similar to those of conventional displacement sensors and can thus replace conventional displacement sensors.

Cover Image, Volume 38, Issue 5

Computer-Aided Civil and Infrastructure EngineeringVolume 38, Issue 5 COVERFree Access Cover Image, Volume 38, Issue 5 First published: 01 March 2023 https://doi.org/10.1111/mice.12986AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Graphical Abstract On the cover: The cover image is based on the Research Article A robust subpixel refinement technique using self-adaptive edge points matching for vision-based structural displacement measurement by Miaomin Wang et al., https://doi.org/10.1111/mice.12889. Volume38, Issue515 March 2023 RelatedInformation

A Mitigation Method for Optical-Turbulence-Induced Errors and Optimal Target Design in Vision-Based Displacement Measurement.

Computer vision-based displacement measurement techniques are increasingly used for structural health monitoring. However, the vision sensors employed are easily affected by optical turbulence when capturing images of the structure, resulting in displacement measurement errors that significantly reduce the accuracy required in engineering applications. Hence, this paper develops a multi-measurement point method to address this problem by mitigating optical-turbulence errors with spatial randomness. Then, the effectiveness of the proposed method in mitigating optical-turbulence errors is verified by static target experiments, in which the RMSE correction rate can reach up to 82%. Meanwhile, the effects of target size and the number of measurement points on the proposed method are evaluated, and the optimal target design criteria are proposed to improve our method's performance in mitigating optical-turbulence errors under different measurement conditions. Additionally, extensive dynamic target experiments reveal that the proposed method achieves an RMSE correction rate of 69% after mitigating the optical-turbulence error. The experimental results demonstrate that the proposed method improves the visual displacement measurement accuracy and retains the detailed information of the displacement measurement results.

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

Vision-based structural displacement measurement under ambient-light changes via deep learning and digital image processing

Structural displacement can provide critical information for structural safety assessment and maintenance. Although vision-based displacement measurement has significant advantages over traditional methods, it still faces challenges in long-term field applications owing to environmental uncertainties. This study proposes a novel displacement measurement method based on deep learning and digital image processing to mitigate the effects of ambient-light changes. The proposed method can automatically extract the calibration object in complex scenarios using You Only Look Once (YOLO) v5 and locate the calibration object under 24-h ambient-light changes precisely. Short- and long-term experiments were conducted in the laboratory to evaluate the performance of the method, and the short-term experimental results were compared with laser displacement sensor (LDS) data, which showed a maximum and minimum relative error of 0.4122% and 0.0024%. The long-term experimental results showed that the displacement responses were within ±1.0 mm. Hence, this method has good potential in structural displacement measurements.

Vision-based volumetric displacement measurement with a self-illuminating target

Thermal deformation is a major source of errors in machine tool positioning. Although the volumetric measurement of thermal deformation is crucial for improving the modeling accuracy of thermal deformations, it is still challenging. This study examines the possibility of vision-based volumetric displacement measurement. A measurement method using three-dimensional (3D) reconstruction is proposed. Combining the extended intensity-fitting method with newly developed self-illuminating target having a good signal-to-noise ratio can achieve accurate position detection. The measurement accuracy of the proposed method was analyzed. A verification experiment was also conducted at a machining center.

A Vision-Based Displacement Measurement System for Foundation Pit

A Vision-Based Displacement Measurement System for Foundation Pit

Development of a low-cost vision-based real-time displacement system using Raspberry Pi

Vision-based real-time displacement measurement systems can measure the structural displacements without contact and output the results immediately, overcoming some drawbacks of traditional contact-type sensors and off-line vision sensors in field applications. However, high cost and complexity of installation restrict the widespread application of the off-the-shelf vision-based real-time displacement systems. In this work, a fast image processing software is developed for operation in single board computers (SBC), firstly, based which a low-cost and portable vision-based real-time displacement measurement system using Raspberry Pi is constructed for civil structural health monitoring. To evaluate the performance of the system in field applications, a new method for validation of on-site measurement accuracy is proposed. The experimental results show that the constructed system accurately measured multipoint structural displacements of a long-span suspension bridge in real-time, with at least 1/17 pixel accuracy, providing a cost-efficient, convenient, high-precision, and contactless approach to obtain the displacement responses of infrastructures in the field.

A non-contact displacement measurement system based on a portable smartphone with digital image methods

Bridges are an indispensable element of modern infrastructure, while the difference in maintenance between the small and medium-sized bridges and large bridges is huge. At present, the health assessments of the small and medium-sized bridges still depend on manual detection, which requires a lot of manpower and material resources with limited efficiency. To overcome these issues, this study proposes a novelty vision-based displacement measurement solution utilizing a smartphone, which includes multiple image recognition methods for different application scenarios and further constructs a complete management and maintenance system. The proposed system is validated in the laboratory tests with a cable-stay bridge and field tests on a continuous plate bridge. Moreover, the field experiments on a continuous girder bridge with far measuring distance under diverse lighting conditions were conducted, investigating the measurement effects of various image methods and providing beneficial explorations for the development of detection and monitoring technology on the small and medium-sized bridges.

Deep learning‐based super‐resolution with feature coordinators preservation for vision‐based measurement

Vision-based displacement measurement is promising and emerging for structural monitoring. However, the accuracy of visual measurement is commonly limited by the resolution of the camera. The super-resolution (SR) technique can reconstruct high-resolution images from the corresponding low-resolution images within the constraints of prior knowledge. Existing SR algorithms mainly focus on improving the overall quality of the image. By contrast, the accurate extraction of the coordinates of feature points is the most important for the visual measurement. Besides, the SR network is usually trained by an artificial dataset whose low-resolution images are obtained by artificially degrading the corresponding high-resolution images, instead of those directly captured by cameras. However, this degradation usually is only a simple bicubic downsampling that cannot reflect the real degradation, which will provide inaccurate constraints to the network training. Therefore, this paper proposes a novel SR framework that can significantly preserve the feature coordinators for visual measurement (SRFCP). First, a deep learning-based SR network that focuses on feature preservation is proposed, which introduces both feature weighted branch and feature preserving loss. Second, an image degradation model is built based on the blur kernel and noise extracted from the images captured in real scene. Experiments on public datasets show that the proposed SRFCP performs well both in terms of the objective evaluation index and the subjective visual effect. Then, a binocular visual measurement platform is set up to measure the distance of adjacent feature points on a chessboard. Lastly, several SR algorithms are evaluated by the improvement they bring to the measurement accuracy. Experimental results show that the distance measurement performance can be significantly improved by the images reconstructed by the SRFCP. Therefore, the proposed SRFCP can accurately reconstruct the high-resolution images while preserving the features coordinates, which is crucial for the visual measurement in structural monitoring.

Vision‐based displacement measurement using a camera mounted on a structure with stationary background targets outside the structure

While structural displacements are essential information for structural health monitoring, they are not being widely used in practice due to the inconvenience. Recently, vision-based displacement measurement methods have been introduced, which are more convenient and cost effective. However, vision-based methods have generally not been used primarily for the continuous monitoring of structures, due to spatial constraints on obtaining an appropriate location to secure the field of view. A vision device shows not only the changes of objects in the scene but also the relative changes of view according to changes in the position to which it is mounted. Accordingly, this study proposes a methodology for measuring the structural displacement of a location where a camera is mounted, based on a camera motion-induced relative view change. The method is organized into three steps. First, camera calibration is conducted with background targets to derive the camera parameters and coordinates of the target feature points. Second, by tracking the relative changes in the feature points according to the camera motion, the changed 2D-image coordinates of the points are derived. Third, the displacement is calculated through the relationship between the changed 2D-image coordinates and fixed 3D-world coordinates of the target feature points using the camera parameters. The changes in view according to the camera motion are analyzed with simulation tests, and the applicability of the proposed method is verified through experimental tests. The results show that the proposed method can be used to rationally measure structural displacements.