Vision-based Displacement Measurement Research Articles

A two-stage mitigation method for optical turbulence-induced errors in vision-based structural displacement measurement

A two-stage mitigation method for optical turbulence-induced errors in vision-based structural displacement measurement

A hybrid approach for vision-based structural displacement measurement using transforming model prediction and KLT

As civil infrastructures age, monitoring their health conditions has become increasingly critical. Dynamic displacement measurement is a prevalent method for assessing structural health. Traditional techniques, which often involve installing instruments and scaffolding, can interfere with the response of the structures. To address the challenge, non-contact measurement methods have been developed; however, these are typically costly and require expert operation. Advances in high-speed industrial cameras and image processing technology now enable vision-based displacement measurement. Despite their effectiveness, existing vision-based methods face significant limitations, including their inability to maintain tracking when line-of-sight is obstructed, sensitivity to lighting variations, and the need for manual intervention when feature points are lost. This study introduces a novel hybrid approach, termed ToMP-KLT, which combines the KLT tracker with a deep learning-based model. This method harnesses the precision of the KLT tracker under favorable conditions and the robustness of the deep learning-based tracker under adverse conditions. Its effectiveness is validated through simulation-based tests, lab-scale experiments, and field testing on Cheonsa Bridge, demonstrating substantial improvements in tracking robustness against occlusions and varying light conditions.

Smart structural health monitoring using computer vision and edge computing

Structural health monitoring (SHM) provides real-time data on the condition and performance of infrastructure, enabling timely and cost-effective maintenance interventions, and hence enhanced safety and extended service life. The computer vision-based non-contact sensor has emerged as a promising alternative to conventional contact-type sensors for structural displacement measurement and SHM. Many of the currently reported vision-based structural displacement measurement systems typically temporarily set up a video camera from a distance to the structure. The collected images or videos are usually stored locally and post-processed offline to obtain structural displacement responses, which is cumbersome and limited to short-term SHM applications. The recent development of technologies empowered by the Internet of Things (IoT) and edge computing has enabled real-time video processing and analysis at the source, minimizing latency, reducing bandwidth requirements, and enabling prompt decision-making, thereby enhancing efficiency and responsiveness compared to traditional offline video recording and processing systems. In this paper, an edge computing vision-based displacement measurement system (EdgeCVDMS) is developed. Video recording, processing, and displacement response identification are entirely performed on an edge device integrated with the vision-based displacement tracking algorithm, thereby greatly reducing the amount of data transmitted to the cloud server. The feasibility and applicability of the developed sensing system are experimentally validated on a laboratory-scaled transmission tower structure. The proposed EdgeCVDMS is cost-effective, easily deployable, and of great potential to be applied for the condition assessment of a larger population of aging civil infrastructure.

A Novel Method for Heat Haze-Induced Error Mitigation in Vision-Based Bridge Displacement Measurement.

Vision-based techniques have become widely applied in structural displacement monitoring. However, heat haze poses a great threat to the precision of vision systems by creating distortions in the images. This paper proposes a vision-based bridge displacement measurement technique with heat haze mitigation capability. The properties of heat haze-induced errors are illustrated. A dual-tree complex wavelet transform (DT-CWT) is used to mitigate the heat haze in images, and the speeded-up robust features (SURF) algorithm is employed to extract the displacement. The proposed method is validated through indoor experiments on a bridge model. The designed vision system achieves high measurement accuracy in a heat haze-free condition. The proposed mitigation method successfully corrects 61.05% of heat haze-induced errors in static experiments and 95.31% in dynamic experiments.

Vision-Based Displacement Measurement of Pedestrian Suspension Bridges

The number of pedestrian suspension bridges in South Korea increased from 188 to 208 between 2020 and 2021, with a corresponding increase in the number of visitors. However, these bridges are susceptible to structural issues due to natural factors and their continual usage, necessitating structural health monitoring (SHM) to prevent accidents. SHM requires data on the structure's vibration, deflection, and deformation. This study proposes a system utilizing targets for vision-based dynamic behavior measurement of pedestrian suspension bridges through image processing and applying these data to SHM. The proposed method demonstrated a deviation of only 1.3150 mm when compared to the direct measurement of the bridge's dynamic behavior and yielded almost identical mode shape results. The proposed approach can be effectively employed to inspect and maintain pedestrian suspension bridges.

Computer vision-based displacement measurement using spatio-temporal context and optical flow considering illumination variation

Computer vision-based displacement measurement using spatio-temporal context and optical flow considering illumination variation

Thermography measurement for bridge displacement in the darkness using power-free target

Vision-based displacement measurement has become increasingly popular in bridge health assessment. However, this technology faces challenge that a complicated power-consuming supply system must be installed in the darkness, making it unable to meet the needs of low energy consumption and low cost. This paper proposes a bridge displacement measurement method using power-free target and thermography to address it. First, a thermography measurement system for bridge displacement in the darkness is developed. The power-free target with a hollowed-out circular feature is adopted to form the contrast to the background, and graphene is coated on the target surface to enhance the infrared emissivity and contrast further. Second, a sub-pixel elliptic center positioning algorithm based on thermography enhancement and data optimization is proposed to improve the accuracy and the ability to resist occlusion. The feasibility of the method is verified by laboratory tests and bridge field tests. When the measurement distance is within 15 m, the results meet the engineering error requirements of 5%, with the prospect of popularization in small and medium-sized bridges displacement measurement.

Vibration displacement measurement method based on vision Gaussian fitting and edge optimisation for rotating shafts

The vibrations of rotating machinery contain crucial information for monitoring the operational status of machines and diagnosing potential faults. Measuring vibration displacement using machine vision is currently a promising approach for machine monitoring in industries. However, the existing vision-based vibration displacement measurement methods cannot satisfy the high-accuracy requirements for detecting micrometre-level displacements generated by low-amplitude vibrations of rotating machinery. Therefore, this study proposes an improved noncontact-type vibration displacement measurement method based on Gaussian fitting and edge optimisation for rotating shafts. The real number-type edge positions of the rotating shafts within the selected region of interest (ROI) are accurately extracted via Gaussian edge fitting. Subsequently, edge-linking schemes, double-threshold detection and edge fitting are preformed to optimise the real number-type position of the edges. Finally, continuous displacement extraction of rotating shafts is achieved by monitoring the edge positions in the ROI. The coefficient of determination is used to evaluate a series of simulated and actual experiments quantitatively. The experimental results verify the feasibility, effectiveness, and robustness of the proposed method. An eddy current sensor is used in the actual experiments to verify the accuracy of the proposed method. The result shows that the coefficient of determination reaches 0.9792 when the amplitude of the measured displacement signals is only 22.7 μm.

Variational mode decomposition-based multirate data-fusion framework for estimating structural dynamic displacement by integrating vision- and acceleration-based measurements

Dynamic displacement is a crucial parameter in structural health monitoring (SHM) for assessing the safety, dependability, and suitability of structures under various types of excitations. Computer vision-based methods for dynamic displacement estimation have attracted much interest owing to their cost-effectiveness and convenience. However, these methods are limited by their low sampling rates and high data sensitivity. To compensate for these limitations, methods for combining data obtained from other sensors have been proposed. In this study, an experimental data-fusion framework for displacement estimation based on variational mode decomposition (VMD) was developed to leverage the advantages of vision- and acceleration-based measurements. The measurements were decomposed into ensembles of modes and recomposed to reconstruct the displacement with a higher accuracy and over a wider frequency range. An optimal mode recomposition method was proposed to achieve optimal mode combinations. Furthermore, this study introduced an improved vision-based displacement measurement method and a VMD-based indirect acceleration measurement method. The proposed framework was validated through four-story RC structure tests, which demonstrated that the method could enhance the accuracy of displacement estimation and extend the feasible frequency range compared with single-source displacement measurements. The method provides a promising solution for more effective health monitoring of modern structures subjected to a wide variety of dynamic loads.

Development of an iterative Otsu method for vision-based structural displacement measurement under low-light conditions

Vision-based structural displacement measurement method presented great potential for structural health monitoring application due to its distinguished advantages, and LED targets were always employed to deal with the issue of poor visibility under low-light conditions. However, the ambient light and unfavorable weather seriously influence the identification performance even with LED target. In this paper, an iterative Otsu method was developed to help to realize the correct segmentation of LED spot image by iteratively calculating the image threshold, so as to obtain the structural displacement under low-light conditions even with environmental interference. Furthermore, a simply supported beam model and a cable-stayed bridge model were experimentally investigated under low-light conditions with strong-light and fog interference, and the final results demonstrated that comparing to existing Otsu method and template matching method, the developed method presents an excellent anti-disturbance ability against the environmental interference to realize the correct identification and guarantee the computational efficiency.

Self-calibrating technique for 3D displacement measurement using monocular vision and planar marker

Although the vision-based displacement measurement of structures has been widely studied for decades, it still poses challenges for practical applications, including the issues of calibration and automatic long-term monitoring. This paper introduces a monocular-vision based self-calibrating technique for accurately measuring the 3D structural displacement. The technique utilizes a tailor-made planar marker with unique graph patterns, enabling precise image recognition and PnP resolution. The proposed technique automatically estimates the 3D position and 3D orientation of the marker. The limitations of marker-free techniques in terms of long-term monitoring have also been discussed. The accuracy and robustness of the proposed technique are systematically examined by Monte-Carlo simulation, and then verified by experiments. Results show that accuracy of the proposed technique for 3D displacement measurement is 0.049 mm, which indicates the effectiveness of the technique for automatically measuring and calibrating stereo displacement of structures using monocular vision.

A Feasibility Study on Extension of Measurement Distance in Vision Sensor Using Super-Resolution for Dynamic Response Measurement.

The current civil infrastructure conditions can be assessed through the measurement of displacement using conventional contact-type sensors. To address the disadvantages of traditional sensors, vision-based sensor measurement systems have been derived in numerous studies and proven as an alternative to traditional sensors. Despite the benefits of the vision sensor, it is well known that the accuracy of the vision-based displacement measurement is largely dependent on the camera extrinsic or intrinsic parameters. In this study, the feasibility study of a deep learning-based single image super-resolution (SISR) technique in a vision-based sensor system is conducted to alleviate the low spatial resolution of image frames at long measurement distance ranges. Additionally, its robustness is evaluated using shaking table tests. As a result, it is confirmed that the SISR can reconstruct definite images of natural targets resulting in an extension of the measurement distance range. Additionally, it is determined that the SISR mitigates displacement measurement error in the vision sensor-based measurement system. Based on this fundamental study of SISR in the feature point-based measurement system, further analysis such as modal analysis, damage detection, and so forth should be continued in order to explore the functionality of SR images by applying low-resolution displacement measurement footage.

Spatial displacement tracking of vibrating structure using multiple feature points assisted binocular visual reconstruction

Current vision-based displacement measurement methods have limitations of requiring manual target and parameter adjustment, along with significant user involvement to achieve the desired results. This paper proposes an advanced and comprehensive vision system based on ZED cameras for full-field three-dimensional (3D) vibration displacement measurement of civil structures. Compared to existing binocular systems consisting of multiple monoculars, ZED binoculars offer greater flexibility on instrument setup and data acquisition. A novel keypoint detection and matching algorithm based on deep learning is used to achieve target-free measurement, which is capable of capturing feature points with high precision. The performance of the proposed vision-based approach is evaluated through experimental studies on a six-story scaled frame structure. Statistical analysis of the comparative results in sinusoidal cases demonstrates that the displacement responses obtained from the proposed vision-based approach show consistent results in compliance with displacement sensor measurements. System identification of the target structure is achieved through a seismic case and the error analysis is also concluded in detail.

Automated region-of-interest selection for computer-vision-based displacement estimation of civil structures

A recent trend in vision-based displacement measurement is to place a camera at the measurement point and capture the images of the surrounding areas. In this scheme, a proper region of interest (ROI) should be selected from the captured images. This paper proposes an automated ROI selection technique to improve displacement estimation accuracy. The image frames that capture larger movements of the surrounding areas were selected, and the features in the selected frames were grouped using clustering algorithms. The feature group with consistent movement and high density was finally selected as the optimum ROI. The proposed technique was validated through laboratory and field tests. A displacements estimation technique previously proposed by the authors were used to compared the optimum ROI and four intuitively selected ROIs. In all the tests, the displacement estimates from the optimum ROI showed a smaller RMSE (less than 2 mm) than those from other ROIs.

Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter

Robust vision-based displacement measurement and acceleration estimation using RANSAC and Kalman filter

Reliability Assessment of a Vision-Based Dynamic Displacement Measurement System Using an Unmanned Aerial Vehicle.

In recent years, many studies have been conducted on the vision-based displacement measurement system using an unmanned aerial vehicle, which has been used in actual structure measurements. In this study, the dynamic measurement reliability of a vision-based displacement measurement system using an unmanned aerial vehicle was examined by measuring various vibrations with a frequency of 0 to 3 Hz and a displacement of 0 to 100 mm. Furthermore, free vibration was applied to model structures with one and two stories, and the response was measured to examine the accuracy of identifying structural dynamic characteristics. The vibration measurement results demonstrated that the vision-based displacement measurement system using an unmanned aerial vehicle has an average root mean square percentage error of 0.662% compared with the laser distance sensor in all experiments. However, the errors were relatively large in the displacement measurement of 10 mm or less regardless of the frequency. In the structure measurements, all sensors demonstrated the same mode frequency based on the accelerometer, and the damping ratios were extremely similar, except for the laser distance sensor measurement value of the two-story structure. Mode shape estimation was obtained and compared using the modal assurance criterion value compared with the accelerometer, and the values for the vision-based displacement measurement system using an unmanned aerial vehicle were close to 1. According to these results, the vision-based displacement measurement system using an unmanned aerial vehicle demonstrated results similar to those of conventional displacement sensors and can thus replace conventional displacement sensors.

Cover Image, Volume 38, Issue 5

Computer-Aided Civil and Infrastructure EngineeringVolume 38, Issue 5 COVERFree Access Cover Image, Volume 38, Issue 5 First published: 01 March 2023 https://doi.org/10.1111/mice.12986AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Graphical Abstract On the cover: The cover image is based on the Research Article A robust subpixel refinement technique using self-adaptive edge points matching for vision-based structural displacement measurement by Miaomin Wang et al., https://doi.org/10.1111/mice.12889. Volume38, Issue515 March 2023 RelatedInformation

A Mitigation Method for Optical-Turbulence-Induced Errors and Optimal Target Design in Vision-Based Displacement Measurement.

Computer vision-based displacement measurement techniques are increasingly used for structural health monitoring. However, the vision sensors employed are easily affected by optical turbulence when capturing images of the structure, resulting in displacement measurement errors that significantly reduce the accuracy required in engineering applications. Hence, this paper develops a multi-measurement point method to address this problem by mitigating optical-turbulence errors with spatial randomness. Then, the effectiveness of the proposed method in mitigating optical-turbulence errors is verified by static target experiments, in which the RMSE correction rate can reach up to 82%. Meanwhile, the effects of target size and the number of measurement points on the proposed method are evaluated, and the optimal target design criteria are proposed to improve our method's performance in mitigating optical-turbulence errors under different measurement conditions. Additionally, extensive dynamic target experiments reveal that the proposed method achieves an RMSE correction rate of 69% after mitigating the optical-turbulence error. The experimental results demonstrate that the proposed method improves the visual displacement measurement accuracy and retains the detailed information of the displacement measurement results.

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

Vision-based structural displacement measurement under ambient-light changes via deep learning and digital image processing

Structural displacement can provide critical information for structural safety assessment and maintenance. Although vision-based displacement measurement has significant advantages over traditional methods, it still faces challenges in long-term field applications owing to environmental uncertainties. This study proposes a novel displacement measurement method based on deep learning and digital image processing to mitigate the effects of ambient-light changes. The proposed method can automatically extract the calibration object in complex scenarios using You Only Look Once (YOLO) v5 and locate the calibration object under 24-h ambient-light changes precisely. Short- and long-term experiments were conducted in the laboratory to evaluate the performance of the method, and the short-term experimental results were compared with laser displacement sensor (LDS) data, which showed a maximum and minimum relative error of 0.4122% and 0.0024%. The long-term experimental results showed that the displacement responses were within ±1.0 mm. Hence, this method has good potential in structural displacement measurements.