Structural Displacement Measurement Research Articles

A two-stage mitigation method for optical turbulence-induced errors in vision-based structural displacement measurement

Vision-based structural displacement measurement techniques have been widely applied. However, the visual sensors used for remote monitoring of structures in high-temperature weather are easily affected by optical turbulence, which introduces errors in displacement measurement. Therefore, this paper proposes a two-stage optical turbulence-induced error alleviation method. In the first stage, the steerable pyramid method is used to decompose the monitoring video and perform temporal filtering on the phase, which can significantly attenuate the motion and distortion caused by optical turbulence in the video. In the second stage, a feature point matching method considering the weighted distance is used to track the multi-point displacement in the reconstructed video to improve the robustness of feature point tracking, and the results are spatially filtered to improve measurement accuracy. The effectiveness of the proposed method has been verified through laboratory experiments and on-site testing.

Non-contact Structural Displacement Measurement using UAV and DIC with Non-coplanar Fixed Region of Interest

The use of unmanned aerial vehicles (UAV) for displacement measurement using Digital Image Correlation (DIC) has been popular. A fixed region of interest (FROI) is often used to calibrate the UAV camera. Existing techniques require the FROI to be coplanar with the target plane, such that a corner of the FROI can be taken directly as the origin of the global coordinates. However, this condition does not exist when the target plane is in motion, and this has been a significant challenge in the correction of the UAV images. This paper proposed an algorithm to estimate the posture of the UAV camera for solving this issue. Numerical simulation shows that the proposed algorithm has high computation accuracy and stability, and it is robust to the amplitude of motion of the target image or camera. Field experiments confirm the feasibility of the proposed algorithm for images with non-coplanar conditions with a maximum error of less than 2.3 mm and a peak error below 5 %. In comparison to the traditional coplanar correction method, the CPS method demonstrates lower errors, making it better suited for cases where the FROI and the target plane are not coplanar.

Non-contact measurement of structural dynamic displacement based on improved edge detection and video interpolation

In structural health monitoring (SHM), smartphones are increasingly used for non-contact dynamic displacement measurement using computer vision. To overcome limitations in capturing high-frame-rate data, a non-contact measurement method of structural dynamic displacement based on an improved edge detection algorithm and video frame interpolation is proposed. Initially, a smartphone is employed to capture the structure’s vibration video and then generates intermediary frames between adjacent frames that match the structure’s motion state by using a video frame interpolation algorithm to obtain a video with a high frame rate. Subsequently, the improved edge detection algorithm is applied to measure the displacement within the interpolated video. In order to address the problem that traditional edge detection may have false edges leading to the degradation of measurement accuracy, a feature point tracking technique is proposed based on the traditional edge detection technique, which effectively excludes the interference of false edges in ROI and improves the accuracy of displacement monitoring. Experimental validation on a cantilever beam and steel frame bridge demonstrates the method’s viability and precision, highlighting its effectiveness in improving displacement measurement accuracy compared to traditional methods.

Monocular vision-based structural out-of-plane motion estimation using a deep learning method

Computer vision-based displacement estimation methods can estimate structural motion from videos, but face challenges in estimating out-of-plane motion. Conventional vision-based methods employ either multi-view camera systems or structural model-aided monocular vision methods to estimate structural out-of-plane motion. Nonetheless, the utilization of multi-view camera systems leads to additional costs and stringent requirements for overlapping views, while structural model-aided methods require comprehensive structural information. To achieve convenient and accurate out-of-plane structural displacement measurement, this study presented a monocular vision-based measurement method using a deep learning technique. Firstly, a deep learning model was employed to estimate the global depth matrix of images. The relative depth of the region of interest (ROI) was then estimated through pyramid-accelerated tracking and relative depth matrix weighting. To convert the relative depth into actual out-of-plane motion, the scale factor was calculated through least squares fit. The performance of the proposed method was validated through static and dynamic tests. The results indicated that the proposed approach yielded reasonable out-of-plane motion estimations, with correlation coefficients with displacement sensor measurements over 0.93. The proposed method therefore offered a cost-effective and convenient alternative for structural out-of-plane motion estimation.

A hybrid approach for vision-based structural displacement measurement using transforming model prediction and KLT

As civil infrastructures age, monitoring their health conditions has become increasingly critical. Dynamic displacement measurement is a prevalent method for assessing structural health. Traditional techniques, which often involve installing instruments and scaffolding, can interfere with the response of the structures. To address the challenge, non-contact measurement methods have been developed; however, these are typically costly and require expert operation. Advances in high-speed industrial cameras and image processing technology now enable vision-based displacement measurement. Despite their effectiveness, existing vision-based methods face significant limitations, including their inability to maintain tracking when line-of-sight is obstructed, sensitivity to lighting variations, and the need for manual intervention when feature points are lost. This study introduces a novel hybrid approach, termed ToMP-KLT, which combines the KLT tracker with a deep learning-based model. This method harnesses the precision of the KLT tracker under favorable conditions and the robustness of the deep learning-based tracker under adverse conditions. Its effectiveness is validated through simulation-based tests, lab-scale experiments, and field testing on Cheonsa Bridge, demonstrating substantial improvements in tracking robustness against occlusions and varying light conditions.

Smart structural health monitoring using computer vision and edge computing

Structural health monitoring (SHM) provides real-time data on the condition and performance of infrastructure, enabling timely and cost-effective maintenance interventions, and hence enhanced safety and extended service life. The computer vision-based non-contact sensor has emerged as a promising alternative to conventional contact-type sensors for structural displacement measurement and SHM. Many of the currently reported vision-based structural displacement measurement systems typically temporarily set up a video camera from a distance to the structure. The collected images or videos are usually stored locally and post-processed offline to obtain structural displacement responses, which is cumbersome and limited to short-term SHM applications. The recent development of technologies empowered by the Internet of Things (IoT) and edge computing has enabled real-time video processing and analysis at the source, minimizing latency, reducing bandwidth requirements, and enabling prompt decision-making, thereby enhancing efficiency and responsiveness compared to traditional offline video recording and processing systems. In this paper, an edge computing vision-based displacement measurement system (EdgeCVDMS) is developed. Video recording, processing, and displacement response identification are entirely performed on an edge device integrated with the vision-based displacement tracking algorithm, thereby greatly reducing the amount of data transmitted to the cloud server. The feasibility and applicability of the developed sensing system are experimentally validated on a laboratory-scaled transmission tower structure. The proposed EdgeCVDMS is cost-effective, easily deployable, and of great potential to be applied for the condition assessment of a larger population of aging civil infrastructure.

Full-scale structural displacement measurement with camera ego-motion compensation using RGB and LiDAR cameras

Displacement measurements based on computer vision techniques have shown significant potential in structural health monitoring. However, as the inevitable camera ego-motion results in a drift in the measured displacement, compensation for such errors must be performed carefully. Although an additional RGB camera is considered in conventional methods for error compensation, the need for the sub-target limits its application. This study presents a dual-camera system, consisting of RGB and light detection and ranging (LiDAR) cameras, that enables accurate camera ego-motion compensation for long-term structural displacement measurements. In particular, the LiDAR camera of the proposed system can measure the six-degrees-of-freedom (6-DOF) ego-motion of the camera body regardless of the sub-target and its elaborate design. The accuracy of the proposed method is evaluated by comparing it with that of a conventional RGB camera. The resulting displacement can be employed to monitor the long-term behavior of full-scale bridges.

Applicability of Digital Image Photogrammetry to Rapid Displacement Measurements of Structures in Restricted-Access and Controlled Areas: Case Study in Korea

Critical facilities are generally located in areas with restricted or controlled access, making it difficult for experts to monitor the structural health of enclosed infrastructures. Hence, a case study was conducted in South Korea to evaluate the applicability of digital image photogrammetry using commercial imaging devices in order to quickly measure the structural deformations of infrastructures in such areas. The applicability evaluation involved measuring the displacement of mechanically stabilized earth (MSE) walls. In the experiment, the displacement of MSE walls was first measured using the traditional monitoring method and the results were compared with those obtained via digital image photogrammetry using commercial imaging devices such as digital cameras and cellphones. The measurement results obtained with the cellphone camera had a maximum error of approximately 20 mm when compared with the results of the traditional monitoring method. Because this is a significant error, even when considering the mechanical error in the traditional monitoring method’s result, it was determined that monitoring using a cellphone camera is infeasible. However, the experimental results of digital image photogrammetry using a digital camera showed a maximum error of approximately 9 mm. Although this is a sizable error, the method was assessed to be technically feasible.

Development of an iterative Otsu method for vision-based structural displacement measurement under low-light conditions

Vision-based structural displacement measurement method presented great potential for structural health monitoring application due to its distinguished advantages, and LED targets were always employed to deal with the issue of poor visibility under low-light conditions. However, the ambient light and unfavorable weather seriously influence the identification performance even with LED target. In this paper, an iterative Otsu method was developed to help to realize the correct segmentation of LED spot image by iteratively calculating the image threshold, so as to obtain the structural displacement under low-light conditions even with environmental interference. Furthermore, a simply supported beam model and a cable-stayed bridge model were experimentally investigated under low-light conditions with strong-light and fog interference, and the final results demonstrated that comparing to existing Otsu method and template matching method, the developed method presents an excellent anti-disturbance ability against the environmental interference to realize the correct identification and guarantee the computational efficiency.

A Novel Method for Full‐Field Scale Factor Calculation under Off‐Axis Measurements for Vision‐Based Structural Multipoint Displacement Measurement and Health Monitoring

For vision‐based structural displacement measurement and health monitoring, the scale factor (SF) calculation plays a pivotal role in converting the pixel displacement into the actual one. On the other hand, for the current SF calculation methods, the object distance, between the to‐be‐measured points on the object plane and the optical center of the shooting instrument, has to be measured in advance due to the existence of the pitch and horizontal angle under off‐axis measurement. Unfortunately, it is usually inefficient, difficult, and even impossible to obtain the object distances for all the to‐be‐measured points, especially for the full‐field displacement monitoring on huge scale structures. In this paper, a novel SF calculation method was proposed to calculate the full‐field scale factor of the host structure under off‐axis measurement by combining with the similarity relation of the camera imaging model. With the help of this method, all the scale factors of the to‐be‐measured points can be calculated accurately and highly efficiently, as long as the object distance of any one point or the geometric dimensions of any feature object on the object surface is provided. In addition, to quantitatively assess the SF calculation accuracy of the method, a static vision‐based measurement investigation was firstly conducted, and then, two experimental investigations about the multipoint and full‐field structural displacement measurement on a cable‐stayed bridge model and a simply supported beam model were conducted to validate the effectiveness and feasibility of the proposed method. Finally, all the results demonstrated that the proposed method exhibits an excellent performance on the SF calculation under off‐axis measurement and provides a great potential to be utilized for the vision‐based structural multipoint and full‐field displacement measurement and health monitoring.

Video Motion Magnification and Subpixel Edge Detection-Based Full-Field Dynamic Displacement Measurement

Noncontact measurement techniques in structural dynamics field have progressed significantly in the past few decades. Vision-based measurement techniques are unique in that they have the ability to achieve full-field measurement and possess the typical advantages associated with noncontact measurement techniques. Recently, vision-based techniques have also been applied to streaming of videos for structural dynamic displacement measurement. The most recent trends in vision-based measurements include target tracing, digital image correlation, and target-less approaches. There are, however, some shortcomings of the vision-based techniques such as susceptibilities to image noise, prevailing light conditions, and limit in measurement resolution. To reduce these shortcomings, a method known as video motion magnification (MM) can be used to amplify small structural motions. Using the phase-based motion magnification (PBMM) and subpixel edge detection methods, the full-field dynamic displacements of the structure can be obtained. The deep convolutional long short-term memory (ConvLSTM) network is applied to aid in the selection of the frequency band for magnification in the PBMM algorithm. To achieve higher measurement accuracy, the displacement results with and without MM are combined with the finite impulse response (FIR) filter which can reduce the error caused by the PBMM procedure. In the tests, plastic optical fiber (POF) displacement sensors are introduced and used as reference measurements to compare the dynamic displacement results from the proposed vision-based method. Compared with the measured displacements with POF sensors, the proposed method offers high level of accuracy for full-field displacement measurement.

Cover Image, Volume 38, Issue 5

Computer-Aided Civil and Infrastructure EngineeringVolume 38, Issue 5 COVERFree Access Cover Image, Volume 38, Issue 5 First published: 01 March 2023 https://doi.org/10.1111/mice.12986AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Graphical Abstract On the cover: The cover image is based on the Research Article A robust subpixel refinement technique using self-adaptive edge points matching for vision-based structural displacement measurement by Miaomin Wang et al., https://doi.org/10.1111/mice.12889. Volume38, Issue515 March 2023 RelatedInformation

Improved image-based, full-field structural displacement measurement using template matching and camera calibration methods

Although the use of artificial-markers improves the performance of the image-based procedures, there is a problem with the access to their installation place. Moreover, increasing the accuracy and efficiency of the available methods for structural-health-monitoring (SHM) is an important issue in recent years. Therefore, the purpose of this study is to enhance the image-based, full-field displacement measurement using new template-matching and camera-calibration methods. Furthermore, considering practical methods to increase the number of pixels in the images, the possibility of using free markers and improving the camera-calibration methods are investigated. The suitability of the proposed procedures was experimentally evaluated using single-degree-of-freedom and multi-degrees-of-freedom structural models and a railway bridge as a realistic application of the introduced methods. The proposed techniques are independent of employing artificial-markers or checkerboards with marker-less calibration abilities. Also, the effectiveness of the proposed schemes lies in the acceptable accuracy, computational efficiency, and sufficient applicability for SHM problems.

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

Vision-based structural displacement measurement under ambient-light changes via deep learning and digital image processing

Structural displacement can provide critical information for structural safety assessment and maintenance. Although vision-based displacement measurement has significant advantages over traditional methods, it still faces challenges in long-term field applications owing to environmental uncertainties. This study proposes a novel displacement measurement method based on deep learning and digital image processing to mitigate the effects of ambient-light changes. The proposed method can automatically extract the calibration object in complex scenarios using You Only Look Once (YOLO) v5 and locate the calibration object under 24-h ambient-light changes precisely. Short- and long-term experiments were conducted in the laboratory to evaluate the performance of the method, and the short-term experimental results were compared with laser displacement sensor (LDS) data, which showed a maximum and minimum relative error of 0.4122% and 0.0024%. The long-term experimental results showed that the displacement responses were within ±1.0 mm. Hence, this method has good potential in structural displacement measurements.

Configuration and Performance Test of GNSS-IMU Sensor Fusion Module for Structural Displacement Measurement

Detecting the displacement of a structure by external forces in advance is very important in monitoring the health of the structure. In this study, a kind of sensor fusion technique that combines low-cost GNSS and IMU for structural displacement measurement was established. In addition, a data acquisition program that can secure the synchronous measurement between sensors was created. In order to evaluate the performance of this measurement module, a model experiment gauging the true value by the laser displacement meter was performed, and the results of GNSS single positioning, VRS positioning, precision single positioning (PPP), and sensor merging positioning were compared and analyzed. As a result, it was found that the PPP positioning did not capture the vibration of the structure at all regardless of the waiting time within the excitation condition of the model structure. On the other hand, GNSS single positioning was able to obtain similar results to true values and VRS positioning by pre-processing with an appropriate high-pass filter and the sensor fusion technique.

Vision‐based displacement measurement using an unmanned aerial vehicle

Vision-based displacement measurement for structural health monitoring has gained popularity in recent years but still has several limitations in practical applications. Unmanned aerial vehicles (UAVs) provide opportunities to address the bottleneck problems of camera resolution insufficiency and mounting inconvenience due to their high maneuverability. However, existing methods using UAVs for structural displacement measurement are often complicated to operate due to the use of multiple stationary markers or multiple UAVs. This paper describes a novel vision-based displacement measurement approach, using only one UAV, along with a motionless laser spot projected from a distance away as a reference. The positions of the marker and the laser spot are precisely calculated using a two-step strategy, in which a designed black and white marker of known size is applied to the structure for scale definition and precise positioning. The adaptive region of interest (ROI) and adaptive binarization methods are utilized to improve the automatic applicability of the proposed approach with various background and brightness values. In this way, the motion of the UAV parallel and perpendicular to the plane of the structure can be eliminated by the stationary reference laser spot and the constantly updated scaling factors, respectively. The performance of the proposed method is validated on a two-story frame and a suspension bridge. The results show that the displacement measured using the UAV agrees with the reference data obtained using the laser displacement sensor and the stationary camera, thereby demonstrating the accuracy and feasibility of the proposed method for displacement measurement for small- and large-scale infrastructure.

A novel gradient-based matching via voting technique for vision-based structural displacement measurement

The effectiveness of vision-sensing system for field applications in remote measurements of structural displacement can be adversely affected by environmental factors, leading to an incomplete recording of a moving feature target in recorded video frames. The result is a non-ideal solution using traditional template matching techniques, which presents significant challenges for accurate measurements. This paper proposes a novel gradient-based matching via voting (GMV) technique to overcome environmental and operational conditions and provide reliable tracking of moved targets with different degrees of feature loss. When evaluating the region similarity between the template and a subset (called the overlapping region) of video frames in which the moved target is located to obtain a similarity score of the subset, the traditional template matching does not ascribe weights to the similarity at the pixel level, whereas GMV uses a voting scheme to weight all the pixel similarities. Three experiments were used to test GMV, evaluating its tracking accuracy and verifying its robustness with different levels of target feature loss based on a comparison with two commonly used template matching techniques. GMV proved to be capable of retrieving accurate displacement data with a high degree of accuracy, even for a moved target with up to 90% feature losses.

Computer vision-based illumination-robust and multi-point simultaneous structural displacement measuring method

Computer vision-based techniques for structural displacement measurement are rapidly becoming popular in civil structural engineering. However, most existing computer vision-based displacement measurement methods require man-made targets for object matching or tracking, besides usually the measurement accuracies are seriously sensitive to the ambient illumination variations. A computer vision-based illumination robust and multi-point simultaneous measuring method is proposed for structural displacement measurements. The method consists of two part, one is for segmenting the beam body from its background, the segmentation is perfectly carried out by fully convolutional network (FCN) and conditional random field (CRF); another is digital image correlation (DIC)-based displacement measurement. A simply supported beam is built in laboratory. The accuracy and illumination robustness are verified through three groups of elaborately designed experiments. Due to the exploitation of FCN and CRF for pixel-wise segmentation, numbers of locations along with the segmented beam body can be chosen and measured simultaneously. It is verified that the method is illumination robust since the displacement measurements are with the smallest fluctuations to the illumination variations. The proposed method does not require any man-made targets attached on the structure, but because of the exploitation of DIC in displacement measurement, the regions centered on the measuring points need to have texture feature.

Vision‐Based Dynamic Displacement Measurement of Isolation Bearing

The existing computer vision‐based structural displacement measurement method can only obtain structural displacement when the camera is fixed or the pose of the camera is known at all times. However, the aforementioned conditions cannot be met when an earthquake occurs. Therefore, this study presents a method to achieve a dynamic displacement measure of an isolated bearing from a video without the known camera motion: two cameras are set up along the horizontal orthogonal direction, and circular targets are set at the upper and lower connecting plates of the isolation bearing. A combination of HSV color space, Canny edge detection, and Hough transform circle detection is used to obtain the dynamic displacement of the isolation bearing. This task is done by measuring the relative displacement between the upper and lower target points. A basic mechanical performance test of LRB500 isolation bearing is used to verify the proposed method. The result shows that the horizontal displacement time history of the isolation bearing obtained by this method is almost the same as the result obtained by the displacement meter whether the camera is moving or fixed. However, compared with the results measured by the displacement meter, the absolute error of the peak horizontal and vertical displacements are both less than 1 mm, which indicates that the proposed method can be used in displacement monitoring under long‐term load and earthquake action of isolation bearing.