3D Displacement Measurement Research Articles

3D displacement measurement using a single-camera and mesh deformation neural network

Computer vision-based methods for civil structure’s vibration displacement measurement have emerged as useful tools in the recent years. These methods offer several benefits including non-contact measurements, cost-effectiveness, and the ability to capture full-field displacement. Yet, there remain certain challenges. Measuring vibration displacement in 3D typically requires multiple cameras, adding complexity to camera configurations. Moreover, existing methods relied heavily on physical markers or natural key points. Placing physical markers on structures is often impractical, and natural key points are difficult to detect on structures with few distinct features or during rapid movements. Contrary to previous approaches, this paper presents a novel technique that uses a monocular camera for 3D displacement measurements. This technique obviates the need for physical markers or the reliance on natural key points, representing a significant advancement. Central to the method is a deep neural network designed to predict 3D mesh deformation directly from a single image input, combined with an initial 3D cube mesh input. A synthetic 3D dataset is generated to train the neural network. In the testing phase for real structures, advanced video segmentation method is employed to remove the background in order to enhance the prediction accuracy. The practical efficacy of this methodology is validated in a laboratory through a series of experimental tests on beam structures, demonstrating reliable results and application potentials.

High Power Pulsed LED Driver for Vibration Measurements.

Vibration measurements pose specific experimental challenges to be faced. In particular, optical methods can be used to obtain full-field vibration information. In this scenario, stereo-camera systems can be developed to obtain 3D displacement measurements. As vibration frequency increases, the common approach is to reduce camera exposure time to avoid blurred images, which can lead to under-exposed images and data loss, as well as issues with the synchronization of the stereo pair. Both of these problems can be solved by using high-intensity light pulses, which can produce high-quality images and guarantee camera synchronization since data is saved by both cameras only during the short-time light pulse. To this extent, high-power Light-Emitting Diodes (LEDs) can be used, but even if the LED itself can have a fast response time, specific electronic drivers are needed to ensure the desired timing of the light pulse. In this paper, a circuit is specifically designed to achieve high-intensity short-time light pulses in the range of 1 µs. A prototype of the designed board was assembled and tested to check its capability to respect the specification. Three different measurement methods are proposed and validated to achieve short-time light pulse measurements: shunt voltage measurement, direct photodiode measurement with a low-cost sensor, and indirect pulse measurement through a low-frame-rate digital camera.

Soft tissue material properties based on human abdominal in vivo macro-indenter measurements

Simulations of human-technology interaction in the context of product development require comprehensive knowledge of biomechanical in vivo behavior. To obtain this knowledge for the abdomen, we measured the continuous mechanical responses of the abdominal soft tissue of ten healthy participants in different lying positions anteriorly, laterally, and posteriorly under local compression depths of up to 30 mm. An experimental setup consisting of a mechatronic indenter with hemispherical tip and two time-of-flight (ToF) sensors for optical 3D displacement measurement of the surface was developed for this purpose. To account for the impact of muscle tone, experiments were conducted with both controlled activation and relaxation of the trunk muscles. Surface electromyography (sEMG) was used to monitor muscle activation levels. The obtained data sets comprise the continuous force-displacement data of six abdominal measurement regions, each synchronized with the local surface displacements resulting from the macro-indentation, and the bipolar sEMG signals at three key trunk muscles. We used inverse finite element analysis (FEA), to derive sets of nonlinear material parameters that numerically approximate the experimentally determined soft tissue behaviors. The physiological standard values obtained for all participants after data processing served as reference data. The mean stiffness of the abdomen was significantly different when the trunk muscles were activated or relaxed. No significant differences were found between the anterior-lateral measurement regions, with exception of those centered on the linea alba and centered on the muscle belly of the rectus abdominis below the intertubercular plane. The shapes and areas of deformation of the skin depended on the region and muscle activity. Using the hyperelastic Ogden model, we identified unique material parameter sets for all regions. Our findings confirmed that, in addition to the indenter force-displacement data, knowledge about tissue deformation is necessary to reliably determine unique material parameter sets using inverse FEA. The presented results can be used for finite element (FE) models of the abdomen, for example, in the context of orthopedic or biomedical product developments.

Self-calibrating technique for 3D displacement measurement using monocular vision and planar marker

Although the vision-based displacement measurement of structures has been widely studied for decades, it still poses challenges for practical applications, including the issues of calibration and automatic long-term monitoring. This paper introduces a monocular-vision based self-calibrating technique for accurately measuring the 3D structural displacement. The technique utilizes a tailor-made planar marker with unique graph patterns, enabling precise image recognition and PnP resolution. The proposed technique automatically estimates the 3D position and 3D orientation of the marker. The limitations of marker-free techniques in terms of long-term monitoring have also been discussed. The accuracy and robustness of the proposed technique are systematically examined by Monte-Carlo simulation, and then verified by experiments. Results show that accuracy of the proposed technique for 3D displacement measurement is 0.049 mm, which indicates the effectiveness of the technique for automatically measuring and calibrating stereo displacement of structures using monocular vision.

Unsupervised CNN-based DIC method for 2D displacement measurement

Digital image correlation (DIC) is a widely used technique for non-contact measurement of deformation. However, traditional DIC methods face challenges in balancing calculation efficiency and the quantity of seed points. Deep learning approaches, particularly supervised learning methods, have shown promise in improving DIC efficiency. However, these methods require high-quality training data, which can be time-consuming to generate ground truth annotations. To address these challenges, we propose an unsupervised convolutional neural network (CNN) based DIC method for 2D displacement measurement. Our approach leverages an encoder-decoder architecture with multi-level feature extraction, a dual-path correlation block, and an attention block to extract informative features from speckle images with varying characteristics. We utilize a speckle image warp model to transform the deformed speckle image to the predicted reference speckle image based on the predicted 2D displacement map. The unsupervised training is achieved by comparing the predicted and original reference speckle images. To optimize the network's parameters, we employ a composite loss function that takes into account both the Mean Squared Error (MSE) and Pearson correlation coefficient. By using unsupervised convolutional neural network (CNN) based DIC method, we eliminate the need for extensive training data annotation, which is a time-consuming process in supervised learning DIC methods. We have conducted several experiments to demonstrate the validity and robustness of our proposed method. The results show a significant reduction in Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) compared to a method proposed by Zhao et al. This indicates that our unsupervised CNN-based DIC approach can achieve accuracy comparable to supervised CNN-based DIC methods. For implementation and evaluation, we provide PyTorch code and datasets, which will be released at the following URL :https://github.com/fead1/DICNet-corr-unsupervised-learning-.

Phase-based motion estimation and SVR smooth for target-free 3D deformation measurement using stereophotogrammetry

Non-contact measurement methods based on computer vision have shown great potential for measuring structural surface displacement and strain due to their low cost and high efficiency. However, the traditional computer vision technology based on digital image correlation (DIC) is limited by its dependence on natural texture features of the surface and its susceptibility to environmental conditions, making it challenging to obtain effective and robust measurements of surface deformation. Additionally, these methods typically only focus on the structural response in the image plane, which makes it difficult to perform a comprehensive health assessment of complex structural components with asymmetry. Therefore, a time-domain visual measurement technique based on image phase information is proposed for measuring three-dimensional (3D) deformation of structures using stereophotogrammetry. The method involves extracting image phase information using 2D Gabor filters and applying it to dense optical flow estimation based on polynomial approximation and semi-global block matching (SGBM) for obtaining robust 3D displacement measurements. The support vector regression (SVR) is then used to perform hyperplane fitting of the original displacements for improving the smoothness of the 3D displacements. Finally, an intuitive strain conversion method is introduced to convert the 3D displacements into 3D strains. The performance of the proposed technique has been validated through numerical simulation experiments based on a physics-based graphics model (PBGM) in a synthetic environment. The robustness of the proposed method has been further proved through free attenuation vibration experiments on a steel ruler. The initial evaluation of the expected performance of the measurement plan has been enabled by performing a 1:1 reduction of the field measurement process in a synthetic environment. The results indicate that the proposed method is a promising method for accurately and reliably measuring structural deformation in both synthetic and real-world environments, especially for complex structural components with asymmetry.

In Situ Structural Health Monitoring of Full-Scale Wind Turbine Blades in Operation Based on Stereo Digital Image Correlation

Structural health monitoring (SHM) and the operational condition assessment of blades are greatly important for the operation of wind turbines that are at a high risk of disease in service for more than 5 years. Since certain types of blade faults only occur during wind turbine operation, it is more significant to perform in situ SHM of rotating full-scale blades than existing SHM of small-scale blades or static testing of full-scale blades. Considering that these blades are usually not prefabricated with relevant sensors, this study performed SHM and condition assessment of full-scale blades in operation with stereo digital image correlation. A self-calibration method adapted to the outdoors with a large field of view was introduced based on the speckled patterns. To accurately obtain the in- and off-plane deformation, a new reference frame is constructed at the center of the rotation of the blades. The 3D displacements of the points of interest (POIs) on the blade of a 2 MW wind turbine were characterized. Furthermore, the frequency spectrum of the measured 3D displacements of the blades was compared with the blades with the faults. The results showed that the introduced technique is a convenient and nondestructive technique that enables SHM of full-scale wind turbine blades in operation.

Monocular vision based 3D vibration displacement measurement for civil engineering structures

Vibration displacement of civil structures are crucial information for structural health monitoring (SHM). However, the challenges and costs associated with traditional physical sensors make displacement measurement difficult. In recent years computer vision (CV) techniques have been employed for measuring vibration displacement in civil structures. There has been a growing interest in CV-based three-dimensional (3D) displacement measurement, as it provides comprehensive information for structural health assessment. Most existing methods use multi-view geometry, requiring multiple cameras for depth measurement. This paper proposes a new system for measuring the 3D vibration displacement utilising a single camera. Instead of using multi-view geometry, deep neural networks are utilised to learn the depth of scenes from monocular images. Compared with the multi-view methods, the proposed 3D measurement system with monocular vision is more cost-effective and much more convenient to set up and use in practice, avoiding the complicated calibration and object matching between multiple cameras. Experimental tests are conducted in the laboratory to investigate the feasibility of the proposed system. Physical displacement sensors are equipped with the testing structure to provide the ground truth data. The results demonstrate that the proposed monocular 3D displacement system is able to produce reasonable 3D full-field displacement measurement, which makes monocular image based CV system a promising approach to achieve 3D displacement measurement, with its obvious advantages in cost and convenience compared to the traditional sensor-based or multi view CV-based methods.

GNSS-Over-Fiber Sensing System for High Precision 3D Nodal Displacement and Vibration Detection

Aiming at realizing comprehensive structural health monitoring, a GNSS (Global Navigation Satellite System) over fiber sensing approach is presented and experimentally demonstrated to achieve 3D nodal displacement and vibration detection simultaneously. GNSS signals from multi-antennas are transferred to a receiver based on an optical fiber link, also used as distributed sensing fiber. Broadband linear frequency-modulated (LFM) probe light with high linearity is generated based on electro-optic modulation to probe distributed vibrations and monitor the transmission time delay along the whole fiber with high precision. The spectral components generated by vibrations are detected using the cross-correlation method to achieve distributed vibration sensing. In addition, with the assistance of high-precision hardware-delay measurement, the variation of transmission time delay can be obtained, improving the precision of 3D displacement measurement in the GNSS-over-fiber architecture. In the proof-of-concept experiment, a 3D baseline can be obtained with the precision of 2.8, 3.6, and 2.0 mm, while vibrations with frequencies of 40 kHz, 80 kHz, and 100 kHz along 250 m fiber are successfully detected with a spatial resolution of 1 m.

InSARTrac Field Tests—Combining Computer Vision and Terrestrial InSAR for 3D Displacement Monitoring

InSARTrac is an innovative method for 3D displacement monitoring that combines terrestrial interferometric synthetic aperture radar (InSAR) and computer vision-based feature tracking. The 3D measurements obtained are considered far superior to 1D or 2D data and facilitate evaluations concerning the mechanisms controlling kinematics. This study presents the results of InSARTrac measurements at the Mölltal Glacier in Carinthia, Austria. The duration of glacier monitoring was four weeks and involved two instrument setup positions to obtain comparative measurements of supraglacial rock debris from different angles without utilizing retroreflectors. The mean displacement rate of the resultant vector is 22 mm/day and includes ~11 mm/day in the downgradient ice surface direction and 6 to 18 mm/day vertically downward. Additionally, the entire glacier surface was measured three times using a LIDAR-equipped UAV, revealing mean vertical displacements of 16 mm/day. The measurements indicate an InSARTrac accuracy of 4.2 ppm, which is 27% lower than in the initial controlled tests utilizing retroreflectors. The field test demonstrates the capability of InSARTrac to provide meaningful 3D displacement measurements of supraglacial rock debris. The material monitored has texture and reflectivity similar to certain classes of landslides, rock glaciers, and other alpine processes, indicating that InSARTrac has promising applications for monitoring a variety of geologic phenomena.

A high-sensitivity rotatable 3D displacement sensor

Aiming at the problems of low sensitivity and low accuracy caused by the displacement transfer mechanism of three displacement sensors used simultaneously in the 3D displacement monitoring of seismic isolation bearings, the paper has proposed a high-sensitivity rotatable 3D displacement sensor. The sensor adds through holes on the surface of the equal-strength cantilever beam to form a cross beam, which increases the bending strain on the beam surface to improve the sensitivity. By adding a gyroscope and a mechanical rotation structure, a single sensor can measure the 3D displacement at the same time, reducing the adverse effects displacement transmission mechanism on the accuracy of the measurement. ANSYS software was used to simulate and optimize the parameters of the size of through-hole of the sensor beam to determine the appropriate size and location of the through-hole. Finally, the sensor was developed and its static characteristics and displacement measurement performance in static and dynamic 3D space were tested based on the simulation results. The test results have shown that the sensor has a sensitivity of 16.29 mV/mm and an accuracy of 0.9% in the range of 0–160 mm. Its static and dynamic 3D spatial displacement measurement errors are less than 2 mm, which can meet the accuracy requirements of 3D displacement measurement and sensitivity for structural health monitoring of seismic isolation bearings.

Optimization of Mirror Positioning for Single‐Camera 3D Displacement Measurements

AbstractCamera‐based 3D displacement measurements are only possible when two different viewing angles of the respective measurement point are available. We earlier introduced a simplified 3D reconstruction algorithm for camera‐mirror measurements where the mirror reflects the second view onto the camera sensor to eliminate the need for a second (costly) high speed camera. In this study, we concentrate on formulating guidelines on how to position the mirror with respect to the camera and the measured structure, the goal being to minimize errors arising in the 3D reconstruction procedure. The focus lies on the angle between the optical axes of both camera views. We hereby also take into account limitations in the experimental setup that the special camera‐mirror‐setup brings with it. This study is conducted in terms of simulations using the animation software BLENDER as well as real experiments on a simple vibrating structure.

Panoramic Digital Image Correlation for 360-Deg Full-Field Displacement Measurement

In full-field 3D displacement measurement, stereo digital image correlation (Stereo-DIC) has strong capabilities. However, as a result of difficulties with stereo camera calibration and surface merging, 360-deg panoramic displacement measurements remain a challenge. This paper proposes a panoramic displacement field measurement method in order to accurately measure the shape and panoramic displacement field of complex shaped objects with natural textures. The proposed method is based on the robust subset-based DIC algorithm and the well-known Zhang’s calibration method to reconstruct the 3D shape and estimate the full-field displacements of a complex surface from multi-view stereo camera pairs. The method is used in the determination of the scale factor of the 3D reconstructed surface and the stitching of multiple 3D reconstructed surfaces with the aid of the laser point cloud data of the object under test. Based on a discussion of the challenges faced by panoramic DIC, this paper details the proposed solution and describes the specific algorithms implemented. The paper tests the performance of the proposed method using an experimental system with a 360-deg six camera setup. The system was evaluated by measuring the rigid body motion of a cylindrical log sample with known 3D point cloud data. The results confirm that the proposed method is able to accurately measure the panoramic shape and full-field displacement of objects with complex morphologies.

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

3D Displacement Measurement of Railway Bridge According to Cyclic Loads of Different Types of Railcars with Sequential Photogrammetry

In the early days of railroads in Korea, railway bridges were constructed as steel plate-girder structures, which are vulnerable to vibration and torsion. Many of these bridges have since been replaced with concrete-slab structures, which have high stability. Nevertheless, steel railway bridges still remain all over the country, and a lot of manpower and cost is being invested in the maintenance and repair of such bridges. Moreover, there have not been experimental analyses aiming to measure the cyclic loads that occur when a train enters. To ensure bridge safety, it is necessary to periodically inspect deformations. To this end, the present study proposed a sequential photogrammetric technique for measuring the deformation of a steel railway bridge for three types of railcars. Sequential stereo images of the bridge with multiple feature points are obtained using sequential photographing cameras, to determine the ground coordinates of each point as a function of time based on the space intersection from the relative orientation with coplanarity and the scale adjustment. All of these processes are performed through automated techniques using only the cameras and the targets. With this setup, the 3-dimensional dynamic motions of the bridge due to the cyclic loading of trains could be measured. In addition, the displacements by the proposed method were compared to those obtained with the 3D Laser tracker. The horizontal displacements errors did not exceed 0.5 mm and the vertical error was within 2.3 mm in root mean square error (RMSE) at camera-to-object distances of about 9 m.

LOW-COST DEPTH-CAMERA: OPEN-SOURCE 3D DISPLACEMENT MEASUREMENTS FOR 4D PRINTED HYGROSCOPIC COMPOSITES

Abstract. 4D printing (4DP) is a growing branch of 3D printing technology that involves the design of composite material architectures capable of shape-change transformations, which occur post printing, in response to external stimulus. Among these, Wood Polymer Composites (WPCs) change their shape in reaction to changes of moisture content, shrinking or swelling like natural wood until the equilibrium with the environment is reached. Such intrinsic material behavior can be particularly useful in the development of passive moisture airflow controllers that can modulate humidity and airflow in indoor environments to improve air quality. Precise measurement of the time-based stimulus induced shape-change response of these composites is critical to assess the responsiveness, velocity of reaction and overall deformation of the designed 4DP composite mechanisms. Up until now, Digital Image Correlation (DIC) techniques have been widely used for such purpose. However, DIC methods require expensive equipment and costly commercial software. This paper presents a Low-Cost Depth-Camera (LCDC) method that uses a free custom algorithm that returns a 3D coloured displacement map with the corresponding meshes of the acquired object. The LCDC method does not require specialized equipment and allows for an overall understanding of the time-dependent deformation of 4DP actuators, this method also facilitates the comparison between composites with different properties under the same external conditions. This new LCDC method has the potential to further 4DP research by providing an open-source, accessible and reliable tool to assess 3D displacement measurements.

Three dimensional deformation measurement method based on image guided point cloud registration

3D geometric capturing technologies based on camera and laser scanning have undergone huge advances. However, accurate 3D displacement measurement especially for in-plane movement based on point clouds remain unsolved problems. To deal with this challenge, a novel 3D deformation measurement method based on image guided point cloud registration is proposed. 2D digital image correlation method is adopted to calculate image displacement field and used to search the corresponding point from deformed point clouds. Then, accurate 3D displacement field can be obtained. Experiments have verified that the proposed method can realize accurate 3D displacement measurement at both short and far distance. The precision of 3D displacement measurement is 0.1 mm and 1 mm with structured light system and camera assisted TLS system separately.

A New 2D Displacement Measurement Method Based on an Eddy Current Sensor and Absolute Encoding

A new method of two-dimensional (2D) plane displacement measurement based on an eddy current sensor is proposed in this paper. A series of grooves with different widths and depths are graved on the linear displacement table to form 2D absolute coding using the idea of pseudorandom coding. The eddy current sensor array is arranged above the groove to identify the coding. An artificial neural network is used to establish a measurement model which is the mapping relationship between the output of the eddy current sensor array and the 2D displacement of the workbench. A feasibility experiment showed that in the range of 20 × 20 mm, the root mean square error of measurement in the X- and Y-directions are 83 and 73 μm, respectively. The new method integrates eddy current sensor and artificial neural network modeling to realize 2D displacement measurement, which provides a new solution for displacement and angle measurement.

Calculating Co-Seismic Three-Dimensional Displacements from InSAR Observations with the Dislocation Model-Based Displacement Direction Constraint: Application to the 23 July 2020 Mw6.3 Nima Earthquake, China

As one of the most prevailing geodetic tools, the interferometric synthetic aperture radar (InSAR) technique can accurately obtain co-seismic displacements, but is limited to the one-dimensional line-of-sight (LOS) measurement. It is therefore difficult to completely reveal the real three-dimensional (3D) surface displacements with InSAR. By employing azimuth displacement observations from pixel offset tracking (POT) and multiple aperture InSAR (MAI) techniques, 3D displacements of large-magnitude earthquakes can be obtained by integrating the ascending and descending data. However, this method cannot be used to accurately realize the 3D surface displacement measurements of small-magnitude earthquakes due to the low accuracies of the POT/MAI-derived azimuth displacement measurements. In this paper, an alternative method is proposed to calculate co-seismic 3D displacements from ascending and descending InSAR-LOS observations with the dislocation model-based displacement direction constraint. The main contribution lies in the two virtual observation equations that are obtained from the dislocation model-based forward-modeling 3D displacements, which are then combined with the ascending/descending InSAR observations to calculate the 3D displacements. The basis of the two virtual observation equations is that the directions of the 3D displacement vectors are very similar for real and model-based 3D displacements. In addition, the weighted least squares (WLS) method is employed to solve the final 3D displacements, which aims to consider and balance the possible errors in the InSAR observations as well as the dislocation model-based displacement direction constraint. A simulation experiment demonstrates that the proposed method can achieve more accurate 3D displacements compared with the existing methods. The co-seismic 3D displacements of the 2020 Nima earthquake are then accurately obtained by the proposed method. The results show that co-seismic displacements are dominated by the vertical displacement, the magnitude of the horizontal displacement is relatively small, and the overall displacement pattern fits well with the tensile rupture.

3D Measurement of Large Deformations on a Tensile Structure during Wind Tunnel Tests Using Microsoft Kinect V2.

Wind tunnel tests often require deformation and displacement measures to determine the behavior of structures to evaluate their response to wind excitation. However, common measurement techniques make it possible to measure these quantities only at a few specific points. Moreover, these kinds of measurements, such as Linear Variable Differential Transformer LVDTs or fiber optics, usually influence the downstream and upstream air fluxes and the structure under test. In order to characterize the displacement of the structure not just at a few points, but for the entire structure, in this article, the application of 3D cameras during a wind tunnel test is presented. In order to validate this measurement technique in this application field, a wind tunnel test was executed. Three Kinect V2 depth sensors were used for a 3D displacement measurement of a test structure that did not present any optical marker or feature. The results highlighted that by using a low-cost and user-friendly measurement system, it is possible to obtain 3D measurements in a volume of several cubic meters (4 m × 4 m × 4 m wind tunnel chamber), without significant disturbance of wind flux and by means of a simple calibration of sensors, executed directly inside the wind tunnel. The obtained results highlighted a displacement directed to the internal part of the structure for the side most exposed to wind, while the sides, parallel to the wind flux, were more subjected to vibrations and with an outwards average displacement. These results are compliant with the expected behavior of the structure.