Three-dimensional Displacement Measurement Research Articles

Vision-based digital shadowing to reveal hidden structural dynamics of a real supertall building

Vision-based digital shadowing is a highly efficient way to monitor the health of buildings in use. However, previous studies on digital shadowing have been limited to laboratory experiments. This paper proposes a novel computer-vision-based digital shadow workflow and presents its successful application in a real engineering case. In this case, a 345.8-m supertall building experienced unexpected shaking under normal meteorological conditions. This study established a digital shadow of the building using three-dimensional displacement measurements based on super-resolution monocular vision, revealing the hidden structural dynamics and inherent mechanical reasons for the abnormal shaking. The proposed digital shadowing workflow is a feasible roadmap for developing vision-based digital shadows of real-world structures using low-cost cameras. The abnormal vibration event in the supertall building considered in this study is the first of its type worldwide. The results of this study offer practical strategies and invaluable insights into the prevention and mitigation of this type of global risk, thereby contributing to the lifespan extension of buildings in use worldwide. Furthermore, with the increasing number of general sensing devices, such as surveillance cameras in cities, the proposed method may unleash the immense potential of general sensing devices in achieving the leap from structural health monitoring to city health monitoring.

An improved DFD method for three-dimensional displacement measurement of vision-based tactile sensor

Currently, traditional tactile sensors based on the principles of capacitance or piezoelectricity have complex structures and difficulty in obtaining tactile information. A vision-based tactile sensor is introduced which can realize visual measurement of three-dimensional displacement in this paper. The vision-based tactile sensor is mainly composed of an elastomer embedded with marker point array, a transparent acrylic plate, 8 LED lights and a micro monocular camera. The elastomer deforms when the tactile sensor contacts an object, and the micro monocular camera is used to capture the elastomer deformation and transmit it to the computer in the form of image, and then the three-dimensional displacement information is obtained by processing the image. In order to more accurately recover the missing dimensional information in the three-dimensional displacement detection of monocular camera, an improved DFD (Depth from Defocus) method based on finite element theory is proposed in this paper. It is verified by experiments that the improved DFD method proposed in this paper can measure the three-dimensional displacement information more accurately compared with the DFD method. In addition, an experiment is conducted to prove the robustness of the improved DFD method on the robotic gripper. The experimental results demonstrate that the three-dimensional displacement measurement method proposed in this paper can provide technical support for the design and development of vision-based tactile sensors.

Research on image deformation monitoring algorithm based on binocular vision

Monitoring the deformation of critical structures such as tunnel、buildings and bridges is critical to ensure the safety of infrastructures. Non-contact deformation monitoring technology based on digital imagery has emerged as a prevalent and effective monitoring method, which enables cost-effective and efficient tracking of structural deformations. This paper aims to develop an automated deformation monitoring methodology using the binocular vision. First, standard preprocessing of digital images is performed through the integration of filtering algorithms and image enhancement techniques. Furthermore, by combining the Scale Invariant Feature Transform (SIFT) feature matching algorithm, the inverse compositional gauss–newton iterative algorithm, and interpolation methodologies, the planar displacement of the monitored targets can be calculated precisely, with sub-pixel precision being achieved. Subsequently, by adopting an enhanced Semi-Global Matching (SGM) algorithm, stereo matching of binocular images were accomplished. This process yields a disparity map, from which the depth map is computed. Then, precise measurement of the vertical displacement of the monitoring target can be obtained. This enables accurate measurement of three-dimensional displacements of the monitored targets. Finally, the developed methodology undergoes a comparative assessment against traditional measurement methods and existing algorithms to validate its measurement accuracy. The results prove that the deformation measurement technique based on binocular vision proposed in this paper captures structural deformation effectively. The accuracy surpasses that of existing measurement algorithms, and its efficiency outperforms traditional measurement methods.

DIC measurement method based on binocular stereo vision for image 3D displacement detection

The deformation detection of large machinery is usually achieved using three-dimensional displacement measurement. Binocular stereo vision measurement technology, as a commonly used digital image correlation method, has received widespread attention in the academic community. Binocular stereo vision achieves the goal of three-dimensional displacement measurement by simulating the working mode of the human eyes, but the measurement is easily affected by light refraction. Based on this, the study introduces particle swarm optimization algorithm for target displacement measurement on Canon imaging dataset, and introduces backpropagation neural network for mutation processing of particles in particle swarm algorithm to generate fusion algorithm. It combines the four coordinate systems of world, pixel, physics, and camera to establish connections. Taking into account environmental factors and lens errors, the camera parameters and deformation coefficients were revised by shooting a black and white checkerboard. Finally, the study first conducted error analysis on binocular stereo vision technology in three dimensions, and the relative error remained stable at 1 % within about 60 seconds. At the same time, three algorithms, including the spotted hyena algorithm, were introduced to conduct performance comparison experiments using particle swarm optimization and backpropagation network algorithms. The experiment shows that the three-dimensional error of the fusion algorithm gradually stabilizes within the range of [–0.5 %, 0.5 %] over time, while the two-dimensional error generally hovers around 0 value. Its performance is significantly superior to other algorithms, so the binocular stereo vision of this fusion algorithm can achieve good measurement results.

Spatial displacement tracking of vibrating structure using multiple feature points assisted binocular visual reconstruction

Current vision-based displacement measurement methods have limitations of requiring manual target and parameter adjustment, along with significant user involvement to achieve the desired results. This paper proposes an advanced and comprehensive vision system based on ZED cameras for full-field three-dimensional (3D) vibration displacement measurement of civil structures. Compared to existing binocular systems consisting of multiple monoculars, ZED binoculars offer greater flexibility on instrument setup and data acquisition. A novel keypoint detection and matching algorithm based on deep learning is used to achieve target-free measurement, which is capable of capturing feature points with high precision. The performance of the proposed vision-based approach is evaluated through experimental studies on a six-story scaled frame structure. Statistical analysis of the comparative results in sinusoidal cases demonstrates that the displacement responses obtained from the proposed vision-based approach show consistent results in compliance with displacement sensor measurements. System identification of the target structure is achieved through a seismic case and the error analysis is also concluded in detail.

Monocular vision based 3D vibration displacement measurement for civil engineering structures

Vibration displacement of civil structures are crucial information for structural health monitoring (SHM). However, the challenges and costs associated with traditional physical sensors make displacement measurement difficult. In recent years computer vision (CV) techniques have been employed for measuring vibration displacement in civil structures. There has been a growing interest in CV-based three-dimensional (3D) displacement measurement, as it provides comprehensive information for structural health assessment. Most existing methods use multi-view geometry, requiring multiple cameras for depth measurement. This paper proposes a new system for measuring the 3D vibration displacement utilising a single camera. Instead of using multi-view geometry, deep neural networks are utilised to learn the depth of scenes from monocular images. Compared with the multi-view methods, the proposed 3D measurement system with monocular vision is more cost-effective and much more convenient to set up and use in practice, avoiding the complicated calibration and object matching between multiple cameras. Experimental tests are conducted in the laboratory to investigate the feasibility of the proposed system. Physical displacement sensors are equipped with the testing structure to provide the ground truth data. The results demonstrate that the proposed monocular 3D displacement system is able to produce reasonable 3D full-field displacement measurement, which makes monocular image based CV system a promising approach to achieve 3D displacement measurement, with its obvious advantages in cost and convenience compared to the traditional sensor-based or multi view CV-based methods.

Underground Displacement Measurement Method Based on Double Mutual Inductance Voltage Contour Model of Optimized Dual Kriging

Landslide is a common geological hazard. Existing displacement monitoring sensors for landslides are classified into surface displacement and underground displacement. However, current underground displacement sensors cannot effectively monitor the vertical displacement caused by soil accumulation of landslides, which cannot meet the needs of landslide monitoring. Therefore, this study has designed a three-dimensional underground displacement measurement sensor based on the double mutual inductance voltage contour method. Since the double mutual inductance voltage measurement model has low measurement accuracy due to low interpolation accuracy and complex location calculation method when solving the spatial position, the improved Dual-Kriging interpolation method based on quantum pigeon inspired algorithm is adopted to improve the original interpolation accuracy. The experimental results show that this method avoids the complex problem of Kriging interpolation calculation and improves the calculation speed. At the same time, the overall accuracy is significantly improved, reducing its fitting error and actual value to a maximum of three ADC12 resolutions. The landslide experiment on the landslide simulation experiment platform shows that the sensor can effectively reflect the displacement of underground soil, and has important application value for slope displacement monitoring.

Real-time profile measurement method for a large-scale satellite antenna.

To improve the detection capability of satellite-based synthetic aperture radar, a large antenna array with a length scale of 100 meters is urgently needed. However, the structural deformation of the large antenna leads to phase errors, which will significantly reduce the antenna gain; hence, real-time and high-precision profile measurements of the antenna are essential for active compensation of the phase and thus improving the antenna gain. Nevertheless, the conditions of antenna in-orbit measurements are rather severe because of limited installation locations of measurement instruments, large areas, and long distance to be measured, and unstable measurement environments. To deal with the issues, we propose a three-dimensional displacement measurement method for the antenna plate based on laser distance measuring and digital image correlation (DIC). The proposed method uses the DIC method to retrieve the in-plane displacement information in combination with a laser range finder to provide depth information. A Scheimpflug camera is used to overcome the limitation of the depth of field of traditional cameras and enable clear imaging of the full field. Moreover, a vibration compensation scheme is proposed to eliminate the measurement error of the target displacement caused by the random vibration (within 0.01°) of the camera support rod. The results of the experiment in a laboratory setting show that the proposed method can effectively eliminate the measurement error caused by camera vibration (50mm) and reduce the displacement measurement error to within 1mm with a measurement range of 60m, which can meet the measurement requirements of next-generation large satellite antennas.

Three-Dimensional Displacement Measurement of Micro-Milling Tool Based on Fiber Array Encoding

The vibration of the micro-milling tool presents a significant chaotic vibration phenomenon, which has a great influence on the tool life and part machining precision, and is one of the basic problems restricting the improvement of machining efficiency and machining accuracy in micro-milling. To overcome the difficulty of the traditional vibration measurement method with the online measurement of micro-milling tool multi-dimensional vibration, a three-dimensional (3D) measurement method of the micro-milling tool is proposed based on multi-fiber array coding, which converts the tool space motion into a decoding process of the optical coding array employing the tool modulating the multi-fiber array encoding. A 6 × 6 optical fiber array was designed, and a 3D motion platform for micro-milling tools was built to verify the characteristics of the optical fiber measurement system. The measurement results show that the measuring accuracy of the system reached 1 µm, and the maximum linear error in x-, y-, and z-direction are 1.5%, 2.58%, and 2.43%, respectively; the tool space motion position measurement results show that the maximum measurement error of the measuring system was 3.4%. The designed system has unique coding characteristics for the tool position in the space of 100 µm3. It provides a new idea and realization means for the online vibration measurement of micro-milling tools.

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

Image Spherizing-Based Three-Dimensional Displacement Measurement Technique for Microscope with Single Camera

The detection of three-dimensional displacement, caused by micro-nano scale deformation, is of great significance for the industrial production of glass or quartz optical fiber products. Considering the detection cost and practicability, an image spherizing-based three-dimensional displacement measurement method for a monocular microscopic imaging system (a microscope with a normal industrial camera) is proposed. Differential theory tells us that an arbitrary large-scale deformation can be approximatively considered as the sum of slope-shaped deformations. In this paper, an ordinary slope model is presented by the idea of image spherizing and differential theory, which reveals the mathematical relation between in-plane displacement and out-of-plane displacement in the deformed images captured by an optical microscope. After obtaining the in-plane displacement field, the out-of-plane displacement field can be extracted from the in-plane displacement field by the ordinary slope model, which indicates that the out-of-plane displacement information carried by the in-plane displacement is fully utilized. Simulation and application experiments are performed to demonstrate the effectiveness and all the absolute errors of the measurement by optical microscope with a magnification of 50× are less than 0.2 μm. Unlike conventional methods, the new method does not need a phase unwrapping process during detection and has many attractive characteristics such as low-cost, simple arrangements and good applicability. Because the measurement precision of this technique can meet the detection requirement, it has been applied to the industrial detection of glass or quartz optical fiber products.

Bayesian inference of dense structural response using vision-based measurements

As-built structures typically behave differently than their analytical models because of uncertainties in material properties, boundary conditions, loading scenarios, or other modeling assumptions. Therefore, to perform structural health monitoring effectively, analytical models need to be calibrated or updated to match the measured responses from as-built structures. Bayesian model updating provides a rigorous framework to integrate data and parameter uncertainty and has been applied successfully in numerous application settings employing traditional sensors that are often sparsely distributed. Computer vision has shown tremendous success for the measurement of static and dynamic displacements of structures. Vision-based measurements have the potential to be transformative for model updating of large-scale civil infrastructure, as measurements can be obtained for dense array of points within the camera frame using a single device. However, model updating using vision-based measurements to date has been limited to the use of a few points selected manually, leading to the investigation of simple structures or their global behaviors (e.g. first few modal responses); developing a systematic and highly automated procedure for estimating structural displacement and its uncertainty densely, registering the measured data to the finite element model, and then updating the model based on all the available data remains a challenge. This study proposes Bayesian inference using dense vision-based measurements to estimate the localized response of the entire structure. Finite element models are incorporated into the three-dimensional displacement measurement and registration process (model-informed approach), and then the Markov chain Monte Carlo method is applied to enable the Bayesian inference for large and complex civil infrastructure. The proposed method is applied to a laboratory-scale three-dimensional steel truss to perform inference of the overall system response based on the a posteriori knowledge. The results demonstrate the adequacy of performing Bayesian inference of structural response using vision-based measurements.

Dynamic measurement of three-dimensional absolute displacements using dual-camera multiplexed digital holography.

Digital holography has been frequently used to measure the micro-deformation in mechanical tests due to its full-field measurement with high resolution and accuracy. To measure dynamic three-dimensional absolute displacements without a known reference displacement, a new technique based on the combination of off-axis multiplexed digital holography and stereo photogrammetry is proposed. Under the illumination of two different wavelength lasers along various directions, two off-axis multiplexed holograms recorded by the dual-camera system are used to extract four phase maps with different sensitivity vectors simultaneously. Meanwhile, the variation of sensitivity vectors and registration of phase maps are carried out by the object shape measured by the dual-camera system. By the four registered phases with four varying sensitivity vectors, three-dimensional absolute displacements can be determined. The feasibility of our method is well demonstrated by a quantitative experiment and finite element analysis, and the dynamic measurement of a resistor undergoing thermal expansion is presented.

Tri-wavelength simultaneous ESPI for 3D micro-deformation field measurement.

Electronic speckle pattern interferometry (ESPI), a well-established technique for micro-deformation measurement, can be used to determine both in-plane and out-of-plane displacement components. Although many works in ESPI have been reported for three-dimensional (3D) displacement measurement, few works have focused on the simultaneous measurement of 3D deformation fields. Here we present an ESPI system that consists of three sub-interferometers for simultaneous measurement of all three displacement components and in-plane strain fields. A 3CCD color camera, a specially designed shifting stage, and three lasers with optimal wavelengths are used in this system. The lasers and 3CCD camera provide independent interferograms with different color signals, while the shifting stage allows the sub-interferometers to achieve simultaneous phase shifting. The results of color separation and experimental measurement demonstrate the utility of the system.

Method for the biomechanical analysis of aqueous veins and perilimbal sclera by three-dimensional photoacoustic imaging and strain field calculation

A method motivated by the eye’s aqueous veins is described for the imaging and strain calculation within soft biological tissues. A challenge to the investigation of the biomechanics of the aqueous vein—perilimbal sclera tissue complex is resolution of tissue deformations as a function of intraocular pressure and the subsequent calculation of strain (a normalized measure of deformation). The method involves perfusion of the eye with a contrast agent during conduction of non-invasive, optical resolution photoacoustic microscopy. This imaging technique permits three-dimensional displacement measurements of tracked points on the inner walls of the veins which are used in a finite element model to determine the corresponding strains. The methods are validated against two standard strain measurement methods. Representative porcine globe perfusion experiments are presented that demonstrate the power of the method to determine complex strain fields in the veins dependent on intraocular pressure as well as vein anatomy. In these cases, veins are observed to move radially outward during increases in intraocular pressure and to possess significant spatial strain variation, possibly influenced by their branching patterns. To the authors’ knowledge, these are the only such quantitative, data driven, calculations of the aqueous vein strains available in the open literature.

Computer vision based target-free 3D vibration displacement measurement of structures

Vibration displacement response is widely used in the field of structural health monitoring (SHM) to monitor the health condition of civil engineering structures. Traditionally, the vibration displacement response is acquired by physical sensors, such as linear variable differential transformers (LVDT) and laser displacement sensors (LDS), or more commonly by accelerometers to measure accelerations instead of directly measuring the displacement due to the fact that direct displacement measurement requires a fixed platform to install the sensors. Recently, to overcome the difficulties in direct displacement measurement, computer vision based methods have become a research hotspot. This paper proposes an advanced binocular vision system for target-free full-field three-dimensional (3D) vibration displacement measurement of civil engineering structures. The state-of-the-art key point detection and matching algorithm based on deep learning is employed to achieve target-free measurement, which greatly improves the quantity and quality of matching natural key points with low contrast. The performance of the proposed vision based approach for 3D vibration displacement measurement is evaluated through an experimental test on a steel cantilever beam in the laboratory. The obtained vibration displacement responses obtained from the proposed approach are compared with those measured by LVDTs and LDSs. The results demonstrate that the displacement responses obtained from the proposed vision based approach are accurate compared with the traditional sensors, while the proposed approach is more cost effective and much easier to achieve accurate 3D vibration displacement measurement.

Optical three-dimensional displacement measurement based on moiré methodology and its application to space structures

Optical methods providing full-field deformation information is of interest to mechanical engineers. In this study, an accurate optical three-dimensional displacement measurement method using a single camera attached to a precision 3-axis moving stage was developed. This paper presents the fundamental measurement principle and experiments for validating the measurement accuracy. The in-plane and out-of-plane displacements are simultaneously determined by calculating the phase difference and the phase gradient difference of the moiré fringe before and after deformation. The developed method was applied to displacement measurement in a loading test of a carbon fiber reinforced plastics (CFRP) cylindrical payload 2.5 m in diameter and 2.0 m long used in a solid rocket. The three-dimensional displacement results obtained from the developed method showed a good agreement with those measured with the extensometers conventionally employed in the displacement measurement. The developed method is useful for inspection in the space field and to secure the health monitoring of large-scale structures.

High-accurate camera calibration in three-dimensional visual displacement measurements based on ordinary planar pattern and analytical distortion model

Camera calibration is crucial for geometric vision-based three-dimensional (3D) deformation measurement tasks. Among existing calibration techniques, the one based on planar targets has attracted much attention in the community due to its flexibility and reliability. Our study proposes a calibration technique to obtain high-accuracy internal and external parameters based on low-cost ordinary planar patterns. The proposed method determines the optimal internal parameters for each camera by refining 3D coordinates of planar control points, where an analytic model of optics distortion is presented to enable lens distortion to be corrected directly in subsequent external calibration and underlying 3D reconstruction. External parameters are estimated by minimizing a bundle adjustment framework, which is carefully designed based on the proposed distortion correction model and depth parameterization. In contrast to the existing techniques, the proposed method is capable of obtaining a high-accuracy calibration with ordinary targets rather than the well-designed and fabricated ones. We experimented the proposed method with a calibration performance analysis and a displacement measurement; both results demonstrated the accuracy and robustness.

A multi-camera based photogrammetric method for three-dimensional full-field displacement measurements of geosynthetics during tensile test

Conventional methods for measuring the deformational response of geosynthetics, such as the linear variable differential transformers (LVDTs), strain gauges, and extensometers have several limitations in fully determining the complete strain distribution in geosynthetics. This paper presents a multi-camera based photogrammetric method to track the 3D full-field displacements of geosynthetics during tensile tests. The proposed method extends the conventional one-camera based photogrammetry for static object measurements to multi-camera-based photogrammetry for object measurements with continuous movements or deformations. It is non-contact, cost-effective, highly accurate, and capable of measuring the 3D full-field displacements of the geosynthetics. A tensile test on a geogrid specimen was performed to verify the effectiveness and accuracy of the proposed photogrammetric method. The results from the tensile tests using both the proposed method and conventional methods, such as machine-controlled displacement measurements and extensometer, were presented and compared. It was observed that the average absolute difference between the proposed photogrammetric method and the machine-controlled movements of the bottom clamp was 0.25%, and the average absolute error was 0.038 mm. The average difference in measurements made by the proposed method and extensometer was 0.07%. It was further found that the proposed method can provide more comprehensive input, such as the complete strain and modulus distributions in the geosynthetics, for a probability-based geosynthetics design.

Drone-Based StereoDIC: System Development, Experimental Validation and Infrastructure Application

Digital Image Correlation (DIC) is widely used for remote and non-destructive structural health evaluation of infrastructure. Current DIC applications are limited to relatively small areas of structures and require the use of stationary stereo vision camera systems that are not easy to transfer and deploy in remote areas. The enclosed work describes the development and validation of an Unmanned Aircraft System (UAS, commonly known as drone) with an onboard stereo-vision system capable of acquiring, storing and transmitting images for analysis to obtain full-field, three-dimensional displacement and strain measurements. The UAS equipped with a StereoDIC system has been developed and tested in the lab. The drone system, named DroneDIC, autonomously hovers in front of a prestressed railroad tie under pressure and DIC data are collected. A stationary DIC system is used in parallel to collect data for the railroad tie. We compare the data to validate the readings from the DroneDIC system. We present the analysis of the results obtained by both systems. Our study shows that the results we obtain from the DroneDIC system are similar to the ones gathered from the stationary DIC system. This work serves as a proof of concept for the successful integration of DIC and drone technologies into the DroneDIC system. DroneDIC combines the high accuracy inspection capabilities of traditional stationary DIC systems with the mobility offered by drone platforms. This is a major step towards autonomous DIC inspection in portions of a structure where access is difficult via conventional methods.