Inverse Compositional Gauss-Newton Research Articles

Computer vision-based real-time deflection monitoring of complex and sizeable steel structures

Deflection, as an intuitive index, plays a pivotal role in assessing the load-bearing capability and structural integrity of complex and sizable steel structures. Notwithstanding, the conventional contact measurement is limited to static deflection and necessitates work stands and manual readings. Therefore, it hinders the attainment of dynamic deflection and fails to cater to the engineering demands of real-time and prolonged monitoring. By leveraging the advancements in computer vision technology, we propose an innovative system for real-time deflection monitoring of complex and sizable steel structures, particularly suitable for monotonic deformation induced by prolonged loading. Specifically, the off-axis-based displacement measurement method was adopted to surmount the constraint of requiring the optical axis of the camera to be perpendicular to the target. Additionally, the inverse compositional Gauss-Newton (IC-GN) algorithm and parallel computation based on seed point diffusion were exploited to boost the matching and computing efficiency for attaining real-time monitoring of multi-points. To validate the efficacy of the proposed system, we conducted static load monitoring tests on a Bailey beam, with a height of 29.5 m and length of 16.5 m per span, as part of the Alibaba Jiangsu headquarters project in Nanjing. The collected test data were compared with the results from a laser displacement sensor and the ABAQUS model. The outcomes substantiate that the system is capable of non-contact, expeditious, and straightforward installation, besides achieving high accuracy and real-time deflection monitoring. This system serves as a sophisticated solution for health monitoring in constructing complex and sizeable steel structures and further contributes to realizing construction intelligence.

Research on image deformation monitoring algorithm based on binocular vision

Monitoring the deformation of critical structures such as tunnel、buildings and bridges is critical to ensure the safety of infrastructures. Non-contact deformation monitoring technology based on digital imagery has emerged as a prevalent and effective monitoring method, which enables cost-effective and efficient tracking of structural deformations. This paper aims to develop an automated deformation monitoring methodology using the binocular vision. First, standard preprocessing of digital images is performed through the integration of filtering algorithms and image enhancement techniques. Furthermore, by combining the Scale Invariant Feature Transform (SIFT) feature matching algorithm, the inverse compositional gauss–newton iterative algorithm, and interpolation methodologies, the planar displacement of the monitored targets can be calculated precisely, with sub-pixel precision being achieved. Subsequently, by adopting an enhanced Semi-Global Matching (SGM) algorithm, stereo matching of binocular images were accomplished. This process yields a disparity map, from which the depth map is computed. Then, precise measurement of the vertical displacement of the monitoring target can be obtained. This enables accurate measurement of three-dimensional displacements of the monitored targets. Finally, the developed methodology undergoes a comparative assessment against traditional measurement methods and existing algorithms to validate its measurement accuracy. The results prove that the deformation measurement technique based on binocular vision proposed in this paper captures structural deformation effectively. The accuracy surpasses that of existing measurement algorithms, and its efficiency outperforms traditional measurement methods.

Reducing the systematic error of DIC using gradient filtering

The inverse compositional Gauss-Newton (IC-GN) DIC algorithm is now the most popular DIC algorithm. The error analysis of the algorithm is necessary. However, the effect of the gradient operator on the error cannot be systematically analysed due to the different dimensions of the gradient operator. In this paper, the 1-D and 2-D gradient operators are incorporated into the same framework by decomposing the gradient operator into two parts: gradient acquisition and gradient filtering. Based on the above analysis, a DIC method based on gradient filtering is constructed and the simulation analysis results show that the systematic error is reduced to less than 10% of the original ICGN-DIC algorithm, enhancing the robustness to noise variations. Finally, validation is performed using an open-source dataset. It is demonstrated that the proposed method can reduce the system error to less than 15%.

Optical coherence elastography based on inverse compositional Gauss-Newton digital volume correlation with second-order shape function.

A digital volume correlation (DVC)-based optical coherence elastography (OCE) method with inverse compositional Gauss-Newton (IC-GN) algorithm and second-order shape function is presented in this study. The systematic measurement errors of displacement and strain from our OCE method were less than 0.2 voxel and 4 × 10-4, respectively. Second-order shape function could better match complex deformation and decrease speckle rigidity-induced error. Compared to conventional methods, our OCE method could track a larger strain range up to 0.095 and reduce relative error by 30-50%. This OCE method has the potential to become an effective tool in characterising mechanical properties of biological tissue.

Convolutional neural network based displacement gradients estimation for a full-parameter initial value guess of digital image correlation

The selection of initial value in digital image correlation (DIC) has significant influence on the search efficiency of image subpixel displacement and the algorithmic convergence speed. An accurate and reasonable initial value can reduce the number of iterations of subsequent IC-GN optimization, accelerate the convergence of the results, and avoid the divergence of the algorithm in the iterative process. This paper proposes a full-parameter initial value estimation method based on a regression convolution neural network with multithreaded calculation. The proposed method sequentially uses the integer-pixel estimation based on neighborhood search, the subpixel estimation based on surface fitting and the first-order displacement gradients estimation based on a regressive convolutional neural network to achieve the initial value estimation of inverse compositional Gauss-Newton (IC-GN) iteration. Experimental results show that the iteration times of the proposed method are reduced by about 30% compared with the integer-pixel initial value estimation method. In the process of IC-GN iteration, the computational efficiency of CPU multithreaded calculation is nearly twice higher as that of the single-thread method. It can not only improve the accuracy of the initial value estimation but also has high adaptability, which can adapt to selecting different subset sizes or different speckle patterns. This study provides a reference for the effect of iterative initial value optimization on efficiency and accuracy in DIC.

Digital image correlation method based on quasi-conformal mapping for large deformation measurement

Digital image correlation (DIC) has the advantages of non-contact full-field measurement and high measurement accuracy in optical mechanics, but still faces challenges in large deformation measurement. This work developed an initial estimation algorithm, using geometric distortion as an indicator. And the more accurate deformation is adjusted by the modified inverse compositional Gauss-Newton (IC-GN) algorithm that introduces the inverse mapping of the initial estimation. The simulation experiment proves that the proposed method has good measurement ability for large deformation, and the accuracy is about 0.02 pixels. The uniaxial tensile test of vulcanized rubber and the three-point bending test of the polycarbonate sheet are used to verify the effectiveness of the method in this paper.

Path independent stereo digital image correlation with high speed and analysis resolution

Path independent strategy is introduced into stereo digital image correlation, in which high accuracy is attained through the inverse compositional Gauss-Newton (IC-GN) algorithm with the second and the first order shape functions for stereo matching and temporal matching, respectively. To gain strong adaptibility to large and complex deformation or considerable parallax betweent the two views, two robust methods are developped to estimate the initial guess for the iterative IC-GN algorithm. One is aided by scale invariant feature transform (SIFT) features and the other is aided by epipolar constraint. Powered by parallel computing on GPU, the proposed stereo DIC demonstrates high computation efficiency. Real-time stereo DIC is realized and used to measure the large deformation at common video recording rate (∼24 fps) with high ananlysis resolution (∼6300 points of interest in region of interest).

Real-time 3D Digital Image Correlation for Large Deformation and Rotation Measurements Based on a Deformation Transfer Scheme

With the recent increase in digital image correlation (DIC) applications, it is important to improve the computational efficiency of DIC and realize the closed loop control of systems. A real-time three-dimensional (3D) DIC method for large deformation and rotation measurements is proposed in this paper. To solve the problem of initial guess estimation under large deformation and rotation, the convergence radius of the inverse compositional Gauss-Newton (IC-GN) algorithm considering large rotation and deformation is analyzed. Then, a temporal-spatial deformation transfer scheme is proposed for efficient temporal and stereo matching in 3D-DIC. The highly efficient IC-GN algorithm with an improved interpolation look-up table is proposed to further improve the computational efficiency. The multipoint and full-field real-time 3D deformation measurements are performed by parallel CPU computation. The convergence radius of IC-GN are estimated to be below 3 pixels for rotation and below 5 pixels for tensile deformation. The accuracy of the initial guess estimation by deformation transfer is improved and can meet the requirements of large deformation and rotation measurements. Benefiting from the temporal-spatial deformation transfer scheme, the efficient IC-GN algorithm, the efficient temporal and stereo matching strategy and the parallel computation, real-time multipoint deformation measurements at 60 Hz with 50 points and full-field deformation measurements at 15 Hz with 5000 points are realized. Real-time multipoint and full-field 3D deformation measurements for large deformations and rotations are realized. High-accuracy, real-time and fully automatic 3D deformation measurement sensors are expected.

The Effect of Low-pass Pre-filtering on Subvoxel Registration Algorithms in Digital Volume Correlation: A revisited study

Abstract In digital volume correlation (DVC), random image noise in volumetric images leads to increased systematic error and random error in the displacements measured by subvoxel registration algorithms. Previous studies in DIC have shown that adopting low-pass pre-filtering to the images prior to the correlation analysis can effectively mitigate the systematic error associated with the classical forward additive Newton-Raphson (FA-NR) algorithm. However, the effect of low-pass pre-filtering on the state-of-the-art inverse compositional Gauss-Newton (ICGN) algorithm has not been investigated so far. In this work, we focus on the effect of low-pass pre-filtering on two mainstream subvoxel registration algorithms (i.e., 3D FA-NR algorithm and 3D IC-GN algorithm) used in DVC. Basic principles and theoretical error analyses of the two algorithms are described first. Then, based on numerical experiments with precisely controlled subvoxel displacements and noise levels, the influences of image noise on the displacements measured by two subvoxel algorithms are examined. Further, the effects of low-pass pre-filtering on these two subvoxel algorithms are examined for simulated image sets with different noise levels and deformation modes. The results show that the low-pass pre-filtering can effectively suppress the systematic errors for the 3D FA-NR algorithm, which is consistent with the previously drawn conclusion in DIC. On the contrary, different form the 3D FA-NR algorithm, the 3D IC-GN algorithm itself can reduce the influence of image noise, and the effect of low-pass pre-filtering on it is not so obvious as on 3D FA-NR algorithm.

3D SIFT aided path independent digital volume correlation and its GPU acceleration

A novel path independent digital volume correlation (DVC) method is proposed, in which the internal deformation field is determined with the aid of the features extracted from the volume images. The deformation vector at each point of interest is first estimated by tracking the motion of the nearby keypoints obtained through 3D scale-invariant feature transform (SIFT), and then fed as the initial guess into the 3D inverse compositional Gauss-Newton algorithm to achieve high accuracy result. Benefiting from the robustness of SIFT to various deformation and distortion of image, the proposed DVC method demonstrates outstanding performance to automatically process the volume images containing large and complex deformation, which keep challenging to current DVC methods for decades. The proposed DVC method is further powered by the parallel computing on GPU. An ultrafast computation speed (about 10,000 POI/s) can be reached on a personal computer when dealing with the volume images of normal sizes. The 3D SIFT aided DVC method, which achieves unprecedented balance between accuracy, adaptability, and efficiency, shows great potential in the quantitative analysis of internal deformation.

Testing System for the Mechanical Properties of Small-Scale Specimens Based on 3D Microscopic Digital Image Correlation.

The testing of the mechanical properties of materials on a small scale is difficult because of the small specimen size and the difficulty of measuring the full-field strain. To tackle this problem, a testing system for investigating the mechanical properties of small-scale specimens based on the three-dimensional (3D) microscopic digital image correlation (DIC) combined with a micro tensile machine is proposed. Firstly, the testing system is described in detail, including the design of the micro tensile machine and the 3D microscopic DIC method. Then, the effects of different shape functions on the matching accuracy obtained by the inverse compositional Gauss–Newton (IC-GN) algorithm are investigated and the numerical experiment results verify that the error due to under matched shape functions is far larger than that of overmatched shape functions. The reprojection error is shown to be smaller than before when employing the modified iteratively weighted radial alignment constraint method. Both displacement and uniaxial measurements were performed to demonstrate the 3D microscopic DIC method and the testing system built. The experimental results confirm that the testing system built can accurately measure the full-field strain and mechanical properties of small-scale specimens.

Accurate and simple digital volume correlation using pre-interpolation

Existing incremental digital volume correlation methods can reduce the number of errors introduced by interpolation calculations in the inverse-compositional Gauss–Newton (IC-GN) algorithm iteration. However, the accuracy of these existing methods is insufficient for some conditions as the curve-fitting method has high computational efficiency but lacks accuracy. A simple pre-interpolation method is proposed to improve the accuracy and computational efficiency of digital volume correlation. First, the pretreatment of a deformed volume image is calculated by the cubic spline interpolation method with the most often chosen interpolation step of 1/2 sub-voxel. Next, the pre-interpolation is calculated only once and the block calculation techniques solve the memory problem. Then, the reference sub-volume in the updated reference volume image is translated into the nearest half-integer voxel position instead of the integer voxel position or other sub-voxel positions. The pre-interpolation method is applied to both the IC-GN algorithm and the curve-fitting method. Experimental results show that the maximum mean bias error and the maximum standard deviation of the improved IC-GN algorithm are reduced by 34% and 75%, respectively. The improved curve-fitting has better accuracy and computational efficiency than the IC-GN algorithm under small strain and the curve-fitting method can achieve about 3.2 times speedup than the IC-GN algorithm.

Global DIC approach guided by a cross-correlation based initial guess for HR-EBSD and on-axis HR-TKD

Digital image correlation (DIC) techniques usually performed on deforming speckle patterns are applied here on electron diffraction patterns (EDP) in order to map disorientations with high angular resolution, as well as geometrically necessary dislocation densities and elastic strains. The proposed approach relies on a DIC analysis which is conducted independently from the microscope calibration. It registers EDP as a whole through a single and large region of interest whose relative deformation with respect to a reference EDP is described by a first-order homography. Subpixel registration is performed iteratively in the spatial domain using an inverse-compositional Gauss-Newton (IC-GN) algorithm which integrates the correction of the optical distortions. Its robustness against large orientation changes as well as its computational efficiency are improved by mean of an automated and path-independent initial guess which fairly captures the effects of large rotations on the EDP. Using successive Fourier-Mellin and Fourier transforms based cross-correlation techniques, the initial guess measures the in-plane rotation and translation between the reference and the target EDP, respectively. The performances of the technique are illustrated in markedly plastically deformed steels using two electron diffraction techniques in the scanning electron microscope. A 15% deformed interstitial free steel is investigated by means of electron backscattered diffraction (EBSD) while a quenched and tempered oxide dispersed strengthened steel subjected to martensitic transformation is characterized using a recently developed on-axis Transmission Kikuchi Diffraction (TKD) configuration.

Recursive-iterative digital image correlation based on salient features

Measuring surface deformation of objects with natural patterns using digital image correlation (DIC) is difficult due to the challenges of the pattern quality and discriminative pattern matching. Existing studies in DIC predominantly focus on the artificial speckle patterns while seldom paying attention to the inevitable natural texture patterns. We propose a recursive-iterative method based on salient features to measure the deformation of objects with natural patterns. The method is proposed to select salient features according to the local intensity gradient and then to compute their displacements by incorporating the inverse compositional Gauss–Newton (IC-GN) algorithm into the classic image pyramidal computation. Compared with the existing IC-GN-based DIC technology, the use of discriminative subsets allows avoidance of displacement computation at pixels with poor spatial gradient distribution. Furthermore, the recursive computation based on the image pyramid can estimate the displacements of the features without the need for initial value estimation. This method remains effective even for large displacement measurements. The results of simulation and experiment prove the method’s feasibility, demonstrating that the method is effective in deformation measurement based on natural texture patterns.

SIFT-aided path-independent digital image correlation accelerated by parallel computing

Current iterative digital image correlation (DIC) algorithms can efficiently converge at the deformation vector with high accuracy when they are fed with reliable initial guess. Thus, the adaptability of DIC method is dominated to a large extent by the estimation of initial guess. In recent years, image feature-based technique, especially the scale-invariant feature transform (SIFT), was introduced to DIC for the estimation of initial guess in the case of large and complex deformation, due to its robustness in handling the images with translation, rotation, scaling, and localized distortion. However, feature extraction and matching in SIFT are very time consuming, which limits the applications of the SIFT-aided DIC. In this study, we developed a SIFT-aided path-independent DIC method and accelerated it by introducing the parallel computing on graphics processing unit (GPU) or multi-core CPU. In our method, SIFT features are used to estimate the initial guess for the inverse compositional Gauss-Newton (IC-GN) algorithm at each point of interest (POI). The experimental study shows that the developed method can deal with large and inhomogeneous deformation with high accuracy. Parallel computing (especially on GPU) accelerates significantly the proposed DIC method. The achieved computation speed satisfies the need for real-time processing with high resolution for the images of normal sizes.

Deformation measurement of high-speed rotating drone blades based on digital image correlation combined with ring projection transform and orientation codes

Digital image correlation (DIC) is a versatile non-contact optical measurement method, but it still has many shortcomings in theory and application. For example, the traditional DIC method has difficulty obtaining the desired result when the surface of the test object is rotated substantially or the deformation involves large rotation. Some scholars argue that a rotation angle greater than 7° will not work with the DIC algorithm, and this phenomenon is called decorrelation. To solve this problem, this study proposes the rotation-invariant DIC (RI-DIC) method that combines rotation-invariant template matching with traditional DIC. A pre-matching method based on ring projection transform (RPT) and orientation codes (OC) is proposed to provide the initial values for the DIC iteration process. The algorithm consists of three stages. In the first stage, the RPT process is used to convert the 2D template in a circular region into a 1D grey-level signal as a function of radius, and the template matching based on RPT is used to obtain a series of candidate points. The advantages of RPT process are the characterisation of its rotation invariance and the reduction of computational complexity. In the second stage, a fine matching process based on OC is performed on a limited number of candidate points to obtain the integer pixel matching position and the initial value. Finally, the inverse compositional Gauss–Newton (IC-GN) iteration method is used to further calculate the displacement and strain information accurately on the basis of the initial values. The novelty of this paper is to fuse the ring projection transform and orientation codes in machine vision into the traditional DIC algorithm, finally propose the RI-DIC algorithm. By the proposed method, we can measure the strain generated by the centrifugal force of a drone blade rotating at a high speed. Experimental results indicate that the method can accurately measure the surface deformation of a test object at any rotation angle.

Enhanced feature-based path-independent initial value estimation for robust point-wise digital image correlation

With the increased complexity in various applications of Digital Image Correlation (DIC), the requirement for its performances as a key technique of optical metrologies becomes more and more crucial, especially in terms of accuracy, efficiency, and robustness. However, the existing DIC methods mainly rely on a calculation path to conduct the DIC analysis, which limits their further improvement in the measurement accuracy, computational efficiency, and robustness to complex deformation. To promote the implementation of a real-time and wide-applied DIC system, this study presents an automatic, fast and robust point-wise DIC (PW-DIC) algorithm. This work introduces an enhanced feature-based path-independent initial value estimation (PI-IVE) method for an accurate subpixel iterative algorithm, which is achieved by a variant of PatchMatch algorithm in the feature image generated from the speckle pattern image. The feature image is derived from DAISY descriptor, which extracts the descriptor value at every pixel effectively rather than some detected keypoints. This method calculates reliable full-field initial values for all points of interest (POIs) simultaneously without an initial guess transfer path, which is path-independent. It cooperates with the state-of-the-art inverse compositional Gauss-Newton (IC-GN) algorithm to perform an accurate DIC analysis for each POI independently. Moreover, the proposed PW-DIC method can solve the problem of low calculation efficiency in path-dependent DIC methods. Due to the high accuracy and flexibility of the proposed PW-DIC method, it outperforms the standard reliability-guided DIC (RG-DIC) methods for large, discontinuous or rotational deformations. Extensive experiments and analysis are conducted to demonstrate the accuracy, effectiveness, and robustness of the proposed PW-DIC method.

New levels of high angular resolution EBSD performance via inverse compositional Gauss–Newton based digital image correlation

Conventional high angular resolution electron backscatter diffraction (HREBSD) uses cross-correlation to track features between diffraction patterns, which are then related to the relative elastic strain and misorientation between the diffracting volumes of material. This paper adapts inverse compositional Gauss Newton (ICGN) digital image correlation (DIC) to be compatible with HREBSD. ICGN-based works by efficiently tracking not just the shift in features, but also the change in their shape. Modeling a shape change as well as a shift results in greater accuracy. This method, ICGN-based HREBSD, is applied to a simulated data set, and its performance is compared to conventional cross-correlation HREBSD, and cross-correlation HREBSD with remapping. ICGN-based HREBSD is shown to have about half the strain error of the best cross-correlation method with a comparable computation time.

Progress in digital image correlation method

Photomechanics Laboratory in the University of Science and Technology of China has concentrated on principles and applications of digital image correlation (DIC) recent years. Photomechanics Laboratory not only has improved the accuracy and efficiency of DIC but also has achieved research findings on speckle pattern standardization, compensation method for 2D DIC, biomedical engineering, and serrated yielding phenomenon. This paper provides a review: (1) a theoretical model and an estimation algorithm were presented for shape function error in local deformations, and a subset size chosen criterion was presented to ensure the measurement performance; inverse compositional Gauss-Newton method using second order shape function was presented, which is two-times faster than the traditional method. (2) The inherent nature of interpolation bias was revealed, an experimental approach was developed to measure the interpolation bias, a speckle pattern assessment that considering the effects of both the systematic error and random error was presented, and a theoretical model for the speckle pattern optimization was built. (3) Compensation methods for 2D DIC based on pre-calibration were presented. (4) A system for orthognathic surgery navigation was built; full-field pulse vibration measurement system was developed. (5) The emergence and evolution of PLC band was observed, and the phenomenon of out-of-band shrink is found. The strain localization in the width-wise direction was observed using two 2D DIC systems. (6) The vibration and strain were dynamically measurement using the speckle correlation method.

A flexible heterogeneous real-time digital image correlation system

An accurate and flexible real-time digital image correlation (RT-DIC) system utilizing a pipelined CPU and GPU parallel computing framework is proposed. First, the respective advantages of CPU and GPU in performing the fast Fourier transform-based cross-correlation (FFT-CC) algorithm and the inverse-compositional Gauss Newton (IC-GN) algorithm of the employed path-independent DIC (PI-DIC) method are elucidated. Second, based on the different properties and performances of CPU and GPU, a pipelined system framework unifying five Variants of combinations of CPU and GPU is proposed, which can be flexibly applied to various practical applications with different requirements of measurement scales and speeds. Last, both the accuracy and speed of the entire pipelined framework are verified by a PC implementation of the RT-DIC system integrating Variants 2–5. Variants 2 and 5 are also implemented on an iPhone 5S for the feasibility investigation of realizing a portable RT-DIC system on mobile devices using the same framework.