Inverse Compositional Algorithm Research Articles

Motion transformation solutions based on Euler angle perturbation model

This paper presents two novel solutions to compute motion transformation by effectively addressing the nonlinear constraints formed by observational data. These solutions have broad applicability in algorithms involving rotation calculation, such as autonomous positioning systems. Instead of the commonly used Lie algebra-based solutions, the two novel solutions are based on the Euler angle perturbation model, which circumvent the problem arising from the singularity of Euler angles. The first solution is a forward compositional algorithm, which emphasizes accelerated iterative processes. The second solution is an inverse compositional algorithm that aims to minimize computational complexity. Both algorithms offer accelerated iteration by employing efficient Jacobian computation and parameter updates. Furthermore, we have developed an odometry algorithm and a point cloud alignment program to assess the performance of these different solutions. The experimental results demonstrate that the two novel solutions achieve an equivalent global minimum while also reducing computational load.

Digital image correlation calculation method for RGB-D camera multi-view matching using variable template

Multi-view matching is an indispensable method for digital image correlation 3D reconstruction. However, as the camera viewing angle changes, traditional DIC methods can lead to the accumulation of matching errors and an increase in mismatches, thereby affecting the 3D reconstruction quality. This paper proposes a digital image correlation method for multi-view RGB-D images based on variable template matching. Initial positioning is achieved by dynamically adjusting the size and sampling direction of the variable template. Sub-pixel registration is then performed using the inverse compositional algorithm, followed by multi-view point cloud fusion using the Iterative Closest Point (ICP) registration method. Experimental results show that the iteration speed of point cloud registration is increased by 30%, and the cumulative error is reduced by 62.5%. These improvements demonstrate that the proposed method is suitable for multi-viewpoint matching scenarios, achieving excellent results in multi-view feature point matching and multi-view point cloud fusion, significantly enhancing 3D reconstruction accuracy and efficiency.

Ego-Motion Estimation With Stereo Cameras Using Efficient 3D–2D Edge Correspondences

This article presents a new edge-based ego-motion estimation method for stereo cameras using 3D–2D edge alignment. To filter out prominent edgelets from Canny edge extractor, a region growing algorithm is developed to remove cluttered edges out of raw edgelets. Edge matching between stereo image pairs is guided by sparse feature correspondence using optical flow. The disparity is then refined using an inverse compositional algorithm, which is far more computationally efficient than searching along the epipolar line. The relative camera motion is solved by 3D–2D edge alignment based on an iterative closest point (ICP) algorithm, minimizing the point-to-tangent distance. A gradient-orientation-based similarity measurement is proposed, and the optimal correspondence is searched between the 3-D edge point projected onto the current image and the extracted edge pixels in the current frame with 1-nearest neighbor. In addition, the inverse depth of edge points in the reference frame is optimized. Experiments show that the proposed method can provide centimeter-level relative pose error (RPE), which has a better performance than the point-based approach efficient perspective-n-point (EPnP) and the edge-based approach direct edge-alignment (DE-A).

Transverse and Longitudinal Vibration Measurement of Vertical Rope of Hoisting System Based on Machine Vision

Lateral and longitudinal vibration of vertical rope, which reflect the force characteristics of itself, are important parameters for fault diagnosis and condition monitoring of hoisting system, and the measurement result is the basis for failure analysis and vibration control. In this study, a novel vision-based measurement framework, which contains image preprocessing (IP) part and target tracking part based on accelerated modified inverse compositional (AMIC) algorithm, called IP-AMIC, is proposed to obtain the rope dynamic displacement. First, IP carried out to eliminate the measurement mutual influence of the transverse and longitudinal vibration of vertical rope. Additionally, a variable step-size strategy is used to increase the speed of the Lucas–Kanade algorithm, tracking speed reaches 500 fps (frame per second) with subpixel accuracy. The accuracy and efficiency of the proposed algorithm are demonstrated in the laboratory experiment. Finally, a field experiment of vertical rope vibration measurement of hoisting system is carried out, analyzed the dynamic displacement of ropes during unloading process. Experimental results show that the proposed algorithm can automatically adjust posture of the rope and obtain the lateral and longitudinal vibration of the vertical rope efficiently.

PDC-Net: Robust point cloud registration using deep cyclic neural network combined with PCA.

It is important to improve the registration precision and speed in the process of registration. In order to solve this problem, we proposed a robust point cloud registration method based on deep learning, called PDC-Net, using a principal component analysis based adjustment network that quickly adjusts the initial position between two slices of the point cloud, then using an iterative neural network based on the inverse compositional algorithm to complete the final registration transformation. We compare it on the ModelNet40 dataset with iterative closest point, which is the traditional point cloud registration method, and the learning-based methods including PointNet-LK and deep closest point. The experimental results show that the registration error is not worse with the increase of the initial phase between point clouds, avoiding the algorithm falling into the local optimal solution and enhancing the robustness of registration.

Error analysis of surface-distribution and non-deformation of fluorescent beads for the IC-GN2 DVC algorithm

Digital volume correlation (DVC) method is extensively used for the internal displacement measurement. In biomechanical experiments, fluorescent beads are used, instead of traditional speckles (or microstructure of materials), as the information feature of the volume image for DVC algorithm. The traditional experimental method and DVC algorithm, together, provide an effective measurement method for the cell deformation. Nevertheless, the aforesaid method has three limitations, viz., (1) low accuracy for complex displacement and poor efficiency for traditional second-order forward additive Gauss-Newton (FA-GN2) algorithm, (2) shape function mismatch induced by the non-deformation of fluorescent beads, and (3) the extreme phototoxicity caused by the long scanning time. To tackle these issues, we introduce the inverse compositional Gauss-Newton DVC algorithm with the second-order shape function (IC-GN2 DVC algorithm) to achieve higher accuracy and establish the four speckle volume image models to analyze the influence of the distribution and non-deformation of fluorescent beads on the results of calculation of DVC method. Henceforth, we propose an experimental method, considering that fluorescent beads are only distributed on the surface of the gel substrate (cells are also placed on the surface of gel substrate) enabling short scanning time (termed speckle-surface-distributed experimental method). The results from the analysis by numerical simulations show that the proposed IC-GN2 DVC algorithm has a very high accuracy (local error < 0.1 voxel). The error analysis of the four models shows that the non-deformation of fluorescent beads will induce significant error into the calculation results, whereas the surface-distribution of fluorescent beads induces little error, which prove the reliability of the speckle-surface-distributed experimental method. The noise robustness of the IC-GN2 DVC algorithm with the speckle-surface-distributed experimental method is also studied. Finally, the migration and deformation of the living cell is analyzed using the IC-GN2 DVC algorithm and the speckle-surface-distributed experimental method.

3D SIFT aided path independent digital volume correlation and its GPU acceleration

A novel path independent digital volume correlation (DVC) method is proposed, in which the internal deformation field is determined with the aid of the features extracted from the volume images. The deformation vector at each point of interest is first estimated by tracking the motion of the nearby keypoints obtained through 3D scale-invariant feature transform (SIFT), and then fed as the initial guess into the 3D inverse compositional Gauss-Newton algorithm to achieve high accuracy result. Benefiting from the robustness of SIFT to various deformation and distortion of image, the proposed DVC method demonstrates outstanding performance to automatically process the volume images containing large and complex deformation, which keep challenging to current DVC methods for decades. The proposed DVC method is further powered by the parallel computing on GPU. An ultrafast computation speed (about 10,000 POI/s) can be reached on a personal computer when dealing with the volume images of normal sizes. The 3D SIFT aided DVC method, which achieves unprecedented balance between accuracy, adaptability, and efficiency, shows great potential in the quantitative analysis of internal deformation.

Three-dimensional static optical coherence elastography based on inverse compositional Gauss-Newton digital volume correlation.

The three-dimensional (3D) mechanical properties characterization of tissue is essential for physiological and pathological studies, as biological tissue is mostly heterogeneous and anisotropic. A digital volume correlation (DVC)-based 3D optical coherence elastography (OCE) method is developed to measure the 3D displacement and strain tensors. The DVC algorithm includes a zero-mean normalized cross-correlation criterion-based coarse search regime, an inverse compositional Gauss-Newton fine search algorithm and a local ternary quadratic polynomial fitting strain calculation method. A 3D optical coherence tomography (OCT) scanning protocol is proposed through theoretical analysis and experimental verification. Measurement errors of the DVC-based 3D OCE method are evaluated to be less than 2.0 μm for displacements and 0.30% for strains by rigid body motion experiments. The 3D displacements and strains of a phantom and a specimen of chicken breast tissue under compression are measured. Results of the phantom show a good agreement with theoretical analysis and tensile testing. The strains of the chicken breast tissue indicate anisotropic biomechanical properties. This study provides an effective method for 3D biomechanical property studies of soft tissue and improves the development of 3D OCE techniques.

Research progress of DSCM in sub-pixel level measurement

The digital speckle correlation method (DSCM) is a full-field and non-contact optical measurement method. This paper analyzes the research status of the sub-pixel algorithm in DSCM, and analyzes the research progress and data of the algorithm. The inverse compositional Gaussian-Newton algorithm (IC-GN) has obvious advantages. The conclusion is that the inverse compositional Gaussian-Newton algorithm has obvious comprehensive advantages over traditional sub-pixel algorithms in terms of measurement accuracy and computational speed.

Improvements of the Inverse Compositional Algorithm for Parametric Motion Estimation

In this work, we propose several improvements of the inverse compositional algorithm for parametric registration. We propose an improved handling of boundary pixels, a different color handling and gradient estimation, and the possibility to skip scales in the multiscale coarse-to-fine scheme. In an experimental part, we analyze the influence of the modifications. The estimation accuracy is at least improved by a factor 1.3 while the computation time is at least reduced by a factor 2.2 for color images.

Efficient and Robust Direct Image Registration Based on Joint Geometric and Photometric Lie Algebra.

This paper considers the joint geometric and photometric image registration problem. The inverse compositional (IC) algorithm and the efficient second-order minimization (ESM) algorithm are two typical efficient methods applied to the geometric registration problem. Their efficiency stems from the utilization of the group structure of geometric transformations. To allow for photometric variations, the dual IC algorithm (DIC) proposed by Bartoli performs joint geometric and photometric image registration by extending the IC algorithm. The group structures of both geometric and photometric transformations are exploited. Despite the robustness to large photometric variations, DIC is vulnerable to large geometric deformations. The ESM algorithm is extended by Silveira et al. to address photometric variations. In their approach, the photometric transformations are modeled in Euclidean space. Their approach is robust to relatively large geometric and photometric transformations; however, it is not efficient for large photometric variations. We propose a new efficient and robust image registration method by exploiting the non-Euclidean Lie group structure of joint geometric and photometric transformations for both grayscale and color images. The image registration is formulated as a nonlinear least squares problem. In our method, the geometric and photometric transformations are jointly parameterized by their corresponding Lie algebras. Based on this parameterization approach, the second-order approximation strategy of ESM is employed to optimize the joint geometric and photometric parameters. The error function in the nonlinear least squares problem is approximated by a second-order Taylor expansion with respect to joint geometric and photometric parameters without computing the Hessian matrix. For further efficiency, independent convergence criteria for geometric and photometric parameters are used in the iterative optimization process. The superiority of our proposed method over the previous methods, in terms of efficiency, accuracy, and robustness, is demonstrated through extensive experiments on synthetic and real data.

Optimized digital speckle patterns for digital image correlation by consideration of both accuracy and efficiency.

The technique of digital image correlation (DIC), which has been widely used for noncontact deformation measurements in both the scientific and engineering fields, is greatly affected by the quality of speckle patterns in terms of its performance. This study was concerned with the optimization of the digital speckle pattern (DSP) for DIC in consideration of both the accuracy and efficiency. The root-mean-square error of the inverse compositional Gauss-Newton algorithm and the average number of iterations were used as quality metrics. Moreover, the influence of subset sizes and the noise level of images, which are the basic parameters in the quality assessment formulations, were also considered. The simulated binary speckle patterns were first compared with the Gaussian speckle patterns and captured DSPs. Both the single-radius and multi-radius DSPs were optimized. Experimental tests and analyses were conducted to obtain the optimized and recommended DSP. The vector diagram of the optimized speckle pattern was also uploaded as reference.

Self-calibration single-lens 3D video extensometer for high-accuracy and real-time strain measurement.

The accuracy of strain measurement using a common optical extensometer with two-dimensional (2D) digital image correlation (DIC) is not sufficient for experimental applications due to the effect of out-of-plane motion. Although three-dimensional (3D) DIC can measure all three components of displacement without introducing in-plane displacement errors, 3D-DIC requires the stringent synchronization between two digital cameras and requires complicated system calibration of binocular stereovision, which makes the measurement rather inconvenient. To solve the problems described above, this paper proposes a self-calibration single-lens 3D video extensometer for non-contact, non-destructive and high-accuracy strain measurement. In the established video extensometer, a single-lens 3D imaging system with a prism and two mirrors is constructed to acquire stereo images of the test sample surface, so the problems of synchronization and out-of-plane displacement can be solved easily. Moreover, a speckle-based self-calibration method which calibrates the single-lens stereo system using the reference speckle image of the specimen instead of the calibration targets is proposed, which will make the system more convenient to be used without complicated calibration. Furthermore, an efficient and robust inverse compositional Gauss-Newton algorithm combined with a robust stereo matching stage is employed to achieve high-accuracy and real-time subset-based stereo matching. Tensile tests of an Al-alloy specimen were performed to demonstrate the feasibility and effectiveness of the proposed self-calibration single-lens 3D video extensometer.

Advanced video extensometer for non-contact, real-time, high-accuracy strain measurement.

We developed an advanced video extensometer for non-contact, real-time, high-accuracy strain measurement in material testing. In the established video extensometer, a "near perfect and ultra-stable" imaging system, combining the idea of active imaging with a high-quality bilateral telecentric lens, is constructed to acquire high-fidelity video images of the test sample surface, which is invariant to ambient lighting changes and small out-of-plane motions occurred between the object surface and image plane. In addition, an efficient and accurate inverse compositional Gauss-Newton algorithm incorporating a temporal initial guess transfer scheme and a high-accuracy interpolation method is employed to achieve real-time, high-accuracy displacement tracking with negligible bias error. Tensile tests of an aluminum sample and a carbon fiber filament sample were performed to demonstrate the efficiency, repeatability and accuracy of the developed advanced video extensometer. The results indicate that longitudinal and transversal strains can be estimated and plotted at a rate of 117 fps and with a maximum strain error less than 30 microstrains.

A Method for Non-Rigid Face Alignment via Combining Local and Holistic Matching.

We propose a method for non-rigid face alignment which only needs a single template, such as using a person’s smile face to match his surprise face. First, in order to be robust to outliers caused by complex geometric deformations, a new local feature matching method called K Patch Pairs (K-PP) is proposed. Specifically, inspired by the state-of-art similarity measure used in template matching, K-PP is to find the mutual K nearest neighbors between two images. A weight matrix is then presented to balance the similarity and the number of local matching. Second, we proposed a modified Lucas-Kanade algorithm combined with local matching constraint to solve the non-rigid face alignment, so that a holistic face representation and local features can be jointly modeled in the object function. Both the flexible ability of local matching and the robust ability of holistic fitting are included in our method. Furthermore, we show that the optimization problem can be efficiently solved by the inverse compositional algorithm. Comparison results with conventional methods demonstrate our superiority in terms of both accuracy and robustness.

Shape-appearance-correlated active appearance model

Among the challenges faced by current active shape or appearance models, facial-feature localization in the wild, with occlusion in a novel face image, i.e. in a generic environment, is regarded as one of the most difficult computer-vision tasks. In this paper, we propose an Active Appearance Model (AAM) to tackle the problem of generic environment. Firstly, a fast face-model initialization scheme is proposed, based on the idea that the local appearance of feature points can be accurately approximated with locality constraints. Nearest neighbors, which have similar poses and textures to a test face, are retrieved from a training set for constructing the initial face model. To further improve the fitting of the initial model to the test face, an orthogonal CCA (oCCA) is employed to increase the correlation between shape features and appearance features represented by Principal Component Analysis (PCA). With these two contributions, we propose a novel AAM, namely the shape-appearance-correlated AAM (SAC-AAM), and the optimization is solved by using the recently proposed fast simultaneous inverse compositional (Fast-SIC) algorithm. Experiment results demonstrate a 5–10% improvement on controlled and semi-controlled datasets, and with around 10% improvement on wild face datasets in terms of fitting accuracy compared to other state-of-the-art AAM models.

Dynamic displacement measurement of large-scale structures based on the Lucas–Kanade template tracking algorithm

The development of optics and computer technologies enables the application of the vision-based technique that uses digital cameras to the displacement measurement of large-scale structures. Compared with traditional contact measurements, vision-based technique allows for remote measurement, has a non-intrusive characteristic, and does not necessitate mass introduction. In this study, a high-speed camera system is developed to complete the displacement measurement in real time. The system consists of a high-speed camera and a notebook computer. The high-speed camera can capture images at a speed of hundreds of frames per second. To process the captured images in computer, the Lucas–Kanade template tracking algorithm in the field of computer vision is introduced. Additionally, a modified inverse compositional algorithm is proposed to reduce the computing time of the original algorithm and improve the efficiency further. The modified algorithm can rapidly accomplish one displacement extraction within 1ms without having to install any pre-designed target panel onto the structures in advance. The accuracy and the efficiency of the system in the remote measurement of dynamic displacement are demonstrated in the experiments on motion platform and sound barrier on suspension viaduct. Experimental results show that the proposed algorithm can extract accurate displacement signal and accomplish the vibration measurement of large-scale structures.

Active Orientation Models for Face Alignment In-the-Wild

We present Active Orientation Models (AOMs), generative models of facial shape and appearance, which extend the well-known paradigm of Active Appearance Models (AAMs) for the case of generic face alignment under unconstrained conditions. Robustness stems from the fact that the proposed AOMs employ a statistically robust appearance model based on the principal components of image gradient orientations. We show that when incorporated within standard optimization frameworks for AAM learning and fitting, this kernel Principal Component Analysis results in robust algorithms for model fitting. At the same time, the resulting optimization problems maintain the same computational cost. As a result, the main similarity of AOMs with AAMs is the computational complexity. In particular, the project-out version of AOMs is as computationally efficient as the standard project-out inverse compositional algorithm, which is admittedly one of the fastest algorithms for fitting AAMs. We verify experimentally that: 1) AOMs generalize well to unseen variations and 2) outperform all other state-of-the-art AAM methods considered by a large margin. This performance improvement brings AOMs at least in par with other contemporary methods for face alignment. Finally, we provide MATLAB code at http://ibug.doc.ic.ac.uk/resources.

Rationalizing Efficient Compositional Image Alignment

We study the issue of computational efficiency for Gauss-Newton (GN) non-linear least-squares optimization in the context of image alignment. We introduce the Constant Jacobian Gauss-Newton (CJGN) optimization, a GN scheme with constant Jacobian and Hessian matrices, and the equivalence and independence conditions as the necessary requirements that any function of residuals must satisfy to be optimized with this efficient approach. We prove that the Inverse Compositional (IC) image alignment algorithm is an instance of a CJGN scheme and formally derive the compositional and extended brightness constancy assumptions as the necessary requirements that must be satisfied by any image alignment problem so it can be solved with an efficient compositional scheme. Moreover, in contradiction with previous results, we also prove that the forward and inverse compositional algorithms are not equivalent. They are equivalent, however, when the extended brightness constancy assumption is satisfied. To analyze the impact of the satisfaction of these requirements we introduce a new image alignment evaluation framework and the concepts of short- and wide-baseline Jacobian. In wide-baseline Jacobian problems the optimization will diverge if the requirements are not satisfied. However, with a good initialization, a short-baseline Jacobian problem may converge even if the requirements are not satisfied.

Elastic registration for airborne multispectral line scanners

The multispectral line scanner is one of the most popular payloads for aerial remote sensing (RS) applications. Scanners with large field of view (FOV) improve efficiency in Earth observation. Small-volume instruments with a short focal length and a large FOV, however, may bring a problem: different nonlinear warping and local transformation exist between bands. Alignment accuracy of bands is a criterion impacting product quality in RS. A band-to-band elastic image registration method is proposed for solving the problem. Rather than ignoring the intensity variation and carrying out an intensity-based registration between bands straightforwardly, we construct feature images and use them to conduct an intensity-based elastic image registration. In this method, the idea of the inverse compositional algorithm is employed and expanded when dealing with local warping, and a smoothness constraint is also added in this procedure. Experimental results show that the proposed band-to-band registration method works well both visually and quantitatively. The outstanding performance of the method also encourages potential applications for other new types of airborne multispectral imagers.