Sub-pixel Registration Algorithm Research Articles

Review of research progress and development trend of digital image correlation

PurposeThe purpose of this paper is to provide a comprehensive review of a non-contact full-field optical measurement technique known as digital image correlation (DIC).Design/methodology/approachThe approach of this review paper is to introduce the research pertaining to DIC. It comprehensively covers crucial facets including its principles, historical development, core challenges, current research status and practical applications. Additionally, it delves into unresolved issues and outlines future research objectives.FindingsThe findings of this review encompass essential aspects of DIC, including core issues like the subpixel registration algorithm, camera calibration, measurement of surface deformation in 3D complex structures and applications in ultra-high-temperature settings. Additionally, the review presents the prevailing strategies for addressing these challenges, the most recent advancements in DIC applications across quasi-static, dynamic, ultra-high-temperature, large-scale and micro-scale engineering domains, along with key directions for future research endeavors.Originality/valueThis review holds a substantial value as it furnishes a comprehensive and in-depth introduction to DIC, while also spotlighting its prospective applications.

Universal method using a pre-deformed reference subset to eliminate the interpolation bias in digital image correlation

The high measurement accuracy of the digital image correlation (DIC) method is derived from the sub-pixel registration algorithm, which interpolates the intensities at the sub-pixel position in the image. The displacement error caused by the interpolation is a systematic bias in the DIC method, known as the sinusoidal bias in the sub-pixel translation experiment. Although the interpolation bias has been well researched, there is a lack of a universal method to eliminate interpolation bias. In this work, we propose a universal method to eliminate the interpolation bias using a pre-deformed reference subset; pixel points in the pre-deformed subset are deviated from the integer-pixel location. The purpose of the adjustment is to set the deformed pixel points at a specific position, so that the interpolation bias of all deformed pixel points cancels each other out, close to zero. The adjustment of the pre-deformed reference subset is related with the subset size and subset deformation. Numerical experiments including DIC challenge data and a real uniaxial tensile test were conducted to verify the effectiveness and universality of the proposed method, contributing to improved measurement accuracy. Considering the effect of pixel point location on the interpolation bias, this work proposes a universal method to eliminate the interpolation bias and provides a perspective to study DIC errors.

An Open-Source Real-Time Motion Correction Plug-In for Single-Photon Calcium Imaging of Head-Mounted Microscopy.

Single-photon-based head-mounted microscopy is widely used to record the brain activities of freely-moving animals. However, during data acquisition, the free movement of animals will cause shaking in the field of view, which deteriorates subsequent neural signal analyses. Existing motion correction methods applied to calcium imaging data either focus on offline analyses or lack sufficient accuracy in real-time processing for single-photon data. In this study, we proposed an open-source real-time motion correction (RTMC) plug-in for single-photon calcium imaging data acquisition. The RTMC plug-in is a real-time subpixel registration algorithm that can run GPUs in UCLA Miniscope data acquisition software. When used with the UCLA Miniscope, the RTMC algorithm satisfies real-time processing requirements in terms of speed, memory, and accuracy. We tested the RTMC algorithm by extending a manual neuron labeling function to extract calcium signals in a real experimental setting. The results demonstrated that the neural calcium dynamics and calcium events can be restored with high accuracy from the calcium data that were collected by the UCLA Miniscope system embedded with our RTMC plug-in. Our method could become an essential component in brain science research, where real-time brain activity is needed for closed-loop experiments.

Noise-robust phase retrieval by optics path modulation with adaptive feedback

We propose a compact computational imaging system with adaptive weighted feedback amplitude-phase retrieval (awfAPR) algorithm, in which the traditional axial linear scanning method is replaced by periodic optical path modulation (OPM) strategy. The introduced adaptive feedback scenario can significantly suppress noise contamination. Specifically, OPM is achieved by the rotation of gradient thicknesses of the transparent optical medium. Compared with multi-distance/height scanning (MDPR), all system components in the OPM system are fixed at discrete lateral positions. Besides, according to the auxiliary of corresponding auto-focusing criterion and sub-pixel registration algorithms, the OPM system is also self-calibration and independent of the accuracy of rotation and thickness parameter. Furthermore, experimental results show that the adaptive weighted feedback strategy remarkably improves the stability and robustness toward the noise. OPM has flexible adaptability to optical path modulation method. The proposed optical system configuration has minimal hardware requirements and is anticipated to expand dynamic real-time imaging.

Lensless Fourier-transform terahertz digital holography for real-time full-field phase imaging

With the development of continuous-wave terahertz (THz) sources and array detectors, the pursuit of high-fidelity real-time imaging is receiving significant attention within the THz community. Here, we report a real-time full-field THz phase imaging approach based on lensless Fourier-transform THz digital holography. A triangular interferometric layout is proposed based on an oblique illumination of 2.52 THz radiation, which is different from other lensless holographic configurations at other frequencies. A spherical reference beam is generated by a reflective parabolic mirror with minor propagation loss. The complex-valued images are reconstructed using a single inverse Fourier transform of the hologram without complex calculation of the diffraction propagation. The experimental result for a Siemens star validates the lateral resolution of ∼ 346 μm in the diagonal direction. Sub-pixel image registration and image stitching algorithms are applied to enlarge the area of the reconstructed images. The dehydration process of an aquatic plant leaf (Hottonia inflata) is monitored for the first time, to the best of our knowledge, at the THz band. Rapid variations in water content and morphology are measured with a time interval of 0.6 s and a total time of 5 min from a series of reconstructed amplitude and phase images, respectively. The proposed method has the potential to become a powerful tool to investigate spontaneous phenomena at the THz band.

A sub-pixel image registration algorithm based on SURF and M-estimator sample consensus

Due to the influence of various conditions and uncertain difficulties for remote sensing images, image registration is still a challenging task. Considering the registration accuracy of pixel level cannot satisfy the requirements of some related applications, we put forward a sub-pixel image registration method based on speeded up robust features and M-estimator sample consensus. It mainly involves four aspects. At first, extract sub-pixel level feature points based on SURF algorithm. Next, obtain the initial matching point pairs based on Sum of Squared Difference and Fast Library for Approximate Nearest Neighbors algorithms. And then, remove the mismatched pair of points based on M-estimator sample consensus algorithm. Finally, calculate geometric transformation matrix based on purified matching points to reach sub-pixel accuracy image registration. Experimental results for several remote sensing image pairs with displacement, noise added, rotation, and different sensors, times and sizes, show that the proposed method can get more anti-interference matches than other methods, and take smaller computational cost in registration process.

A Fast Subpixel Registration Algorithm Based on Single-Step DFT Combined with Phase Correlation Constraint in Multimodality Brain Image.

Multimodality brain image registration technology is the key technology to determine the accuracy and speed of brain diagnosis and treatment. In order to achieve high-precision image registration, a fast subpixel registration algorithm based on single-step DFT combined with phase correlation constraint in multimodality brain image was proposed in this paper. Firstly, the coarse positioning at the pixel level was achieved by using the downsampling cross-correlation model, which reduced the Fourier transform dimension of the cross-correlation matrix and the multiplication of the discrete Fourier transform matrix, so as to speed up the coarse registration process. Then, the improved DFT multiplier of the matrix multiplication was used in the neighborhood of the coarse point, and the subpixel fast location was achieved by the bidirectional search strategy. Qualitative and quantitative simulation experiment results show that, compared with comparison registration algorithms, our proposed algorithm could greatly reduce space and time complexity without losing accuracy.

Non-contact Quantification of Jugular Venous Pulse Waveforms from Skin Displacements

The jugular venous (JV) pressure waveform is a non-invasive, proven indicator of cardiovascular disease. Conventional clinical methods for assessing these waveforms are often overlooked because they require specialised expertise, and are invasive and expensive to implement. Recently, image-based methods have been used to quantify JV pulsation waveforms on the skin as an indirect way of estimating the pressure waveforms. However, these existing image-based methods cannot explicitly measure skin deformations and rely on the use of photoplethysmography (PPG) devices for identification of the pulsatile waveforms. As a result, they often have limited accuracy and robustness and are unsuitable in the clinical environment. Here, we propose a technique to directly measure skin deformations caused by the JV pulse using a very accurate subpixel registration algorithm. The method simply requires images obtained from the subject’s neck using a commodity camera. The results show that our measured waveforms contained all of the essential features of diagnostic JV waveforms in all of 19 healthy subjects tested in this study, indicating a significantly important capability for a potential future diagnostic device. The shape of our measured JV displacement waveforms was validated using waveforms measured with a laser displacement sensor, where the average correlation score between the two waveforms was 0.93 ± 0.05. In addition, synchronously recorded ECG signals were used to verify the timings of diagnostic features of the measured waveforms. To our knowledge, this is the first use of image registration for direct measurement of JV displacement waveforms. Significant advantages of our novel method include the high precision of our measurements, and the ability to use ordinary cameras, such as those in modern mobile phones. These advantages will enable the development of affordable and accessible devices to measure JV waveforms for cardiac diagnostics in the clinical environment. Future devices based on this technology may provide viable options for telemedicine applications, point of care diagnostics, and mobile-based cardiac health monitoring systems.

Elimination of systematic error in digital image correlation caused by intensity interpolation by introducing position randomness to subset points

A strategy called random subset is presented to reduce systematic error in digital image correlation caused by intensity interpolation. In classical digital image correlation techniques, the subset points fall at integer locations; by contrast, the proposed random subset strategy imposes random offsets on the conventional locations, and consequently the subset points are located at non-integer positions. The introduction of position randomness to subset points makes the random subset strategy distinctly different from the classical digital image correlation technique. The random offsets can follow various probability distributions. In this work, two types of random offset were studied as examples, and the maximum of the random offsets was adjusted to modify performance. Using the random subset strategy, random-subset-based subpixel registration algorithms were developed in this study, including the random-subset-based forward additive Gaussian–Newton algorithm and the random-subset-based inverse compositional Gauss–Newton algorithm. The measurement accuracy of the random subset-based subpixel registration algorithms was investigated. Numerical simulations and real-world experiments demonstrated that as the maximum of the random offsets increases, the amplitude of the sinusoidal-shaped systematic error changes from positive to negative. Hence, if an appropriate random offset is selected, systematic error can be considerably reduced. In fact, the accuracy enhancement is greater than one order of magnitude, and systematic error is virtually eliminated.

Fast registration of UAV aerial images based on improved optical-flow model combined with feature-point matching

With a large number of registration algorithms proposed, image registration techniques have achieved rapid development. However, there still exist many deficiencies in aerial images registration where high speed and accuracy are difficult to simultaneously achieve for real-time processing. In order to achieve large-scale and high-precision image registration for unmanned aerial vehicle(UAV) aerial images, a novel and fast sub-pixel image registration algorithm based on improved optical-flow model combined with feature-point matching is proposed in this paper. Firstly, the coarse selection at the feature level is achieved by using the feature-point model, which reduces the number of non-feature points so as to speed up the coarse registration process. Then, the improved pyramid optical-flow model is adopted in the neighborhood of the coarse point, and the sub-pixel fast location is achieved by the bidirectional search strategy. Simulation experiment results show that compared with common image registration based LK optical-flow or feature-point matching, our proposed algorithm will greatly reduce space complexity and time complexity without losing accuracy.

Equal Noise Resistance of Two Mainstream Iterative Sub-pixel Registration Algorithms in Digital Image Correlation

The inverse compositional Gauss-Newton (IC-GN) algorithm and the forward additive Newton-Raphson (FA-NR) algorithm are two mainstream iterative sub-pixel registration algorithms in digital image correlation. This study compares the accuracy and convergence ability of the two algorithms by theoretical analysis and numerical experiments in the speckle images that have been contaminated with artificial Gaussian noise. Based on the derived error model, the systematic errors of the two algorithms are dominated by interpolation-induced error and are insensitive to noise. The random errors are proportional to the noise level. The noise also reduces the convergence radius and rate in the two algorithms. The two algorithms demonstrate equal noise resistance due to their mathematical equivalence. These conclusions are well supported by the experimental study. The recently reported vulnerability of the FA-NR algorithm to noise is not associated with the inherent flaw of the algorithm but with its implementation. If an inappropriate method is employed to estimate the gradients at sub-pixel locations in the FA-NR algorithm, abnormally large errors may be induced. This problem can be eliminated using the method that is proposed in this study, which has an insignificant extra-computation cost.

An image registration algorithm based on phase correlation and the classical Lucas–Kanade technique

Image registration is defined as an important process in image processing in order to align two or more images. A new image registration algorithm for translated and rotated pairs of 2D images is presented in order to achieve subpixel accuracy and spend a small fraction of computation time. To achieve the accurate rotation estimation, we propose a two-step method. The first step uses the Fourier Mellin Transform and phase correlation technique to get the large rotation, then the second one uses the Fourier Mellin Transform combined with an enhance Lucas–Kanade technique to estimate the accurate rotation. For the subpixel translation estimation, the proposed algorithm suggests an improved Hanning window as a preprocessing task to reduce the noise in images then achieves a subpixel registration in two steps. The first step uses the spatial domain approach which consists of locating the peak of the cross-correlation surface, while the second uses the frequency domain approach, based on low-frequency (aliasing-free part) of aliased images. Experimental results presented in this work show that the proposed algorithm reduces the computational complexities with a better accuracy compared to other subpixel registration algorithms.

Super Resolution Imaging Based on a Dynamic Single Pixel Camera

In this paper, we present a super resolution imaging method based on a dynamic single pixel camera. First, the proposed method obtains subhigh-resolution (HR) images with subpixel shifts by the dynamic single pixel camera. To achieve this purpose, the patterns on the digital micromirror device are subpixel shifted instead of mechanical scanning. Then, the final HR image can be reconstructed by sub-HR images and wiener filter. In order to avoid the subpixel shifts error that caused by the instability of the system, a subpixel registration algorithm is adopted. Besides, to improve the signal-noise-ratio (SNR) performance of the measurements, a simple and effective method based on the correlation of the signal and the randomness of the noise is adopted. Compared with traditional single pixel imaging system, the proposed method has a remarkable performance at spatial resolution and SNR. The feasibility of the proposed method is demonstrated by simulations and experiments.

Differential standard deviation of log-scale intensity based optical coherence tomography angiography.

In this paper, a differential standard deviation of log-scale intensity (DSDLI) based optical coherence tomography angiography (OCTA) is presented for calculating microvascular images of human skin. The DSDLI algorithm calculates the variance in difference images of two consecutive log-scale intensity based structural images from the same position along depth direction to contrast blood flow. The en face microvascular images were then generated by calculating the standard deviation of the differential log-scale intensities within the specific depth range, resulting in an improvement in spatial resolution and SNR in microvascular images compared to speckle variance OCT and power intensity differential method. The performance of DSDLI was testified by both phantom and in vivo experiments. In in vivo experiments, a self-adaptive sub-pixel image registration algorithm was performed to remove the bulk motion noise, where 2D Fourier transform was utilized to generate new images with spatial interval equal to half of the distance between two pixels in both fast-scanning and depth directions. The SNRs of signals of flowing particles are improved by 7.3 dB and 6.8dB on average in phantom and in vivo experiments, respectively, while the average spatial resolution of images of in vivo blood vessels is increased by 21%.

Quality assessment of speckle patterns for DIC by consideration of both systematic errors and random errors

The performance of digital image correlation (DIC) is influenced by the quality of speckle patterns significantly. Thus, it is crucial to present a valid and practical method to assess the quality of speckle patterns. However, existing assessment methods either lack a solid theoretical foundation or fail to consider the errors due to interpolation. In this work, it is proposed to assess the quality of speckle patterns by estimating the root mean square error (RMSE) of DIC, which is the square root of the sum of square of systematic error and random error. Two performance evaluation parameters, respectively the maximum and the quadratic mean of RMSE, are proposed to characterize the total error. An efficient algorithm is developed to estimate these parameters, and the correctness of this algorithm is verified by numerical experiments for both 1 dimensional signal and actual speckle images. The influences of correlation criterion, shape function order, and sub-pixel registration algorithm are briefly discussed. Compared to existing methods, method presented by this paper is more valid due to the consideration of both measurement accuracy and precision.

Digital Image Correlation with Enhanced Accuracy and Efficiency: A Comparison of Two Subpixel Registration Algorithms

The two major subpixel registration algorithms, currently being used in subset-based digital image correlation, are the classic Newton-Raphson (FA-NR) algorithm with forward additive mapping strategy and the recently introduced inverse compositional Gauss-Newton (IC-GN) algorithm. Although the equivalence of these two algorithms has been proved in existing studies, practical implementations of the two subpixel registration algorithms do involve differences, and therefore lead to different performance. In the present work, detailed theoretical error analyses of the two algorithms are performed. Based on the simple sum of squared difference criterion and the practical first-order shape function, analytic formulae that can quantify both the bias error (systematic error) and the variability (random error) in the displacements measured by IC-GN and FA-NR algorithms with various interpolation methods (i.e., cubic convolution interpolation, cubic polynomial interpolation, cubic B-spline interpolation and quintic B-spline interpolation) are derived. It is shown that, compared with FA-NR algorithm, IC-GN algorithm leads to reduced bias error in displacement estimation by eliminating noise-induced bias error, and gives rise on the average to smaller random errors in displacement estimation in the cases of high noise levels or using small subsets. Numerical tests with precisely controlled subpixel displacements confirm the correctness of the theoretical derivations. The results reveal that IC-GN algorithm outperforms the classic FA-NR algorithm not only in terms of computational efficiency, but also in respect of subpixel registration accuracy and noise-proof performance, and is strongly recommended as a standard subpixel registration algorithm for practical DIC applications instead of FA-NR algorithm.

Noise robustness and parallel computation of the inverse compositional Gauss–Newton algorithm in digital image correlation

Abstract The inverse compositional Gauss–Newton (IC-GN) algorithm is one of the most popular sub-pixel registration algorithms in digital image correlation (DIC). The IC-GN algorithm, compared with the traditional forward additive Newton–Raphson (FA-NR) algorithm, can achieve the same accuracy in less time. However, there are no clear results regarding the noise robustness of IC-GN algorithm and the computational efficiency is still in need of further improvements. In this paper, a theoretical model of the IC-GN algorithm was derived based on the sum of squared differences correlation criterion and linear interpolation. The model indicates that the IC-GN algorithm has better noise robustness than the FA-NR algorithm, and shows no noise-induced bias if the gray gradient operator is chosen properly. Both numerical simulations and experiments show good agreements with the theoretical predictions. Furthermore, a seed point-based parallel method is proposed to improve the calculation speed. Compared with the recently proposed path-independent method, our model is feasible and practical, and it can maximize the computing speed using an improved initial guess. Moreover, we compared the computational efficiency of our method with that of the reliability-guided method using a four-point bending experiment, and the results show that the computational efficiency is greatly improved. This proposed parallel IC-GN algorithm has good noise robustness and is expected to be a practical option for real-time DIC.

New measurement method of Poisson’s ratio of thin films by applying digital image correlation technique

This paper describes a new method to measure the Poisson’s ratio of thin films by applying digital image correlation technique. Image regions in region of interest (ROI) before and after deformation are compared to each other for precise measurement. For the microsized sample, a representative sub-pixel registration algorithm is applied which is the Newton-Raphson iteration method combined with bicubic intensity interpolation. The first order derivatives of the intensities in the axial loading direction and the transverse direction to loading axis were solved and Poisson’s ratio is the absolute value of the ratio of transverse strain to the corresponding axial strain resulting from uniformly distributed axial stress below the proportional limit of the material. To verify this method, microtensile tests of BeCu thin film were performed and the micro-tensile properties including Poisson’s ratio of BeCu thin film were measured. To obtain more accurate Poisson’s ratio, the stress-strain curve was divided into several intervals to calculate the Poisson’s ratios of BeCu. And, the average value of the Poisson’s ratios of several intervals came to 0.304. Besides, the ultimate tensile strength, 0.2% yield strength and elastic modulus of BeCu were obtained and came to 1065 MPa, 1037 MPa, 102 GPa, respectively.

Detecting the Storm Movement by Sub Pixel Registration Approach of Newton Raphson Method

The change detection, storm detection and forecasting the weather condition consist of in the field of natural disaster management as the essential role. The detecting of the storm direction is provided to manage and to prevent the possible attacked region of the land. The sub pixel registration approach is applied for computing the movement of the interested cloud region. The direction of the storm can be tracked from these computations. Especially, the attacked region of Myanmar Land is forecasted from multi-date satellite image by using the digital image correlation based on the sub-pixel accuracy with Newton Raphson method. The proposed method is very effective because of its experiment and its result. Human can't be terminated the natural disaster. The weather prediction is very important for human daily life to prevent and reduce the attack of natural disaster. The storm direction detecting system is developed based on the sub-pixel accuracy with Newton Raphson method. The pixel correlation among the images is computed accurately by the Digital Image Correlation (DIC) for motion detection. The main application of DIC is in experimental mechanics. During the last two decades, various applications had been reported for two-dimensional measurement of displacement and strain field using the method of DIC. The most fundamental applications of two-dimensional DIC are found in fracture mechanics studies (1), (2), including measurement of strain field near crack-tips at high temperatures (3), strain measurement near stationary and growing crack-tips (4) and measurement of crack-tip opening displacement during crack growth (5), (6). H. A. Bruck et al. (1) proves that the computation of the digital image correlation using Newton Raphson method of partial differential correction. The integer pixel registration algorithm is used in their correlation computation. Moreover, DIC should serve as a valuable tool for biomedical research and the two-dimensional in-plane displacement and strain calculation problem and strain deformation (7)-(9). et al. (10) describes the performance of three different types of sub-pixel displacement registration algorithms in terms of the registration accuracy and the computational efficiency.

Accurate displacement measurement via a self-adaptive digital image correlation method based on a weighted ZNSSD criterion

Digital image correlation (DIC) technique has been increasingly employed to implement surface deformation measurements in many engineering fields. Practically, it has been demonstrated that the choice of subset sizes exerts a strong influence on measurement results of DIC, especially when there exists locally larger deformation over the subsets involved. This paper proposes a novel subpixel registration algorithm with Gaussian windows to implicitly optimize the subset sizes by adjusting the shape of Gaussian windows in a self-adaptive fashion with the aid of a so-called weighted zero-normalized sum-of-squared difference correlation criterion. The feasibility and effectiveness of the self-adaptive algorithm are carefully verified through a set of well-designed synthetic speckle images, which indicates that the presented algorithm is able to greatly enhance the accuracy and precision of displacement measurements as compared with the traditional subpixel registration methods.