Sub-pixel Precision Research Articles

An Image Registration Method Based on Correlation Matching of Dominant Scatters for Distributed Array ISAR.

Distributed array radar provides new prospects for three-dimensional (3D) inverse synthetic aperture radar (ISAR) imaging. The accuracy of image registration, as an essential part of 3D ISAR imaging, affects the performance of 3D reconstruction. In this paper, the imaging process of distributed array ISAR is proposed according to the imaging model. The ISAR images of distributed array radar at different APCs have different distribution of scatters. When the local distribution of scatters for the same target are quite different, the performance of the existing ISAR image registration methods may not be optimal. Therefore, an image registration method is proposed by integrating the feature-based method and the area-based method. The proposed method consists of two stages: coarse registration and fine registration. In the first stage, a dominant scatters model is established based on scale-invariant feature transform (SIFT). In the second stage, sub-pixel precision registration is achieved using the local correlation matching method. The effectiveness of the proposed method is verified by comparison with other image registration methods. The 3D reconstruction of the registered experimental data is carried out to assess the practicability of the proposed method.

A comprehensive geometric quality assessment approach for MSG SEVIRI imagery

Geometric Quality Assessment (GQA) of Earth Observation satellites is crucial for monitoring the system performance and anticipate the accuracy of their products. The Spinning Enhanced Visible and InfraRed Imager (SEVIRI) sensors aboard the geostationary Meteosat satellites of European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) acquire multispectral images at 5 min (rapid-scan) and 15 min (full-disk scan) intervals in whiskbroom fashion. The existing GQA method used for the SEVIRI images involves matching of landmark points located on the shorelines, which prevents from comprehensive analysis inland. Limitations on detecting further landmarks (keypoints) in the images arise from low spatial resolution (1 and 3 km), large radiometric differences between the spectral bands and low textural content, which is even poorer for night time thermal infrared images. Within the present study initiated by EUMETSAT, a new approach for SEVIRI Level 1.5 imagery was developed for the relative, absolute and interband GQA by using dense image matching with sub-pixel precision. In the study, various image filtering and texture extraction methods were evaluated to obtain dense and reliable keypoints over the full-disk image area. Global image mosaics with superior spatial resolution and accuracy were employed as external reference. 2D shifts between search and reference images were determined the using a sub-pixel image matching technique together with a robust outlier elimination strategy. The results show that the textural information in SEVIRI images can be enhanced greatly by using various filtering methods, such as Laplacian, Sobel, Wallis and Local Binary Patterns. The Laplacian filter was found more suitable for the absolute GQA of infrared band images, which was the most difficult to band to assess. Depending on the cloud cover, thousands of well-distributed keypoints can be detected and employed as independent check points in the absolute GQA process. With the developed GQA Tool, new acquisitions can be assessed rapidly and various accuracy measures on the image geometric quality can also be achieved.

Automated georeferencing of Diwata-2 multispectral imagery using feature matching

The Diwata-2 microsatellite is a 56 kg optical microsatellite that features different optical sensors such as the medium resolution Spaceborne Multispectral Sensor (SMI) and a high-resolution sensor called High Precision Telescope (HPT). This research aims to develop an automated georeferencing workflow for coregistered multiband HPT and SMI imagery with a GSD of 5 meters and 127 meters respectively. Georeferencing is done using a mixture of FAST and SIFT feature matching algorithms where HPT imagery used SIFT descriptors and detectors and SMI used FAST detectors and SIFT descriptors. The research used freely available Landsat- 8 imagery which was processed into cloud-free mosaics as reference data. Accuracy assessment is automated using AROSICS software which performs geometric correction and calculates the RMSE of local shifts based on image tie points which are automatically generated using the Diwata image as secondary image and Landsat-8 as reference image. Output of the feature matching algorithms has achieved high accuracy with an average RMSE value of 2.55 meters for HPT imagery and 65.96 meters for SMI imagery. Both RMSE values are close to half the GSD of both sensors which indicate sub-pixel precision. Future research for this method includes additional method for keypoint filtering to further increase the accuracy of the feature matching.

Data Acquisition and Preparation for Dual-Reference Deep Learning of Image Super-Resolution.

The performance of deep learning based image super-resolution (SR) methods depend on how accurately the paired low and high resolution images for training characterize the sampling process of real cameras. Low and high resolution (LR ∼ HR) image pairs synthesized by degradation models (e.g., bicubic downsampling) deviate from those in reality; thus the synthetically-trained DCNN SR models work disappointingly when being applied to real-world images. To address this issue, we propose a novel data acquisition process to shoot a large set of LR ∼ HR image pairs using real cameras. The images are displayed on an ultra-high quality screen and captured at different resolutions. The resulting LR ∼ HR image pairs can be aligned at very high sub-pixel precision by a novel spatial-frequency dual-domain registration method, and hence they provide more appropriate training data for the learning task of super-resolution. Moreover, the captured HR image and the original digital image offer dual references to strengthen supervised learning. Experimental results show that training a super-resolution DCNN by our LR ∼ HR dataset achieves higher image quality than training it by other datasets in the literature. Moreover, the proposed screen-capturing data collection process can be automated; it can be carried out for any target camera with ease and low cost, offering a practical way of tailoring the training of a DCNN SR model separately to each of the given cameras.

On the Relationship Between EB-3 Profiles and Microtubules Growth in Cultured Cells.

Microtubules are dynamic structures undergoing rapid growth and shrinkage in living cells and in vitro. The growth of microtubules in vitro was analyzed with subpixel precision (Maurer et al., Current Biology, 2014, 24 (4), 372–384); however, to what extent these results could be applied for microtubules growing in vivo remains largely unknown. Particularly, the question is whether microtubule growth velocity in cells could be sufficiently approximated by a Gaussian distribution or its variability requires a more sophisticated description? Addressing this question, we used time-lapse microscopy and mathematical modeling, and we analyzed EB-3 comets forming on microtubules of cultured cells with subpixel precision. Parameters of comets (shape, form, and velocity) were used as topological characteristics of 3D voxel objects. Using regression analysis, we determined the real positions of the microtubule tips in time-lapse sequences. By exponential decay fitting of the restored comet intensity profile, we found that in vivo EB-3 rapidly exchanges on growing microtubule ends with a decoration time ∼ 2 s. We next developed the model showing that the best correlation between comet length and microtubule end growth velocity is at time intervals close to the decoration time. In the cells, EB comet length positively correlates with microtubule growth velocity in preceding time intervals, while demonstrating no correlation in subsequent time intervals. Correlation between comet length and instantaneous growth velocity of microtubules remains under nocodazole treatment when mean values of both parameters decrease. Our data show that the growth of microtubules in living cells is well-approximated by a constant velocity with large stochastic fluctuations.

Assessment of the Cherenkov camera alignment through Variance images for the ASTRI telescope

A peculiar aspect of Cherenkov telescopes is that they are designed to detect atmospheric light flashes on the time scale of nanoseconds, being almost blind to stellar sources. As a consequence, the pointing calibration of these instruments cannot be done in general exploiting the standard astrometry of the focal plane. In this paper we validate a procedure to overcome this problem for the case of the innovative ASTRI telescope, developed by INAF, exploiting sky images produced as an ancillary output by its novel Cherenkov camera. In fact, this instrument implements a statistical technique called "Variance method" (VAR) owning the potentiality to image the star field (angular resolution $\sim 11'$). We demonstrate here that VAR images can be exploited to assess the alignment of the Cherenkov camera with the optical axis of the telescope down to $\sim 1''$. To this end, we evaluate the position of the stars with sub-pixel precision thanks to a deep investigation of the convolution between the point spread function and the pixel distribution of the camera, resulting in a transformation matrix that we validated with simulations. After that, we considered the rotation of the field of view during long observing runs, obtaining light arcs that we exploited to investigate the alignment of the Cherenkov camera with high precision, in a procedure that we have already tested on real data. The strategy we have adopted, inherited from optical astronomy, has never been performed on Variance images from a Cherenkov telescope until now, and it can be crucial to optimize the scientific accuracy of the incoming MiniArray of ASTRI telescopes.

Research and improvement of feature detection algorithm based on FAST

At present, in the application of feature-based medical 3D reconstruction technology, there are still problems such as low matching accuracy of feature points in endoscope images and slow processing speed of image data. Therefore, the feature-based 3D reconstruction theory is of great research value and has great application value. This paper proposed a new feature detection method to improve the problems. This paper divides feature detection into two parts for further improvements: feature extraction and feature description. For feature extraction, the FAST algorithm shows a poor classification effect, so this paper adds the decision tree based on the C4.5 algorithm into the traditional FAST. The original data are divided into two decision trees to make the feature extraction performance more stable and feature point extraction more efficient. For the feature description part, the FREAK descriptor is used, combined with this paper's improved feature extraction algorithm. The feature points are extracted in scale space. The second-order function fitting is carried out according to the feature points' response scores in different scales. The scale-invariant descriptor of sub-pixel precision is obtained. The experimental results on the endoscope image show that the feature extraction method has a higher extraction accuracy and faster extraction speed. In addition, the feature description algorithm has higher calculation efficiency.

Method for acquiring accurate coordinates of the source point in electron backscatter diffraction

An electron backscatter diffraction device is an important accessory for a scanning electron microscope and can provide crystal structure orientation and phase content data through analysis of electron backscatter diffraction patterns. The acquisition of these data depends on pattern indexing, including interplanar angle calculation and crystal plane indexation. The coordinates of the source point are key points for interplanar angle calculation, and they vary with the movement of the incident beam. In this study, we first combined the grey gradient calculation with screen moving method to achieve accurate positioning of source point and obtained coordinates of source point with sub-pixel precision. The errors of three coordinates were 0.07%, 0.06% and 0.04%, respectively. By using this coordinate of source point to conduct interplanar angle calculation the maximum error was 0.53°, which was a good proof of the accuracy of source point positioning. Then we established the relationship between source point coordinates variation and incident beam movement. Coordinates can be given out based on the displacement of beam directly. And to illustrate the accuracy, interplanar angle calculation was performed and the maximum error was 0.81°. This means that the relationship between variation of source point coordinates and beam movement is highly accurate.

Point proposal network for reconstructing 3D particle endpoints with subpixel precision in liquid argon time projection chambers

Liquid Argon Time Projection Chambers are used as precision detectors in ongoing and upcoming neutrino experiments. In this paper the authors adapt machine learning techniques to better identify the beginning- and end-points of particle tracks in LarTPCs, offering significant improvement over traditional methods in reconstructing the true event.

Automatic Sub-Pixel Co-Registration of Remote Sensing Images Using Phase Correlation and Harris Detector

In this paper, we propose a new approach for sub-pixel co-registration based on Fourier phase correlation combined with the Harris detector. Due to the limitation of the standard phase correlation method to achieve only pixel-level accuracy, another approach is required to reach sub-pixel matching precision. We first applied the Harris corner detector to extract corners from both references and sensed images. Then, we identified their corresponding points using phase correlation between the image pairs. To achieve sub-pixel registration accuracy, two optimization algorithms were used. The effectiveness of the proposed method was tested with very high-resolution (VHR) remote sensing images, including Pleiades satellite images and aerial imagery. Compared with the speeded-up robust features (SURF)-based method, phase correlation with the Blackman window function produced 91% more matches with high reliability. Moreover, the results of the optimization analysis have revealed that Nelder–Mead algorithm performs better than the two-point step size gradient algorithm regarding localization accuracy and computation time. The proposed approach achieves better accuracy than 0.5 pixels and outperforms the speeded-up robust features (SURF)-based method. It can achieve sub-pixel accuracy in the presence of noise and produces large numbers of correct matching points.

Metrological assessment of multi‐sensor camera technology for spatially‐resolved ultra‐high‐speed imaging of transient high strain‐rate deformation processes

AbstractThe present work proposes a metrological route for capturing spatially‐resolved ultra‐high‐speed kinematic full‐field data from high strain‐rate experiments and multi‐sensor camera technology. This paper focuses, from an application point of view, on highly resolved rotating mirror cameras, such as the Cordin‐580. This camera allows 78 frames of 8 megapixels to be recorded at up to 4 million frames per second (fps). The optical apparatus induces distortions that need to be taken into consideration. Distortions are modelled with Zernike polynomials and recovered using Digital Image Correlation (DIC) with a tailored synthetic speckle pattern. Effective displacements can then be quantitatively obtained with subpixel precision. After an assessment of the calibrated camera performance, this methodology is used to record, at 480,000 fps, the fracture of a pre‐notched sample subjected to an inertial impact test. The kinematic fields obtained quantitatively captured the events occurring during the test, such as the compression wave and the induced Poisson effect, the Mode‐I crack initiation and the shear strain concentration at the notch tip. The achievement of a DIC displacement and strain random error of, respectively, 5 μm (0.15 pixels) and 2 mm m−1, combined with a high spatio‐temporal sampling, provides a promising way for quantitatively analysing very fast transient and heterogeneous phenomena.

A Depth Multiple Hierarchical StereoNet by Using Thermodynamic Color Guidance

Thermodynamics is devoted to the study of energy or materials, and few researchers have combined the laws of thermodynamics with stereoscopic vision. Thus, we propose Depth Multiple Hierarchical StereoNet (DMH-Net) by using thermodynamic color guidance, which present end-to-end deep architecture for real-time stereo matching, realize sub-pixel precision and produce high-quality refinement disparity on dataset benchmarks. At the same time, we use color thermodynamics to guide the image output of an accurate Hill disparity image. Our DMH-Net consists of four modules: the first module uses a deep spatial pooling method to extract multiple features that work on the initial left and right images. The second module use color guided to produce high-quality cost volume. We use the ahead of two modules to calculate depth disparity precision, which can segment coarse to fine to obtain background and edge of objects information. Thirdly module integrates the information about 3D cost volume and color channel to frame a 4D hierarchical cost volume. The fourth module is designed by depth hierarchical refinement to regress the final disparity. Finally, we achieve sub-pixel precision and real-time performance on benchmarks, which can produce high-quality thermodynamic disparity regression maps.

Reverse Procedure Detection of Space Target Streaks Based on Motion Parameter Estimation

This paper proposes a reverse procedure detection method for space target streaks based on motion parameter estimation. According to the phase shift characteristics of the spectrum, the interframe phase difference is used to obtain the displacement vectors of the target streak and image background, and then the morphological parameters of the target streak are estimated. In addition, a streak window is designed on the basis of the parameters, utilizing the local grayscale correlation in two image frames to carry out target searching and positioning with the interval of the target displacement vector. Images from different scenes are used to evaluate the adaptability of our method, and the experimental results suggest that the expected targets can be detected successfully with subpixel precision in all telescope operating modes. The proposed frequency-domain reverse procedure detection method avoids the problem of fake target confirmation caused by blind detection in traditional methods and overcomes the interference produced when the target streak passes through a bright star. Moreover, first target tracking and then positioning, which cannot be performed with time-domain methods, can be achieved. Our proposed method is attractive for multiple applications, such as space target cataloging, early warning and orbit prediction.

Enhancement of Contour Smoothness by Substitution of Interpolated Sub-Pixel Points for Edge Pixels

This study designed a sub-pixel precision edge detecting algorithm to enhance contour smoothness. First, the coordinate value of RGB pixel is projected on the space line of R=G=B to obtain gray image. Then, pixel edges are located using a Canny detector. Next, the edge width is thinned to a single pixel using a morphological thinning operation. Finally, sub-pixel-level smooth contours are extracted by interpolation. In this sub-pixel level contour extraction process, a Single-Pixel-Multi-Point interpolation method was developed to enhance edge smoothness and obtain high precision in edge estimation. This method divides edges in a $3\times 3$ pixels block into nine arrangement modes. According to the arrangement of the eight neighborhoods of a centered edge pixel, different locations of interpolated sub-pixel points are calculated by interpolation with Bezier curves. For symmetrically arranged linear edge pixels, this method can be used to determine the exact contour. Experimental results showed that the proposed algorithm can improve the smoothness of image edge contour. As the curvature of the edge increases, the maximum systematic error will increase. For the edge pixel centered in the $3\times 3$ pixels block and two pixels located at the corner of one side, the max systematic error is 0.5 pixel. For two edge pixels aligned in a single row or column with one located at a corner, the max systematic error is 0.25 pixel.

Hardware-Oriented Algorithm for High-Speed Laser Centerline Extraction Based on Hessian Matrix

Centerline extraction is the basic and key procedure in line-structured laser 3-D scanners. In this article, we propose a hardware-oriented algorithm for fast and accurate extraction of a laser centerline based on the Hessian matrix. The algorithm is divided into three low-coupling modules that can be processed in parallel—coarse positioning, linewidth estimation, and precise positioning module. In the coarse positioning module, a window slider is used to traverse all image pixels in the raster-scan mode for collecting local image features, based on which the potential region of interest (ROI) containing laser lines is detected. In the linewidth estimation module, the second-order moment features in the detected ROI are calculated to estimate the linewidth according to the local rectangular similarity characteristics of the laser line. In the precise positioning module, the optimal Gaussian template estimated by the linewidth is convolved with the ROI to obtain the Hessian matrix. Based on the Hessian matrix, the normal direction of the laser line is obtained, and the second-order Taylor expansion is performed in that direction to determine the subpixel position of the center point. Finally, non-maximum suppression is used to remove noise points and obtain the most reliable single-pixel centerline. The proposed algorithm is evaluated using thousands of sample images with different materials, exposure times, and laser line shapes, and it presents high robustness for subpixel precision extraction. By implementing the proposed algorithm on a field-programmable gate array (FPGA), a high-speed, high-precision laser centerline extraction system is achieved, which operates at 1350 frames/s for a 1-million-pixel video stream.

Image Processing and Edge Detection Techniques to Quantify Shock Wave Dynamics Experiments

Experimental studies of multiple shock wave interaction to study transition from regular to irregular reflection rely on the processing of a large amount of schlieren photographs. Here we present an automated algorithm to track individual shock fronts and triple points. First, correction to any optical distortions is applied to the photographs. Next, noise removal and edge detection algorithms are implemented to extract the pixel locations of the shocks. The edge detection algorithm takes advantage of the light intensity feature of the shock waves to distinguish shock fronts from background noise. This algorithm is also capable of separating entangled shock fronts through pattern recognition, which utilizes a discretization method to reduce complex shock geometries to localized linear patterns. Collectively, the algorithms can track shock wave characteristics to sub-pixel precision. This algorithm has been deployed for post processing of shock wave experiments to extract shock wave characteristics including positions and propagation velocities of shock fronts, vertical and horizontal velocities of Mach stems, and triple point trajectories during shock-shock interactions. Results show that the algorithm can process large volumes of data with minimal manual operations, making image processing more precise, efficient and productive while allowing for tracking of Mach stems and triple points.

Subpixel precision in registration of multimodal datasets

The motivation for our research is the huge demand for registration of multimodal datasets in restorers practice. With an increasing number of various screening modalities, each analysis built on the acquired dataset starts with the registration of images acquired from different scanners and with varying levels of mutual correspondence. There is currently no well-suited state of the art method for this task. There are many existing approaches, i.e. based on control points or mutual information, but they do not provide satisfying (subpixel) precision, thus the registration is very often realized manually in Adobe Photoshop or any similar tool. Another popular option is to use scanners able to produce registered datasets by design. During the last 10 years, datasets from these devices have extended available analytical techniques the most.In our research, we focus on solving the mentioned registration task. In [1] we concluded that the work with misregistered modalities is possible but limited. Now we present results of our experiments challenging these limits and conditions under which we can precisely register data from different modalities. The achieved results are promising and allow usage of more complex artificial neural networks (ANN) for dataset analysis e.g. [2]. We describe the construction of registration layers for estimation of shift, rotation and scale and a useful strategy and parametrization for ANN optimizer.

Vision Positioning method for Autonomous Precise Landing of UAV Based on Square Landing Mark

Rotor-craft is a kind of VTOL UAV and is widely used in multiple fields. Among relative researches, vision guided autonomous landing of rotor-craft has been a hot spot, where vision positioning is the most crucial. The core of the algorithm is to calculate the position and attitude information according to the change of the visual image of the same object at different time or frame. Based on the real-time self calibration technology of airborne monocular vision, this paper puts forward a method that takes the designed landing mark composed of black and white squares as the cooperative target, solving the relative pose of camera and cooperative target to carry out the landing positioning of UAV, and realizes the full-automatic sub-pixel precision linear edge detection to ensure the accuracy of visual positioning. A series of simulation experiments show that for 768 * 576 pixel image, when the camera is about 12m away from the target and the noise deviation reaches 3 pixels, the total time of edge extraction and calibration is 0.511s, and the position and attitude estimation are still satisfactory, which shows that the proposed algorithm can effectively realize the autonomous landing of UAV.

GVLD: A Fast and Accurate GPU-Based Variational Light-Field Disparity Estimation Approach

Disparity estimation is an essential task taking part in many light-field applications. Due to the complexity of algorithms and high dimensional property of light-field data, performing this task involves a significant computational effort and results in very long processing time on CPU. Graphics processing units (GPUs), which is capable of massively parallel processing, is a promising solution to cover the computation requirement and speed up the task. In this paper, we develop a GPU-accelerated approach for light-field disparity estimation using a variational computation framework (GVLD). Our algorithm combines the intrinsic sub-pixel precision of variational formulation and the effectiveness of weighted median filtering to produce a highly accurate solution. The proposed algorithm is fully parallelized and optimized for the implementation using the OpenCL framework. An intensive evaluation including a quantitative comparison to related works and a detailed analysis of the proposed approach’s performance is presented. Experimental results demonstrate our superior performance compared to state-of-the-art approaches. The proposed approach is 10+ times faster than other approaches running on a similar GPU platform and provides the most accurate solution among optimization-based approaches. Compared to the implementation running on CPU, our GPU-accelerated method achieves up to $365\times $ speed up.

Dynamic penetration of cellular solids: Experimental investigation using Hopkinson bar and computed tomography

Light-weight cellular solids, such as aluminium foams, are promising materials for use in ballistic impact mitigation applications for their high specific deformation energy absorption capabilities. In this study, three different types of aluminium alloy based in-house fabricated cellular materials were subjected to dynamic penetration testing using an in-house experimental setup to evaluate their deformation and microstructural response. A two-sided direct impact Hopkinson bar apparatus instrumented with two high-speed cameras observing the impact area and the penetrated surface of the specimens was used. An advanced wave separation technique was employed to process the strain-gauge signals recorded during the penetration. The images captured by one of the cameras were processed using an in-house Digital Image Correlation method with sub-pixel precision, that enabled the validation of the wave separation results of the strain-gauge signals. The second camera was used to observe the penetration into the tested specimens for the correct interpretation of the measured signals with respect to the derived mechanical and the microstructural properties at the different impact velocities. A differential X-ray computed tomography of the selected specimens was performed, which allowed for an advanced pre- and post-impact volumetric analysis. The results of the performed experiments and elaborate analysis of the measured experimental data are shown in this study.