Sub-pixel Method Research Articles

Checkerboard corner detection method based onneighborhood linear fitting.

To improve the calibration accuracy of a vision measurement system, a checkerboard corner detection method based on linear fitting of the checkerboard local contour is proposed. First, by binarization and morphological dilation of the checkerboard image, the coordinates of two adjacent vertices of adjacent dark squares are obtained; the midpoint of the two vertices is taken as the reference point; the reference dotted array is obtained; and the Zernike moment subpixel method is used to obtain the checkerboard contour data points in the neighborhood of each reference point. Finally, the contour points are classified according to the orientation based on the reference points; two intersecting lines are fitted; and the intersection of the two lines is exactly the corner point that we want to find. A camera calibration experiment was conducted on the same group of checkerboard images. The results show that the calibration results of the corner points obtained based on this method are highly consistent with the OpenCV library function method and the MATLAB Toolbox calibration method, and the reprojection error is smaller. At the same time, it is robust to changes in the light source brightness.

Using the band ratio approach, FULL-PIXEL (SAM and SFF) and SUB-PIXEL (MF and SID) methodologies to discover areas with alteration using the Aster sensor: a case study in north westen Iran - Sivardaghi Area

ASTER sensor data is among the most potent satellite data accessible for doing geological investigations, with images for the whole earth's surface. In order to test the capability of this sensor to detect places with geochemical alterations, photographs of Mount Seiver Daghi in the western Iranian province of Samal were utilized in this study. This region, which comprises of magmatic and volcanic terrain, is part of the Arsbaran territory and is covered by intrusive masses with alluvial and sedimentary deposits. To conduct this study, an ASTER measuring frame was utilized, which, after performing atmospheric corrections using the internal average relative reflectance (IARR) method of false color composite images and principal component analysis (PCA), was able to differentiate between different lithological units using the Band assignment method, full-pixel methods of spectral angle mapper (SAM) and base spectrum algorithm of spectral feature fitting (SFF) as well as sub-pixel methods of matched filtering. The study demonstrates that the approach of principal component analysis and false color composition is efficient for distinguishing sedimentary rock units from igneous rock units, and its application is suggested for the designated rock units. Due to the lack of spectral characteristics of feldspars and quartz in the short infrared wavelength range, the basic spectrum methods utilized in this work are incapable of identifying such minerals. It is not advised to use these algorithms to distinguish between various magmatic units.

Deep learning-based image super-resolution restoration for mobile infrared imaging system

Mobile infrared imaging system (MIIS) has important applications in the areas of the military affairs, medical diagnosis, and industrial detection. However, MIIS usually suffers the problems of the low resolution, low contrast, and noise interference. To solve these problems, we propose a deep learning-based image super-resolution restoration method for the MIIS. We design a multiscale feature distillation residual network to retain image features at various stages during the training process. The network uses dilated convolution to expand the receptive field and outputs the super-resolution (SR) infrared image with a sub-pixel method. Many infrared images that are captured by a MIIS are used for experiments. Experimental results indicate that the proposed method can realize the infrared image SR restoration and performs outstandingly against the four recently published SR methods both in visual quality and indicator performance. The proposed method is helpful for the practical applications of the MIIS.

Feedback attention network for cardiac magnetic resonance imaging super-resolution

Background and objectiveAtrial fibrillation (AF) is a common clinical arrhythmia with a high disability and mortality rate. Improving the resolution of atrial structure and its changes in patients with AF is very important for understanding and treating AF. MethodsAiming at the problems of previous deep learning-based image super-resolution (SR) reconstruction methods simply deepening the network, loss of upsampling information, and difficulty in the reconstruction of high-frequency information, we propose the Feedback Attention Network (FBAN) for cardiac magnetic resonance imaging (CMRI) super-resolution. The network comprises a preprocessing module, a multi-scale residual group module, an upsampling module, and a reconstruction module. The preprocessing module uses a convolutional layer to extract shallow features and dilate the number of channels of the feature map. The multi-scale residual group module adds a multi-channel network, a mixed attention mechanism, and a long and short skip connection to expand the receptive field of the feature map, improve the multiplexing of multi-scale features and strengthen the reconstruction of high-frequency information. The upsampling module adopts the sub-pixel method to upsample the feature map to the target image size. The reconstruction module consists of a convolutional layer, which is used to restore the number of channels of the feature map to the original number to obtain the reconstructed high-resolution (HR) image. ResultsFurthermore, the test results on the public dataset of CMRI show that the HR images reconstructed by the FBAN method not only have a good improvement in reconstructed edge and texture information but also have a good improvement in the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) objective evaluation indicators. ConclusionCompared with the local magnified image, the edge information of the FBAN method reconstructed image has been enhanced, more high-frequency information of the CMRI is restored, the texture details are less lost, and the reconstructed image is less blurry. Overall, the reconstructed image has a lighter feeling of smearing, and the visual experience is more apparent and sharper.

Measurement of Apparent Building Damage by Using a Single Image

In the proposed detection method, a single image was used for the rapid quantitative measurement of the damage area of a building. By using cross-ratio invariability between real and image coordinates, the damage area and the side lengths of the selected region were measured using the vanishing point, the vanishing line, and a line segment with a known length. Perspective transformation and image binarisation were performed on the selected frame to convert the length into area information. The damage area was obtained rapidly from the proportion of pixels in the damage region to pixels in the selected region. Corner detection and subpixel methods were combined to determine measurement errors because of the selection of measurement points. The experimental results from the example test revealed that quantitative area measurement of surface damage on buildings could be realised using the proposed method.

Super-resolution infrared imaging via multi-receptive field information distillation network

We propose a super-resolution (SR) infrared imaging method with a multi-receptive field information distillation network. We develop a parallel progressive feature purification model to optimize the feature extraction progress and retain feature in each dimension. We use the dilation convolution to enlarge the network's receptive field and keep the number of parameters steady. We reconstruct a SR infrared image by a sub-pixel method. A series of experiments are implemented. The imaging performance of the proposed method is validated by comparing with the results from classical interpolate Bicubic, and deep learning methods VDSR, SRResNet and IMDN. Experimental results suggest that the proposed method performs favorably against the four state-of-the-art SR algorithms in visual quality. The proposed system can realize high quality image reconstruction and 2-scale SR, and requires much less images numbers for training.

Active high-resolution imaging technology using single-photon echo signals to eliminate diffusion in MCP-based image sensors

Image sensors based on microchannel plates (MCPs) have important applications as array image sensors owing to their high sensitivity and wide field of view, and because no scanning is required. When MCP-based image sensors, such as ICCD and ICMOS, are used for laser-active imaging, severe diffusion phenomenon occurs due to high electronic gain, and this significantly affects the resulting image quality. This study uses single-photon echo signals to remove the diffusion phenomenon in MCP-based image sensors. A laser-active imaging system based on a MCP image sensor produces single-photon echo signals of the target. The single-photon echo signals are then processed using a weighting algorithm and sub-pixel method, which effectively remove the diffusion phenomenon and reconstruct a high-resolution image of the target. Therefore, when the MCP-based image sensor operates in a high-electronic-gain environment, this method improves the quality of laser-active imaging. The study shows that the method can improve the image resolution obtained by a MCP-based image sensor to 25.51 lp/mm in high-electronic-gain conditions.

Subpixel matching method for remote sensing image of ground features based on geographic information

AbstractIn order to solve the problem of large error of subpixel matching and poor filtering effect in traditional methods, a subpixel matching method based on geographical information is proposed. First, the image quality of the remote sensing image is enhanced by the image enhancement method based on light energy allocation. Then, the boundary geographic information is extracted by the improved thresholding segmentation algorithm based on histogram exponential convex hull for the enhanced remote sensing image of ground features. Based on the extracted geographic information, by matching the boundary image with the function measurement method, the center coordinates of the image block corresponding to the actual measurement map and the reference submap which achieve the best matching are obtained. According to the corresponding geometric transformation relationship between the measured image and the reference image, the subpixel matching of the measured remote sensing image and the reference image can be carried out under the least-square-error criterion. The experimental results show that the enhancement performance and noise filtering performance of the proposed method are better than those of the same type of method, the matching residual is very small, the matching accuracy is high, and the application value is significant.

Subpixel Localization of Isolated Edges and Streaks in Digital Images.

Many modern sensing systems rely on the accurate extraction of measurement data from digital images. The localization of edges and streaks in digital images is an important example of this type of measurement, with these techniques appearing in many image processing pipelines. Several approaches attempt to solve this problem at both the pixel level and subpixel level. While the subpixel methods are often necessary for applications requiring best-possible accuracy, they are often susceptible to noise, use iterative methods, or require pre-processing. This work investigates a unified framework for subpixel edge and streak localization using Zernike moments with ramp-based and wedge-based signal models. The method described here is found to outperform the current state-of-the-art for digital images with common signal-to-noise ratios. Performance is demonstrated on both synthetic and real images.

Precise Positioning of Linear Motor Mover Directly From the Phase Difference Analysis

This article presents a new visual detection method with high precision, robustness, and processing rate to estimate the linear motor displacements. Subpixel level location is achieved by using the phase-difference matrix and a structured aperiodic fence stripe placed on the mover. Based on Fourier shifted properties, this subpixel method builds a linear relation between the mover's spatial displacements and the direct frequency-domain phase difference (not the phase correlation matrix) to calculate the translational shift in a series of stripe images; thereby, precisely determining the mover location of linear motor. Unlike the traditional phase correlation algorithm, the proposed method converts 2-D stripe images into 1-D signals via the projection technique and then uses the least squares method to linearly fit the phase difference slope, thereby acquiring subpixel shifts with less time complexity. Due to the motion parameter that is directly estimated from the phase difference, this method is also insensitive to the gray-scale change with certain anti-interference capability, thereby making it suitable for measuring the moving position of the mover. Experimental results show that the proposed vision-based detection method is efficient and applicable for real-time accurate positioning of linear motor, and the measuring root mean square error is about 0.05 mm.

Development of video analytics with template matching methods for using camera as sensor and application to highway bridge structural health monitoring

Structural health monitoring (SHM) has been conducted by placing sensors on structures, so-called contact sensors, to measure displacements, strains, and accelerations of engineering structures. Contact sensing is accurate and effective, but suffers from some critical shortcomings, such as high cost, intensive labor, and traffic interruption. With the advancement and wide availability of digital video cameras, compounded with the development and enhancement of computer vision algorithms, vision-based sensing using a video camera as a sensor for SHM has become a viable alternative and a complementary method to remotely capture structural responses. Several computer vision algorithms are proposed, together with specific type of video camera, for capturing and processing the SHM video for the dynamic structural responses, e.g. displacements of selected targets. Up to date, there is a limited number of successful applications using video camera as a sensor in SHM practice. To facilitate the real-world applications of vision-based SHM, this paper presents the integrated methods, and a software tool for analyzing SHM videos using template matching algorithms along with a subpixel method that enhances the accuracy of the vision-based structural responses. The integrated approach is flexible and versatile for not only extracting displacements, strains, velocities, and accelerations, but also detecting damage of the desired multiple points or areas (templates) from videos. The methods and the software tool have been applied to a comprehensive analysis of the lab structural test, which was video-recorded by a third party without any involvement of the authors or any intension in using video camera as sensor. Good agreement for the lab test has been achieved for the structural responses (e.g. displacement and strain) and the damage detection recorded by the conventional sensors and extracted from the video by applying the integrated tool. The approach has been further validated on a highway bridge in the field, where a contact-based SHM system is permanently installed on the bridge and serves a good baseline reference for validating the proposed approach. The vision-based SHM results obtained for the highway bridge application show good correlation with the structural responses captured by conventional sensors and indicate that the developed analysis methods and software tool perform well for highway bridge test and structural health monitoring.

Area-Based Dense Image Matching with Subpixel Accuracy for Remote Sensing Applications: Practical Analysis and Comparative Study

Dense image matching is a crucial step in many image processing tasks. Subpixel accuracy and fractional measurement are commonly pursued, considering the image resolution and application requirement, especially in the field of remote sensing. In this study, we conducted a practical analysis and comparative study on area-based dense image matching with subpixel accuracy for remote sensing applications, with a specific focus on the subpixel capability and robustness. Twelve representative matching algorithms with two types of correlation-based similarity measures and seven types of subpixel methods were selected. The existing matching algorithms were compared and evaluated in a simulated experiment using synthetic image pairs with varying amounts of aliasing and two real applications of attitude jitter detection and disparity estimation. The experimental results indicate that there are two types of systematic errors: displacement-dependent errors, depending on the fractional values of displacement, and displacement-independent errors represented as unexpected wave artifacts in this study. In addition, the strengths and limitations of different matching algorithms on the robustness to these two types of systematic errors were investigated and discussed.

Aerial Video-Spectral Survey in the Search for Fragments of Separated Parts of Launch Vehicles on the Ground

The possibility of finding fragments of separated parts (SPs) of launch vehicles (LVs) on the ground on the basis of processing data from a video spectral (hyperspectral) aerial survey in the range of 0.4–1.0 μm is considered. The quality of special precorrection methods is estimated by comparison during thematic processing by spectral proximity measures, including the special delta-vector metric, modified Terebizh metric, and correlation factors, as well as with the use of the subpixel method.

Image Registration With Fourier-Based Image Correlation: A Comprehensive Review of Developments and Applications

Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation.

A Cost-Effective Single-Shot Structured Light System for 3D Shape Measurement

Single-shot three-dimensional (3D) shape measurement techniques have attracted extensive researches, as they are suitable for dynamic measurement. In this study, a cost-effective single-shot structured light system (CES-SLS) is proposed for 3D shape measurement by using a color camera and a normal projector. With the aid of two planar mirrors and an isosceles right-angle mirror, a single color camera functions as a binocular vision. A single-shot color random speckle pattern (CRSP) is used to encode the object by the normal projector. The dense corresponding points (DCP) are derived by the spatial-temporal correlation of RGB image subsets instead of the spatial correlation of gray image subsets in the conventional method. An inverse compositional Gaussian Newton (IC-GN) iteration method with the second-order shape function is then introduced to find the sub-pixel corresponding points (CP). Furthermore, a geometrical 3D recovery method is presented to calculate the 3D point by minimizing the re-projection error. The experimental results demonstrate the comparative advantages of the proposed CES-SLS against the system using a single-shot random speckle pattern (RSP) and the curve fitting sub-pixel method in the aspects of measurement accuracy and noise robustness. In addition, the re-projection error of each 3D point from the geometrical method is smaller than that of the conventional mid-point.

Application of aviation video-spectral survey to search for fragments of separating parts of launch vehicles on the ground

The possibility of detecting debris fragments of separating parts of launch vehicles on the ground on the basis of video spectral (hyperspectral) aerial survey data processing in the range of 0.4-1.0 µm is considered. Evaluation of the quality of special methods pre-correction when comparing, during thematic processing, measures the spectral vicinity – the special delta-vector metric, modified metric Terebizh, factor correlations, and sub-pixel method.

Illumination-Robust Subpixel Fourier-Based Image Correlation Methods Based on Phase Congruency

The Fourier-based image correlation technique has been widely concerned due to its accuracy, efficiency, and robustness to image contrast and brightness. Accordingly, a variety of subpixel methods have been proposed. However, the detailed subpixel-level influence of the complicated radiometric variations has yet to be investigated, and few corresponding improvements have been made. This paper presents a novel illumination-robust subpixel Fourier-based image correlation method based on phase congruency. Both the magnitude and orientation information of the phase congruency features are adopted to construct a structural image representation. The image representation is then embedded into the correlation scheme of the subpixel methods, either by linear phase estimation in the frequency domain or by kernel fitting in the spatial domain, achieving two improved subpixel methods. The proposed methods integrate the advantages of the structural image representation and the original correlation scheme, and make full use of both global and local phase information to achieve illumination-robust correlation. Experiments undertaken with both simulated and real radiometric differences were carried out with ground-truth subpixel shifts. The performances of the proposed methods and the other state-of-the-art subpixel Fourier-based correlation methods were evaluated and compared. The experimental results indicate that the proposed methods outperform the other methods in the presence of diverse radiometric variations, in both accuracy and robustness.

Real-time three-dimensional vibration monitoring of rotating shafts using constant-density sinusoidal fringe pattern as tri-axial sensor

Abstract The radial and axial vibration signals of a rotating machine are crucial information to understand the machine operation and to diagnose potential faults. Instead of using three single sensors to measure the horizontal, vertical and axial displacements of rotating shafts, a novel non-projection vision-based system was proposed to realize simultaneous measurement of the radial and axial displacements with high accuracy and good reliability using an tailored artificial constant density sinusoidal fringe pattern (CDSFP), which was pasted around the shaft surface and worked as a tri-axial (i.e. the horizontal, vertical and axial) displacement sensor. The measurement principle and setup of the proposed measurement system were well established. The horizontal displacement could be correctly obtained from the fringe period density changes of the CDSFP image sequence recorded by a high-speed camera. Simultaneously, the vertical displacement could be acquired by tracking the centerline of shaft whilst the axial displacement could be obtained by locating the peaks of the cross-correlation sequence of the fringe intensities. A sub-pixel method was employed to improve the displacement resolution of the developed system. The performance of the proposed system was demonstrated by the comparison of the experiments using eddy current sensors. It showed that the proposed method was an effective and accurate technique for real-time tri-axial vibration monitoring of rotating shaft. Experimental results verified the feasibility, effectiveness and good robustness of the proposed methodology, which demonstrated that the proposed system was capable of achieving accurate tri-axial vibration displacements of rotating shaft compared to the commercial eddy current sensor which could only measure one dimensional displacement at each measurement. Therefore, the vision-based tri-axial vibration monitoring system could be recommended for real engineering applications in condition monitoring of rotating shafts.

Robust vision sensor for multi-point displacement monitoring of bridges in the field

Computer vision sensors have great potential for accurate remote displacement monitoring in the field. This paper presents InnoVision, a video image processing technique developed by the authors to address a number of difficulties associated with the application of the vision sensors to monitoring structural displacement responses in the outdoor condition that are rarely comprehensively studied in literatures. First, limited lighting condition in the field presents a challenge to tracking low contrast features on the structural surface using intensity-based template matching algorithms. For tackling this challenge, a gradient based template matching algorithm is formulated. Second, to cost-effectively monitor structural displacements at multiple points using one camera, widely used interpolation subpixel methods are investigated and incorporated into InnoVision. Third, camera vibration in the field causes displacement measurement errors. A practical solution is proposed by applying the multi-point monitoring to track both the structure and a stationary reference point. The effect of the camera vibration can be canceled by subtracting the reference displacement from the structural displacements. Several laboratory and field tests are conducted to evaluate the InnoVision’s performance. One of the field tests is conducted in a challenging low lighting condition at night on a steel girder bridge to validate the robustness of InnoVision in comparison with two other vision sensing methods. Another field test is carried out on the Manhattan Bridge to demonstrate the efficacy of the proposed technique for canceling camera vibration and the capability of InnoVision to simultaneously monitor multiple points under the effect of camera vibration.

Super-resolution compressed sensing imaging algorithm based on sub-pixel shift

At present, some digital signal processing methods have attracted more and more attention in improving the resolution of images. Sub-pixel shift has been widely applied in improving the resolution of compressed sensing imaging system. The resolution of the compressed sensing imaging system is limited by pixel size of the modulation system. To overcome the resolution limitation of compressed sensing imaging system, a sub-pixel shift method is proposed to enhance the resolution of modulation information and achieve super-resolution images by compressed sensing imaging system. The principle of the proposed method is introduced and the proposed method is verified using numerical simulations. Experimental results revealed that the proposed method can effectively improve the resolution of compressed sensing imaging system and obtain super-resolution image information. Additionally, the signal to noise ratio of restoration results is positively related to the sample size.