Sub-pixel Shifts Research Articles

Four-frame pixel super-resolution method for lensless imaging systems

Lensless imaging systems have gained significant attention recently due to their advantages in terms of reduced size and weight compared to traditional lens-based systems. However, like other imaging methods, lensless imaging encounters challenges in resolving scenes with more details. In this article, we propose a novel four-frame super-resolution method specifically tailored for lensless imaging systems. Our approach shares similarities with previous lensless imaging systems, involving a sensor and a modulation device placed in front of the image sensor. We develop an explicit degradation downsampling model with sub-pixel shifts and provide the solution to corresponding inverse problem, may offering valuable guidance for other super-resolution imaging algorithms based on spatial displacements. By applying random lateral sub-pixel shifts, acquiring four low-resolution (LR) images, and fusing their spatial information, we achieve high-resolution (HR) sensor recordings, enabling super-resolution reconstruction of the imaging scene. Numerical simulations demonstrate approximately an improvement in spatial resolution compared to single-measurement methods. Furthermore, we evaluate the performance of our method across various lensless imaging systems utilizing different masks, validating its versatility and effectiveness in achieving higher resolution outcomes. Experimental results also support our proposed scheme's ability to achieve higher spatial resolution reconstruction in a real system.

Super-resolution lensless on-chip microscopy based on array illumination and sub-pixel shift search.

The resolution of a lensless on-chip microscopy system is constrained by the pixel size of image sensors. This Letter introduces a super-resolution on-chip microscopy system based on a compact array light source illumination and sub-pixel shift search. The system utilizes a closely spaced array light source composed by four RGB LED modules, sequentially illuminating the sample. A sub-pixel shift search algorithm is proposed, which determines the sub-pixel shift by comparing the frequency of captured low-resolution holograms. Leveraging this sub-pixel shift, a super-resolution reconstruction algorithm is introduced, building upon a multi-wavelength phase retrieval method, enabling the rapid super-resolution reconstruction of holograms with the region-of-interest. The system and algorithms presented herein obviate the need for a displacement control platform and calibration of the illumination angles of the light source, facilitating a super-resolution phase reconstruction under partially coherent illumination.

Noise tolerance of a sub-pixel shift method for upsampling diffraction patterns in coherent X-ray diffraction imaging

In coherent X-ray diffraction imaging, speckles on a coherent diffraction pattern must be sampled at intervals sufficiently finer than the Nyquist interval, which imposes an upper limit on the sample size. To overcome the size limitation, a sub-pixel shift method for upsampling coherent diffraction patterns was proposed. This paper reports on the evaluation of the noise tolerance of the upsampling algorithm by a simulation. The quality of the images reconstructed from the upsampled diffraction pattern and pattern recorded by a detector with an equivalent pixel size was comparable when the optimum number of upsampling iterations is adopted.

Super Resolution Dual-Energy Cone-Beam CT Imaging With Dual-Layer Flat-Panel Detector.

In flat-panel detector (FPD) based cone-beam computed tomography (CBCT) imaging, the native receptor array is usually binned into a smaller matrix size. By doing so, the signal readout speed could be increased by 4-9 times at the expense of a spatial resolution loss of 50%-67%. Clearly, such manipulation poses a key bottleneck in generating high spatial and high temporal resolution CBCT images at the same time. In addition, the conventional FPD is also difficult in generating dual-energy CBCT images. In this paper, we propose an innovative super resolution dual-energy CBCT imaging method, named as suRi, based on dual-layer FPD (DL-FPD) to overcome these aforementioned difficulties at once. With suRi, specifically, a 1D or 2D sub-pixel (half pixel in this study) shifted binning is applied instead of the conventionally aligned binning to double the spatial sampling rate during the dual-energy data acquisition. As a result, the suRi approach provides a new strategy to enable high spatial resolution CBCT imaging while at high readout speed. Moreover, a penalized likelihood material decomposition algorithm is developed to directly reconstruct the high resolution bases from these dual-energy CBCT projections containing sub-pixel shifts. Numerical and physical experiments are performed to validate this newly developed suRi method with phantoms and biological specimen. Results demonstrate that suRi can significantly improve the spatial resolution of the CBCT image. We believe this developed suRi method would greatly enhance the imaging performance of the DL-FPD based dual-energy CBCT systems in future.

True-color and super-resolution imaging using an optical axis shiftable liquid crystal lens.

This study explores the utilization of a liquid crystal lens with a shiftable axis for true-color and super-resolution imaging. By maintaining the optical power and shifting the axis of the liquid crystal lens, precise sub-pixel level shifts are applied to the images formed on the sensor, enabling the construction of true-color and super-resolution images. A comparative analysis with the traditional interpolation-based demosaicing method reveals that true-color imaging not only enhances clarity and effective pixel count, but also significantly reduces occurrences of false color, edge aliasing, and color moiré artifacts.

Повышение пространственного разрешения цифровых голограмм с помощью одномерного субпиксельного сканирования

The article considers the issue of increasing the spatial resolution during the registration of digital holograms. In practice, discretization is carried out by measuring signal samples using sensors with a certain finite aperture. The resolution is determined by the size of the aperture area over which the averaging takes place. The resolution enhancement method is based on the sampling equation for signals obtained by the subpixel shift method using generalized functions. A subpixel shift is carried out using a spatial shift by a value less than the element of resolution. Apertures of various shapes are used, for example, elliptical, diamond-shaped, hexagonal, but rectangular apertures are most commonly used. The discretization equation involves the use of a two-dimensional aperture function. Below is a way to increase the resolution when using a one-dimensional function. This can be done based on the structure of the holographic signal. In this case, you can use a subpixel shift in only one direction. Digital holography is distinguished by the fact that photodetector matrices which have a spatial resolution much lower than photographic media used in traditional holography methods are used to register the signal,. Therefore, digital holography uses optical schemes with small angles between interfering beams. However, certain restrictions are imposed on the shape of the objects under study. If the objects have a shape that is significantly different from a flat one, it is necessary to use traditional schemes. The article presents mathematical modeling of the resolution enhancement method based on the recovery of signals from a real hologram obtained in the usual way. The increase in resolution is achieved using a one-dimensional subpixel shift. The use of a one-dimensional subpixel shift makes it possible to significantly simplify the optical scheme of the holographic setup.

Towards Real-Time Hyperspectral Multi-Image Super-Resolution Reconstruction Applied to Histological Samples.

Hyperspectral Imaging (HSI) is increasingly adopted in medical applications for the usefulness of understanding the spectral signature of specific organic and non-organic elements. The acquisition of such images is a complex task, and the commercial sensors that can measure such images is scarce down to the point that some of them have limited spatial resolution in the bands of interest. This work proposes an approach to enhance the spatial resolution of hyperspectral histology samples using super-resolution. As the data volume associated to HSI has always been an inconvenience for the image processing in practical terms, this work proposes a relatively low computationally intensive algorithm. Using multiple images of the same scene taken in a controlled environment (hyperspectral microscopic system) with sub-pixel shifts between them, the proposed algorithm can effectively enhance the spatial resolution of the sensor while maintaining the spectral signature of the pixels, competing in performance with other state-of-the-art super-resolution techniques, and paving the way towards its use in real-time applications.

Super-Resolution and Wide-Field-of-View Imaging Based on Large-Angle Deflection with Risley Prisms.

A novel single camera combined with Risley prisms is proposed to achieve a super-resolution (SR) imaging and field-of-view extension (FOV) imaging method. We develop a mathematical model to consider the imaging aberrations caused by large-angle beam deflection and propose an SR reconstruction scheme that uses a beam backtracking method for image correction combined with a sub-pixel shift alignment technique. For the FOV extension, we provide a new scheme for the scanning position path of the Risley prisms and the number of image acquisitions, which improves the acquisition efficiency and reduces the complexity of image stitching. Simulation results show that the method can increase the image resolution to the diffraction limit of the optical system for imaging systems where the resolution is limited by the pixel size. Experimental results and analytical verification yield that the resolution of the image can be improved by a factor of 2.5, and the FOV extended by a factor of 3 at a reconstruction factor of 5. The FOV extension is in general agreement with the simulation results. Risley prisms can provide a more general, low-cost, and efficient method for SR reconstruction, FOV expansion, central concave imaging, and various scanning imaging.

Full-resolution, full-field-of-view, and high-quality fast Fourier single-pixel imaging.

Fourier single-pixel imaging (FSI) uses Fourier basis patterns for spatial light modulation to acquire the Fourier spectrum of the object image. The object image can be reconstructed via an inverse Fourier transform. However, the Fourier basis patterns are inherently gray scale, which results in the difficulty that the patterns can hardly be generated at a high speed by using a commonly used spatial light modulator-digital micromirrors device. To tackle this problem, fast FSI, which uses upsampled and dithered Fourier basis patterns to approximate the gray scale patterns, has been reported, but the achievable spatial resolution has to be sacrificed in the pattern upsampling process. Here we propose a method that can achieve not only full-resolution but also full-field-of-view and high-quality FSI. The key to the proposed method is to use a new, to the best of our knowledge, error diffusion dithering algorithm combined with two different scanning strategies to generate two sets of binarized Fourier basis patterns for spatial light modulation. As a result, two images with a sub-pixel shift from each other are reconstructed. It results in the final high-quality reconstruction by synthesizing the two images. We experimentally demonstrate the method can produce a high-quality 1024 × 768-pixel and full resolution image with a digital micromirror device with 1024 × 768 micromirrors.

A Novel Sub-Pixel-Shift-Based High-Resolution X-ray Flat Panel Detector

In this paper, we describe a novel sub-pixel shift (SPS)-based X-ray flat panel detector (FPD), which can achieve high resolution while maintaining a high SNR (signal-to-noise ratio). In the proposed architecture, an XY precision shift stage is applied to complete the sub-pixel shift process. In addition, image acquisition and high-resolution image composition are integrated in the FPD hardware. According to the relevant standards for detector image quality evaluation, we tested and evaluated some image quality indicators. The results show that the proposed FPD with SPS outperforms the original FPD without SPS technology. More specifically, the measured pixel size of the proposed FPD was reduced from 162 to 140 μm for 2 × 2 sub-pixel shift mode, and 132 μm for 4 × 4 sub-pixel shift mode, that is, the basic spatial detector resolution was improved by 13.6% for the simplest 2 × 2 sub-pixel shift mode, and by 18.5% for 4 × 4 sub-pixel shift mode. With this method, a lower-price FPD is elevated both in resolution and SNRn to meet imaging quality requirements.

Enhanced field of view multiplexing super-resolution incorporating geometric super-resolution by time multiplexing sub-pixeling.

In this paper, we show an enhancement of a super-resolution field of view multiplexing approach that, in addition to overcoming the diffraction related resolution limitation while sacrificing the field of view, also allows generating geometric super-resolution by creating sub-pixel shifts versus time. Thus, the proposed approach is both field of view as well as time multiplexing super-resolution, and it overcomes the resolution limits of both the diffraction and geometric limitation of spatial sampling caused by the stringent size of a camera's pixels.

Increasing the spatial resolution of signals in optical systems

Reconstruction of the signal in the intervals between discrete values is of great importance in solving the problem of spatial superresolution in optical microscopy and digital holography. The article deals with the issue of restoring high-resolution image elements from a certain number of raster images displaced by a sub-pixel shift. The numerical values of the image samples are obtained by spatial integration over some finite area of regular rasters. The spatial resolution is increased using an analytical expression for the spectrum of discrete signals obtained using the apparatus of generalized functions. Unlike ideal sampling, the spectrum of the function is supplemented by a multiplier, whose form depends on the type of aperture. To obtain high-resolution image elements, it is necessary to divide the Fourier spectrum of the sampled image by a factor depending on the selected aperture. The spectrum of the aperture is usually used, therefore, if the spectrum of the image obtained by averaging over a certain aperture is known, then the spectrum of the original image can also be obtained. Apertures of various shapes are used, for example, elliptical, diamond-shaped, hexagonal, but most often rectangular apertures. The simulation results are presented for a rectangular aperture but in the case of its substitution with, for example, a set of regular apertures in the form of a circle, the expression will be true for regular circular rasters. The analytical expression for the spectrum of the image obtained by averaging over a certain aperture can be used to reconstruct the spectrum of the original image. Having received the inverse Fourier transform from it, it is possible to obtain the original image. With an increase in the spatial resolution it becomes possible to carry out studies by methods of digital holography of volumetric diffuse objects while retaining the quality of analog holography (when recording in photographic media) and create optical superresolution systems based on optical microscopes.

Learning an epipolar shift compensation for light field image super-resolution

Light field imaging has drawn broad attention since the advent of practical light field capturing systems that facilitate a wide range of applications in computer vision. However, existing learning-based methods for improving the spatial resolution of light field images neglect the shifts in the sub-pixel domain that are widely used by super-resolution techniques, thus, fail in recovering rich high-frequency information. To fully exploit the shift information, our method attempts to learn an epipolar shift compensation for light field image super-resolution that allows the restored light field image to be angular coherent with the enhancement of spatial resolution. The proposed method first utilizes the rich surrounding views along some typical epipolar directions to explore the inter-view correlations. We then implement feature-level registration to capture accurate sub-pixel shifts of central view, which is constructed by the compensation module equipped with dynamic deformable convolution. Finally, the complementary information from different spatial directions is fused to provide high-frequency details for the target view. By taking each sub-aperture image as a central view, our method could be applied for light field images with any angular resolution. Extensive experiments on both synthetic and real scene datasets demonstrate the superiority of our method over the state-of-the-art qualitatively and quantitatively. Moreover, the proposed method shows good performance in preserving the inherent epipolar structures in light field images. Specifically, our LFESCN method outperforms the state-of-the-art method with about 0.7 dB (PSNR) on average.

High-throughput lensless whole slide imaging via continuous height-varying modulation of a tilted sensor.

We report a new, to the best of our knowledge, lensless microscopy configuration by integrating the concepts of transverse translational ptychography and defocus multi-height phase retrieval. In this approach, we place a tilted image sensor under the specimen for introducing linearly increasing phase modulation along one lateral direction. Similar to the operation of ptychography, we laterally translate the specimen and acquire the diffraction images for reconstruction. Since the axial distance between the specimen and the sensor varies at different lateral positions, laterally translating the specimen effectively introduces defocus multi-height measurements while eliminating axial scanning. Lateral translation further introduces sub-pixel shift for pixel super-resolution imaging and naturally expands the field of view for rapid whole slide imaging. We show that the equivalent height variation can be precisely estimated from the lateral shift of the specimen, thereby addressing the challenge of precise axial positioning in conventional multi-height phase retrieval. Using a sensor with 1.67 µm pixel size, our low-cost and field-portable prototype can resolve the 690 nm linewidth on the resolution target. We show that a whole slide image of a blood smear with a 120mm2 field of view can be acquired in 18 s. We also demonstrate accurate automatic white blood cell counting from the recovered image. The reported approach may provide a turnkey solution for addressing point-of-care and telemedicine-related challenges.

Spatiotemporal Projection‐Based Additive Manufacturing: A Data‐Driven Image Planning Method for Subpixel Shifting in a Split Second

Additive manufacturing (AM) is a digital manufacturing process that can directly convert a computer‐aided design model into a physical object in a layer‐by‐layer manner. Due to the additive and discrete nature of the digital manufacturing process, AM needs to find a trade‐off between process resolution and production efficiency. Traditional AM processes balance the resolution and efficiency by tuning the processes either in the temporal domain (e.g., higher speed in serial processes) or in the spatial domain (e.g., more tools in parallel processes). To improve the resolution without sacrificing efficiency, a data‐driven mask image planning method based on subpixel shifting in a split second by tuning the process in both temporal and spatial domains is presented. The method is based on the optimized pixel blending principle and a fast error diffusion‐based optimization model. Various simulation and experimental tests are carried out to verify the developed subpixel shifting method. The experimental results demonstrate the data‐driven‐based mask image calibration and planning techniques significantly improve the fabricated part quality without compromising the process efficiency. The presented spatiotemporal strategy may shed light for future research on the projection‐based AM processes.

Super resolution ghost imaging based on Fourier spectrum acquisition

In this paper, we propose a novel super-resolution ghost imaging based on Fourier spectrum acquisition. In the scheme, we use a set of sinusoidal speckle patterns with specific phase shift to illuminate the object, and then the corresponding total intensity of reflective light is measured by a bucket detector. By performing the inverse Fourier transformation on the bucket detection results, a set of low-resolution images with subpixel shift could be reconstructed. Subsequently, a super-resolution image could be obtained by combining all the reconstructed subpixel shifted images. The experimental results demonstrate that our proposed scheme can capture a super-resolution image with both direct and indirect line of sight to the object. The proposed scheme has the advantage of achieving super-resolution ghost imaging beyond the limitation of the pixel resolution of the illuminated speckle patterns and without the use of any high-precision motorized translation stages for shifting the speckle patterns.

An experimental sky-image-derived cloud validation dataset for Sentinel-2 and Landsat 8 satellites over NASA GSFC

Availability of a reliable cloud mask for optical satellite imagery is a prerequisite, when generating high-quality high-level geoinformation products. Creation of a reference (ground truth) cloud mask for moderate spatial resolution sensors, such as Operational Land Imager (OLI) aboard Landsat 8 and Multispectral Instrument (MSI) aboard Sentinel-2A/B satellites, is a challenging and time-consuming task. Existing reference datasets were mainly produced through photointerpretation of satellite images by an analyst, which can introduce subjectivity in detecting clouds. Therefore, other methods for generating cloud reference data shall be explored and evaluated that can complement existing datasets. In this paper, we document generation and provide the description of a new reference cloud dataset, named GSFC-Cloud, which is based on the extensive use of ground-based images of the sky. The dataset is collected over the same area, covers various cloud conditions, and is available for six Landsat 8 and twenty-eight Sentinel-2 scenes spanning the period of September 2017 to November 2018. The dataset is available in the vector format, so cloud masks at various spatial resolutions can be validated. We also describe a system to automate the process of ground-based data collection using low-cost off-the-shelf parts with the long-term objective to replicate this set-up in multiple locations around the world. We use the proposed dataset to validate and improve the Land Surface Reflectance Code (LaSRC) for cloud detection in Sentinel-2 imagery. We show that adding a parallax feature to estimate a subpixel shift between red and green bands with a phase correlation method can reduce overdetection of clouds and improve performance of LaSRC.

Precise Positioning of Linear Motor Mover Directly From the Phase Difference Analysis

This article presents a new visual detection method with high precision, robustness, and processing rate to estimate the linear motor displacements. Subpixel level location is achieved by using the phase-difference matrix and a structured aperiodic fence stripe placed on the mover. Based on Fourier shifted properties, this subpixel method builds a linear relation between the mover's spatial displacements and the direct frequency-domain phase difference (not the phase correlation matrix) to calculate the translational shift in a series of stripe images; thereby, precisely determining the mover location of linear motor. Unlike the traditional phase correlation algorithm, the proposed method converts 2-D stripe images into 1-D signals via the projection technique and then uses the least squares method to linearly fit the phase difference slope, thereby acquiring subpixel shifts with less time complexity. Due to the motion parameter that is directly estimated from the phase difference, this method is also insensitive to the gray-scale change with certain anti-interference capability, thereby making it suitable for measuring the moving position of the mover. Experimental results show that the proposed vision-based detection method is efficient and applicable for real-time accurate positioning of linear motor, and the measuring root mean square error is about 0.05 mm.

Subpixel Motion Detection Using a Commercial Video Camera

Performance metrics for detecting subpixel motion are calculated for a commercial video camera using image difference data processed with three Neyman–Pearson-based algorithms. High-signal-to-noise-ratio data are collected and analyzed for a thin black bar that slowly oscillates against a white background. The position and velocity of the bar are estimated using Fourier-based processing techniques. The probability of detecting subpixel motion as a function of false alarm rate, number of pixels tested, subpixel shift, and detection algorithm are calculated with Monte Carlo simulations using the experimental data. The results characterize the best performance curves for detecting subpixel motion for most commercial video cameras and targets.

Estimation of mutual subpixel shift between satellite images: software implementation

The special-purpose software implementation for estimating the subpixel shift between satellite images using advanced computer technology is described in this paper. The automatic calculation of the mutual subpixel shift between a pair of digital satellite images by correlation algorithm is performed. The proposed implementation was tested on a statistically representative number of satellite images and reached acceptable accuracy in determining their subpixel shift values.