Super-resolution Field Research Articles

Light Field Super-Resolution Using a Low-Rank Prior and Deep Convolutional Neural Networks.

Light field imaging has recently known a regain of interest due to the availability of practical light field capturing systems that offer a wide range of applications in the field of computer vision. However, capturing high-resolution light fields remains technologically challenging since the increase in angular resolution is often accompanied by a significant reduction in spatial resolution. This paper describes a learning-based spatial light field super-resolution method that allows the restoration of the entire light field with consistency across all angular views. The algorithm first uses optical flow to align the light field and then reduces its angular dimension using low-rank approximation. We then consider the linearly independent columns of the resulting low-rank model as an embedding, which is restored using a deep convolutional neural network (DCNN). The super-resolved embedding is then used to reconstruct the remaining views. The original disparities are restored using inverse warping where missing pixels are approximated using a novel light field inpainting algorithm. Experimental results show that the proposed method outperforms existing light field super-resolution algorithms, achieving PSNR gains of 0.23 dB over the second best performing method. The performance is shown to be further improved using iterative back-projection as a post-processing step.

Geometry-Consistent Light Field Super-Resolution via Graph-Based Regularization.

Light field cameras capture the 3D information in a scene with a single exposure. This special feature makes light field cameras very appealing for a variety of applications: from post-capture refocus to depth estimation and image-based rendering. However, light field cameras suffer by design from strong limitations in their spatial resolution. Off-the-shelf super-resolution algorithms are not ideal for light field data, as they do not consider its structure. On the other hand, the few super-resolution algorithms explicitly tailored for light field data exhibit significant limitations, such as the need to carry out a costly disparity estimation procedure with sub-pixel precision. We propose a new light field super-resolution algorithm meant to address these limitations. We use the complementary information in the different light field views to augment the spatial resolution of the whole light field at once. In particular, we show that coupling the multi-view approach with a graph-based regularizer, which enforces the light field geometric structure, permits to avoid the need of a precise and costly disparity estimation step. Extensive experiments show that the new algorithm compares favorably to the state-of-the-art methods for light field super-resolution, both in terms of visual quality and in terms of reconstruction error.

Cross-Scale Reference-Based Light Field Super-Resolution

Light fields suffer from a fundamental resolution tradeoff between the angular and the spatial domain. In this paper, we present a novel cross-scale light field super-resolution approach (up to $\text{8}\times$ resolution gap) to super-resolve low-resolution (LR) light field images that are arranged around a high-resolution (HR) reference image. To bridge the enormous resolution gap between the cross-scale inputs, we introduce an intermediate view denoted as single image super-resolution (SISR) image, i.e., super-resolving LR input via single image based super-resolution scheme, which owns identical resolution as HR image yet lacks high-frequency details that SISR scheme cannot recover under such significant resolution gap. By treating the intermediate SISR image as the low-frequency part of our desired HR image, the remaining issue of recovering high-frequency components can be effectively solved by the proposed high-frequency compensation super-resolution (HCSR) method. Essentially, HCSR works by transferring as much as possible the high-frequency details from the HR reference view to the LR light field image views. Moreover, to solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, we compute multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained. The superiority of our proposed HCSR method is validated on extensive datasets including synthetic, real-world and challenging microscope scenes.

Light field super resolution through controlled micro-shifts of light field sensor

Light field cameras enable new capabilities, such as post-capture refocusing and aperture control, through capturing directional and spatial distribution of light rays in space. Micro-lens array based light field camera design is often preferred due to its light transmission efficiency, cost-effectiveness and compactness. One drawback of the micro-lens array based light field cameras is low spatial resolution due to the fact that a single sensor is shared to capture both spatial and angular information. To address the low spatial resolution issue, we present a light field imaging approach, where multiple light fields are captured and fused to improve the spatial resolution. For each capture, the light field sensor is shifted by a pre-determined fraction of a micro-lens size using an xy translation stage for optimal performance.

Superresolution Focusing Using Metasurface with Circularly Arranged Nanoantennas

Circular lens composed of annularly arranged metal nanoantennas is proposed to achieve far field superresolution focusing. Light illuminating on the nanoantennas’ layer approximately acquires paraboloidal phase profile and then focuses into a spot. Lens constructed by monolayer nanoantennas achieve focusing with FWHM (full width at half maximum) of 924 nm and a focal length of 3385 nm, breaking the diffraction limit. Moreover, tri-layered lens realizes subwavelength focusing with FWHM of 320 nm (about 0.49λ) and the field intensity of focus is optimized to 0.97 a.u. (arbitrary unit). Our proposal shows advances in focusing performance compared with previous work, making it promising in many applications, such as nanolithography, dense storage, and integrated optics.

PatchTable

This paper presents a data structure that reduces approximate nearest neighbor query times for image patches in large datasets. Previous work in texture synthesis has demonstrated real-time synthesis from small exemplar textures. However, high performance has proved elusive for modern patch-based optimization techniques which frequently use many exemplar images in the tens of megapixels or above. Our new algorithm, PatchTable, offloads as much of the computation as possible to a pre-computation stage that takes modest time, so patch queries can be as efficient as possible. There are three key insights behind our algorithm: (1) a lookup table similar to locality sensitive hashing can be precomputed, and used to seed sufficiently good initial patch correspondences during querying, (2) missing entries in the table can be filled during pre-computation with our fast Voronoi transform, and (3) the initially seeded correspondences can be improved with a precomputed k-nearest neighbors mapping. We show experimentally that this accelerates the patch query operation by up to 9× over k-coherence, up to 12× over TreeCANN, and up to 200× over PatchMatch. Our fast algorithm allows us to explore efficient and practical imaging and computational photography applications. We show results for artistic video stylization, light field super-resolution, and multi-image editing.

Wide Field Super-Resolution Surface Imaging through Plasmonic Structured Illumination Microscopy

We experimentally demonstrate a wide field surface plasmon (SP) assisted super-resolution imaging technique, plasmonic structured illumination microscopy (PSIM), by combining tunable SP interference (SPI) with structured illumination microscopy (SIM). By replacing the laser interference fringes in conventional SIM with SPI patterns, PSIM exhibits greatly enhanced resolving power thanks to the unique properties of SP waves. This PSIM technique is a wide field, surface super-resolution imaging technique with potential applications in the field of high-speed biomedical imaging.

Analysis of Near-Field Intensity Distributions of High-Compression Talbot Array Illuminators Using Rigorous Diffraction Theory

The technology nowadays employed in the field of binary optics allows a relatively easy fabrication of grating structures down to the wavelength region. The analysis of light propagation through such grating structures requires the use of rigorous diffraction theories such as the rigorous coupled-wave analysis (RCWA), in order to take proper account of polarization effects in the electric and magnetic field and the thickness of the grating. In this paper, we examine the near-field intensity distributions and associated illumination efficiencies of Talbot array illuminators (TAIL#x0027;s) with compression ratios of 8 and 16 using RCWA. The high illumination efficiency observed close to the grating surface is expected to bring about new applications of TAIL#x0027;s in the field of super-resolution and near-field optics.