Sparse Camera Array Research Articles

Optimization of Sparse Camera Array Arrangement using Grid-based Method

—This study explores the application of multi-camera array technology in super-resolution (SR) imaging system. We propose an innovative method based on grid representation to optimize camera arrangements using sparse Gaussian processes and variational inference. This approach significantly enhances the image reconstruction quality of sparse camera arrays. Unlike conventional techniques that provide only qualitative conclusions, our grid-based method precisely optimizes camera positions and effectively simulates the impact of assembly errors. The optimized camera arrangement substantially improves the SR reconstruction quality of sparse array imaging systems, reducing contour jaggedness and high-frequency aliasing. Extensive simulations and experimental validations confirm the robustness and practical applicability of our method. The results demonstrate that the optimized camera arrangements can achieve high-resolution imaging with enhanced robustness against assembly errors, indicating their potential for various applications such as biological microscopy, remote sensing, and smartphone photography. This work presents a new and effective strategy for designing sparse camera array imaging systems, offering significant improvements in both image quality and system reliability.

Computational Integral Imaging Reconstruction Based on Generative Adversarial Network Super-Resolution

To improve acquisition efficiency and achieve super high-resolution reconstruction, a computational integral imaging reconstruction (CIIR) method based on the generative adversarial network (GAN) network is proposed. Firstly, a sparse camera array is used to generate an elemental image array of the 3D object. Then, the elemental image array is mapped to a low-resolution sparse view image. Finally, a lite GAN super-resolution network is presented to up-sample the low-resolution 3D images to high-resolution 3D images with realistic image quality. By removing batch normalization (BN) layers, reducing basic blocks, and adding intra-block operations, better image details and faster generation of super high-resolution images can be achieved. Experimental results demonstrate that the proposed method can effectively enhance the image quality, with the structural similarity (SSIM) reaching over 0.90, and can also reduce the training time by about 20%.

Computational Super-Resolution Imaging With a Sparse Rotational Camera Array

A sparse camera array utilizes several identical cameras for multi-frame super-resolution. However, the influence of camera layout on super-resolution performance remains unclear. In conventional camera arrays, all cameras share a similar observation model, resulting in redundancy during information collection. This study presents a novel super-resolution framework that actively controls cameras in a sparse camera array to rotate to certain degrees. Through a detailed analysis of the forward imaging model, it is shown that the rotation of cameras uses the rotational asymmetry property of pixel layouts and shape. The proposed camera layout method provides more diverse and informative spatial sampling than a conventional camera array with only translations between cameras. A physics-enabled deep neural network was introduced for multi-frame super-resolution. With the explicitly defined gradients and affine models as guidance, it can effectively decode additional information collected by the rotational camera array and overcome the large-scale spatial feature alignment problem. The experimental results on synthetic and real prototype datasets validated the proposed theory and solution. We believe our study can inspire innovations for future task-specific camera array system design.

Real-time dense-view imaging for three-dimensional light-field display based on image color calibration and self-supervised view synthesis.

Three-Dimensional (3D) light-field display has achieved promising improvement in recent years. However, since the dense-view images cannot be collected fast in real-world 3D scenes, the real-time 3D light-field display is still challenging to achieve in real scenes, especially at the high-resolution 3D display. Here, a real-time 3D light-field display method with dense-view is proposed based on image color correction and self-supervised optical flow estimation, and a high-quality and high frame rate of 3D light-field display can be realized simultaneously. A sparse camera array is firstly used to capture sparse-view images in the proposed method. To eliminate the color deviation of the sparse views, the imaging process of the camera is analyzed, and a practical multi-layer perception (MLP) network is proposed to perform color calibration. Given sparse views with consistent color, the optical flow can be estimated by a lightweight convolutional neural network (CNN) at high speed, which uses the input image pairs to learn the optical flow in a self-supervised manner. With inverse warp operation, dense-view images can be synthesized in the end. Quantitative and qualitative experiments are performed to evaluate the feasibility of the proposed method. Experimental results show that over 60 dense-view images at a resolution of 1024 × 512 can be generated with 11 input views at a frame rate over 20 fps, which is 4× faster than previous optical flow estimation methods PWC-Net and LiteFlowNet3. Finally, large viewing angles and high-quality 3D light-field display at 3840 × 2160 resolution can be achieved in real-time.

Super-multiview integral imaging scheme based on sparse camera array and CNN super-resolution.

In this paper, we propose a scheme based on sparse camera array and convolution neural network super-resolution for super-multiview integral imaging. In particular, the proposed scheme is adequate to not only the virtual-world three-dimensional scene with high performance and efficiency, but also the real-world 3D scene with higher availability than the traditional methods. In the proposed scheme, we first adopt the sparse camera array strategy to capture the sparse viewpoint images and use these images to synthesize the low-resolution elemental image array, then the convolution neural network super-resolution scheme is used to restore the high-resolution elemental image array from the low-resolution elemental image array for super-multiview integral image display. Experimental results verify the feasibility of the proposed scheme.

Hybrid camera array based calibration for computer-generated integral photography display.

Integral photography (IP) is one of the most promising 3D displays that can achieve a full parallax 3D display without glasses. There is a great need to render a correct, high-precision 3D image from an IP display. To achieve a correct 3D display, calibration is needed to correct optical misalignment and optical aberrations, while it is challenging to achieve correct mapping between a microlens array and matrix display. We propose an IP calibration method for a 3D autostereoscopic integral photography display based on a sparse camera array. Our method distinguishes itself from previous methods by estimating parameters for a dense correspondence map of an IP display with a relatively flexible setup and high precision in a reasonable time cost. We also propose a workflow to enable our method to handle both a visible and invisible microlens array and obtain a great outcome. One prototype is fabricated to evaluate the feasibility of the proposed method. Moreover, we evaluate our proposed method in geometry accuracy and image quality.

Optimized sampling for view interpolation in light fields using local dictionaries

We present an angular superresolution method for light fields captured with a sparse camera array. Our method uses local dictionaries extracted from a sampling mask for upsampling a sparse light field to a dense light field by applying compressed sensing reconstruction. We derive optimal sampling masks by minimizing the coherence for representative global dictionaries. The desired output perspectives and the number of available cameras can be arbitrarily specified. We show that our method yields qualitative improvements compared to previous techniques.

Dictionary-based light field acquisition using sparse camera array.

We propose a dictionary-based dense light field acquisition technique. This technique captures light field successfully from a sparse camera array with no mask or any other optical modifications on cameras. Light rays in wider field are captured by our system to achieve larger disparity and higher angular resolution. We also accelerate the reconstruction of light field significantly by a local sliding window which applies median filter only in disaster areas and acquire satisfactory quality. In our experiments, light field with 7x7 views at resolution of 384x512 is restored from 5 cameras with PSNR of 33.0192dB with a computing time of 1.85 hours on a consumer-grade desktop computer.

35.2: Generation method of orthoscopic elemental image array from a sparse camera array

AbstractWe propose a generation method of orthoscopic elemental image array from a sparse camera array. A parallax image array obtained by a sparse camera array is different perspectives of a real 3D scene, and it has all the information that the elemental image array needs. In‐depth analysis on the generation method and relationships between the sparse camera array and the elemental image array are presented. The experimental results demonstrate the correctness of the proposed method.

Method of generating orthoscopic elemental image array from sparse camera array

We propose a method of generating orthoscopic elemental image array from a sparse camera array. A parallax image array obtained by a sparse camera array provides different perspectives of a real threedimensional (3D) scene, and has all the information the elemental image array needs. In-depth analysis of the generation method and the relationships between the sparse camera array and the elemental image array are presented. The experimental results demonstrate the correctness of the proposed method.