Inverse Camera Research Articles

Compressed-SDR to HDR Video Reconstruction.

The new generation of organic light emitting diode display is designed to enable the high dynamic range (HDR), going beyond the standard dynamic range (SDR) supported by the traditional display devices. However, a large quantity of videos are still of SDR format. Further, most pre-existing videos are compressed at varying degrees for minimizing the storage and traffic flow demands. To enable movie-going experience on new generation devices, converting the compressed SDR videos to the HDR format (i.e., compressed-SDR to HDR conversion) is in great demands. The key challenge with this new problem is how to solve the intrinsic many-to-many mapping issue. However, without constraining the solution space or simply imitating the inverse camera imaging pipeline in stages, existing SDR-to-HDR methods can not formulate the HDR video generation process explicitly. Besides, they ignore the fact that videos are often compressed. To address these challenges, in this work we propose a novel imaging knowledge-inspired parallel networks (termed as KPNet) for compressed-SDR to HDR (CSDR-to-HDR) video reconstruction. KPNet has two key designs: Knowledge-Inspired Block (KIB) and Information Fusion Module (IFM). Concretely, mathematically formulated using some priors with compressed videos, our conversion from a CSDR-to-HDR video reconstruction is conceptually divided into four synergistic parts: reducing compression artifacts, recovering missing details, adjusting imaging parameters, and reducing image noise. We approximate this process by a compact KIB. To capture richer details, we learn HDR representations with a set of KIBs connected in parallel and fused with the IFM. Extensive evaluations show that our KPNet achieves superior performance over the state-of-the-art methods.

Fast and Accurate Illumination Estimation Using LDR Panoramic Images for Realistic Rendering.

A high dynamic range (HDR) image is commonly used to reveal stereo illumination, which is crucial for generating high-quality realistic rendering effects. Compared to the high-cost HDR imaging technique, low dynamic range (LDR) imaging provides a low-cost alternative and is preferable for interactive graphics applications. However, the limited LDR pixel bit depth significantly bothers accurate illumination estimation using LDR images. The conflict between the realism and promptness of illumination estimation for realistic rendering is yet to be resolved. In this paper, an efficient method that accurately infers illuminations of real-world scenes using LDR panoramic images is proposed. It estimates multiple lighting parameters, including locations, types and intensities of light sources. In our approach, a new algorithm that extracts illuminant characteristics during the exposure attenuation process is developed to locate light sources and outline their boundaries. To better predict realistic illuminations, a new deep learning model is designed to efficiently parse complex LDR panoramas and classify detected light sources. Finally, realistic illumination intensities are calculated by recovering the inverse camera response function and extending the dynamic range of pixel values based on previously estimated parameters of light sources. The reconstructed radiance map can be used to compute high-quality image-based lighting of virtual models. Experimental results demonstrate that the proposed method is capable of efficiently and accurately computing comprehensive illuminations using LDR images. Our method can be used to produce better realistic rendering results than existing approaches.

A novel triangular stereo model for 3D reconstruction of uniaxial MEMS-based structured light system

Benefiting from the small size and low driven voltage of the Micro Electro Mechanical Systems (MEMS) mirror, MEMS-based structured light (SL) 3D reconstruction has recently attracted much attention. In this paper, we propose a novel triangular stereo model for uniaxial MEMS-based 3D reconstruction. Existing SL methods based on MEMS mirrors often require the construction of a look-up table(LUT), but the proposed method only requires 34 parameters for accurate reconstruction. By analyzing the anatomy of the MEMS mirror, we put forward a transition function that allows the MEMS mirror to be regarded as an inverse camera model. Then, the triangular stereo principle can be directly employed to calculate 3D information. Furthermore, we present a calibration method to obtain the parameters of our model, since MEMS mirrors can only perform unidirectional fringe projection. Extensive experiments are performed to verify the feasibility and accuracy of our proposed mathematical model and calibration method.

Consensus-Based Optimization for 3D Human Pose Estimation in Camera Coordinates

3D human pose estimation is frequently seen as the task of estimating 3D poses relative to the root body joint. Alternatively, we propose a 3D human pose estimation method in camera coordinates, which allows effective combination of 2D annotated data and 3D poses and a straightforward multi-view generalization. To that end, we cast the problem as a view frustum space pose estimation, where absolute depth prediction and joint relative depth estimations are disentangled. Final 3D predictions are obtained in camera coordinates by the inverse camera projection. Based on this, we also present a consensus-based optimization algorithm for multi-view predictions from uncalibrated images, which requires a single monocular training procedure. Although our method is indirectly tied to the training camera intrinsics, it still converges for cameras with different intrinsic parameters, resulting in coherent estimations up to a scale factor. Our method improves the state of the art on well known 3D human pose datasets, reducing the prediction error by 32% in the most common benchmark. We also reported our results in absolute pose position error, achieving 80 mm for monocular estimations and 51 mm for multi-view, on average. Source code is available at https://github.com/dluvizon/3d-pose-consensus .

Calibration for Camera-Projector Pairs Using Spheres.

A newly developed calibration algorithm for camera-projector system using spheres is presented in this paper. Previous studies have exploited image conics of sphere to calibrate the camera, whereas this approach can be strengthened to apply in the projector and ultimately achieve the overall calibration for single or multiple pairs of camera-projector. Following the concept of taking the projector as an inverse camera, we retrieve the image conic of the sphere on the projector plane based on a pole-polar relationship we found. At least 3 image conics on the image plane of each device are required to calculate the intrinsic parameters of the device. The extrinsic parameters for all devices in the system are determined by the position of sphere centers in each coordinates frame of the device. Based on the isotropy of the calibration object (sphere), this work is mainly interested in accomplishing the entire calibration for multiple camera-projector systems in which sensors surround a central observation volume. Experiments are conducted on both synthetic and real datasets to evaluate its performance.

Multispectral Image Super-Resolution via RGB Image Fusion and Radiometric Calibration.

Multispectral imaging is of wide application for its capability in acquiring the spectral information of scenes. Due to hardware limitation, multispectral imaging device usually cannot achieve high-spatial resolution. To address the issue, this paper proposes a multispectral image super-resolution algorithm, referred as SRIF, by fusing the low-resolution multispectral image and the high-resolution (HR) RGB image. It deals with the general circumstance that image intensity is linear to scene radiance for multispectral imaging devices while is nonlinear and unknown for most RGB cameras. The SRIF algorithm first solves the inverse camera response function and a spectral sensitivity function of RGB camera, and establishes the linear relationship between multispectral and RGB images. Then the unknown HR multispectral image is efficiently reconstructed according to the linear image degradation models. Meanwhile, the edge structure of the reconstructed HR multispectral image is kept in accordance with that of the RGB image using a weighted total variation regularizer. The effectiveness of the SRIF algorithm is evaluated on both public datasets and our image set. Experimental results validate that the SRIF algorithm outperforms the state-of-the-arts in terms of both reconstruction accuracy and computational efficiency.

Illumination consistency based on single low dynamic range images

In this paper, a light consistency solution for generating high dynamic range (HDR) images based on a single low dynamic range image(LDR) is proposed, and the virtual object is rendered by illumination. The solution can reduce the time of image acquisition and processing, and solve the problems caused by the limitations of image acquisition equipment. The solution is divided into three stages: image preprocessing, high dynamic range image generation and virtual object relighting. Firstly, in the stage of image pretreatment, the wavelet noise reduction method based on a Gaussian mixture model is used to remove image noise and avoid image detail distortion. The inverse camera response function is utilized to linearize the image, the pixel brightness range is expanded based on the inverse tone mapping function, and the threshold segmentation method is combined with flooding Gaussian smoothing to calculate the highlight spread diagram to compensate for scene highlights lost during camera shooting. Then, the extended dynamic range image is interpolated linearly by using the specular expansion image to get the high dynamic range image. Based on the analysis and experimental simulation, compared with other methods, it is found that using a single low-dynamic-range image can greatly reduce the time of image acquisition and processing and reduce the limitations of image acquisition equipment, while maintaining good light fusion. Based on the simulation results, the efficiency of light consistency processing is improved.

Structured light system calibration with unidirectional fringe patterns

For the calibration of a structured light system, one type of highly accurate calibration method was developed by treating the projector as an inverse camera. This type of method typically creates pixel-to-pixel mapping between a camera point and a projector point using fringe patterns and resultant phase maps in orthogonal directions. However, requiring orthogonal patterns limits its feasibility of implementation on systems where the illumination device (e.g. grating projectors, interferometers, etc.) only supports fringe projection in one direction. This paper introduces a novel calibration method that only uses patterns in a single direction. We have theoretically proved that there exists one degree-of-freedom of redundancy in conventional calibration methods, making it possible to reduce the requirement of using orthogonal fringe patterns. Experiments show that under a measurement range of 200 mm(X) × 150 mm (Y) × 120 mm (Z), our measurement results are comparable to the results obtained using conventional calibration method. Evaluated by repeatedly measuring a sphere with 147.726 mm diameter, our measurement accuracy on average can be as high as 0.20 mm with a standard deviation of 0.12 mm.

Stereo Vision-Based High Dynamic Range Imaging Using Differently-Exposed Image Pair.

In this paper, a high dynamic range (HDR) imaging method based on the stereo vision system is presented. The proposed method uses differently exposed low dynamic range (LDR) images captured from a stereo camera. The stereo LDR images are first converted to initial stereo HDR images using the inverse camera response function estimated from the LDR images. However, due to the limited dynamic range of the stereo LDR camera, the radiance values in under/over-exposed regions of the initial main-view (MV) HDR image can be lost. To restore these radiance values, the proposed stereo matching and hole-filling algorithms are applied to the stereo HDR images. Specifically, the auxiliary-view (AV) HDR image is warped by using the estimated disparity between initial the stereo HDR images and then effective hole-filling is applied to the warped AV HDR image. To reconstruct the final MV HDR, the warped and hole-filled AV HDR image is fused with the initial MV HDR image using the weight map. The experimental results demonstrate objectively and subjectively that the proposed stereo HDR imaging method provides better performance compared to the conventional method.

Accurate projector calibration method by using an optical coaxial camera.

Digital light processing (DLP) projectors have been widely utilized to project digital structured-light patterns in 3D imaging systems. In order to obtain accurate 3D shape data, it is important to calibrate DLP projectors to obtain the internal parameters. The existing projector calibration methods have complicated procedures or low accuracy of the obtained parameters. This paper presents a novel method to accurately calibrate a DLP projector by using an optical coaxial camera. The optical coaxial geometry is realized by a plate beam splitter, so the DLP projector can be treated as a true inverse camera. A plate having discrete markers on the surface is used to calibrate the projector. The corresponding projector pixel coordinate of each marker on the plate is determined by projecting vertical and horizontal sinusoidal fringe patterns on the plate surface and calculating the absolute phase. The internal parameters of the DLP projector are obtained by the corresponding point pair between the projector pixel coordinate and the world coordinate of discrete markers. Experimental results show that the proposed method can accurately calibrate the internal parameters of a DLP projector.

Structured light system calibration method with optimal fringe angle

For structured light system calibration, one popular approach is to treat the projector as an inverse camera. This is usually performed by projecting horizontal and vertical sequences of patterns to establish one-to-one mapping between camera points and projector points. However, for a well-designed system, either horizontal or vertical fringe images are not sensitive to depth variation and thus yield inaccurate mapping. As a result, the calibration accuracy is jeopardized if a conventional calibration method is used. To address this limitation, this paper proposes a novel calibration method based on optimal fringe angle determination. Experiments demonstrate that our calibration approach can increase the measurement accuracy up to 38% compared to the conventional calibration method with a calibration volume of 300(H) mm×250(W) mm×500(D) mm.

Projector calibration for 3D scanning using virtual target images

In this work we propose a novel method to calibrate the projector. Calibration of the projector deals with the calculation of geometric parameters of the projector which are intrinsic and extrinsic parameters. An un-calibrated camera assists the process of finding the geometric parameters for projector. This method exploits the fact that a projector can be treated as inverse camera. The camera captures the scene and saves it as an image while the projector projects an image on the scene. Camera to projector transformation is used to make the projector able to see the calibration pattern. Then the projector is calibrated in a similar fashion like camera. Real data is used to evaluate the proposed method of projector calibration and good results are obtained.

Assembly of Non-Standard Electrical Components Using Stereoscopic Image Processing Techniques

Summary Although the assembly of standard, well defined, high volume electrical components into printed circuit boards is adequately handled using high speed automatic equipment, more irregular parts such as those found in Dower supply designs are not suited to such techniques. This paper outlines an approach to solving this problem using a robot to perform the insertion and a vision system to identify the precise location of the part being held in the robot's gripper. To enable the vision system to produce the required accuracy, the camera has to be located close to the component being held by the robot. The design of a low cost, miniature, lightweight image sensor to accomplish this objective is outlined together with its interfacing to a small computer. Using this system, together with a camera model based on the pinhole concept, both the forward and inverse object/image transformations may be defined. Although the missing depth information is not available directly from the inverse camera transformation, it may be calculated using stereoscopic (triangulation) type image processing techniques. A novel approach to generating the data for this process is outlined. Finally experimental tests comparing the estimated position of the leads of a large capacitor with those by direct measurement are presented. It is concluded that sufficient accuracy is available to complete insertion.