Light Field Approach Research Articles

Projection Mapping with a Brightly Lit Surrounding Using a Mixed Light Field Approach.

Projection mapping (PM) exhibits suboptimal performance in well-lit environments because of the interference caused by ambient light. This interference degrades the contrast of the projected images. Consequently, conventional methodologies restrict the application of PM to dimly lit settings, leading to an unnatural visual experience, as only the PM target is prominently illuminated. To overcome these limitations, we introduce an innovative approach that leverages a mixed light field, blending traditional PM with ray-controllable ambient lighting. This methodological combination, despite its simplicity, is effective because it ensures that the projector exclusively illuminates the PM target, preserving the optimal contrast. Precise control of ambient light rays is essential to prevent them from illuminating the PM target while adequately illuminating the surrounding environment. Furthermore, we propose the integration of a kaleidoscopic array with integral photography to generate dense light fields for ray-controllable ambient lighting. Additionally, we present an efficient binary-search-based calibration method tailored to this intricate optical system. Our optical simulations and the developed system collectively validate the effectiveness of our approach. Our results show that PM targets and ordinary objects coexist naturally in environments that are brightly lit as a result of our method, enhancing the overall visual experience.

Computer-generated photorealistic hologram using ray-wavefront conversion based on the additive compressive light field approach

The conventional computer-generated hologram reconstructing photorealistic three-dimensional (3D) images based on ray-wavefront conversion has the disadvantage of spatio-angular resolution trade-off. In this Letter, we propose for the first time, to the best of our knowledge, a computer-generated photorealistic hologram without spatio-angular resolution trade-off based on the additive compressive light field (CLF) approach. The original light field is compressed into multiple layer images through numerical optimization based on the additive light field principle. Then, by independently calculating the wave propagation from each layer image to the hologram plane and adding them together, a CLF hologram is generated. Since the CLF information is presented through a holographic method, the advantage of high resolution in CLF is preserved while the limitation of the number of physically stacked layers (such as liquid crystal displays) is removed, leading to higher quality, larger depth of field, and higher brightness compared with a conventional CLF display. The proposed method is verified with a photorealistic optical experiment.

Flexibly Connectable Light Field System For Free View Exploration

Conventional captured-image-based virtual reality (VR) systems have three degrees of freedom (DoFs), where only rotational user motion is tracked for view rendering. This is a major cause of the reduced sense of reality. To increase user immersion levels akin to the real world, 3-DoF+ VR systems that support not only rotational but also translational view changes have been proposed. The light-field (LF) approach is suitable for this type of 3-DoF+ VR because it renders a view from a free view position by simply combining lights. Many previous systems have limited scalability because they assume a single LF for the acquisition and representation of light. One recent work connects multiple LFs at a physical intersection to increase the scalability. However, these fixed connection points limit the renderable view range and the layout of multiple LFs. Furthermore, in conventional single- or multiple-LF systems, the representable ranges of the view positions are highly dependent on the input field-of-view (FOV) of the camera used. In order to realize a wide view exploration range, the above-mentioned limitations must be overcome. This paper proposes a flexible connection scheme for multiple-LF systems taking advantage of the constant radiance of rays in LF theory. The proposed flexibly connectable LF system is able to widen the range of the renderable view position under reasonable conditions of the camera FOV. A light-field unit (LFU) which uses the proposed flexible connection is implemented. The LFU has a square-shaped structure and is thus easily stackable. This offers the advantage of dramatically expanding the scope of view explorations. The proposed LFU achieves 3-DoF+ VR with good quality as well as high scalability. Its cost-performance outcome is also better compared to those in previous works.

Comparison between the plenoptic sensor and the light field camera in restoring images through turbulence

Similar to the “lucky imaging” technique that selects the best local features over time, spatial redundancy allows for the localization of turbulence induced image distortions and selection of the best features that are least distorted by turbulence. A new technique to restore turbulence degraded images is proposed based on imaging with spatial redundancies. Two imaging frameworks that are candidates for implementation of the technique are the plenoptic sensor and the light field camera, which collect multiple depictions of the target through sub-aperture imaging. Preliminary studies have demonstrated the effectiveness of either device in imaging through turbulence. However, as visual distortions vary significantly from weak to strong turbulence conditions, it is unclear when and how a light field approach should be applied to enhance target recognition over distorted media. We present an in-depth study on the fundamental differences between the two devices with regards to turbulence distortion, as well as their image restoration mechanisms. Our analysis combined with proof-of-concept experiments show that the turbulence resilience of light field imaging techniques depends strongly on the mechanism of mapping the light field. Such universal finding serves as guidance for imaging and object recognition with light field approaches.

Surface Light Field Compression Using a Point Cloud Codec

Light field (LF) representations aim to provide photo-realistic, free-viewpoint viewing experiences. However, the most popular LF representations are the images from multiple views. Multi-view image-based representations generally need to restrict the range or degrees of freedom of the viewing experience to what can be interpolated in the image domain, essentially because they lack explicit geometry information. We present a new surface LF (SLF) representation based on explicit geometry and a method for SLF compression. First, we map the multi-view images of a scene onto a 3-D geometric point cloud. The color of each point in the point cloud is a function of viewing direction known as a view map. We represent each view map efficiently in a B-Spline wavelet basis. This representation is capable of modeling diverse surface materials and complex lighting conditions in a highly scalable and adaptive manner. The coefficients of the B-Spline wavelet representation are then compressed spatially. To increase the spatial correlation and, thus, improve compression efficiency, we introduce a smoothing term to make the coefficients more similar across the 3-D space. We compress the coefficients spatially using existing point cloud compression methods. On the decoder side, the scene is rendered efficiently from any viewing direction by reconstructing the view map at each point. In contrast to multi-view image-based LF approaches, our method supports photo-realistic rendering of real-world scenes from arbitrary viewpoints, i.e., with an unlimited six degrees of freedom. In terms of rate and distortion, experimental results show that our method achieves superior performance with lighter decoder complexity compared with a reference image-plus-geometry compression scheme, indicating its potential in practical virtual and augmented reality applications.

Characterization of the Light Field and Apparent Optical Properties in the Ocean Euphotic Layer Based on Hyperspectral Measurements of Irradiance Quartet

Although the light fields and apparent optical properties (AOPs) within the ocean euphotic layer have been studied for many decades through extensive measurements and theoretical modeling, there is virtually a lack of simultaneous high spectral resolution measurements of plane and scalar downwelling and upwelling irradiances (the so-called irradiance quartet). We describe a unique dataset of hyperspectral irradiance quartet, which was acquired under a broad range of environmental conditions within the water column from the near-surface depths to about 80 m in the Gulf of California. This dataset enabled the characterization of a comprehensive suite of AOPs for realistic non-uniform vertical distributions of seawater inherent optical properties (IOPs) and chlorophyll-a concentration (Chl) in the common presence of inelastic radiative processes within the water column, in particular Raman scattering by water molecules and chlorophyll-a fluorescence. In the blue and green spectral regions, the vertical patterns of AOPs are driven primarily by IOPs of seawater with weak or no discernible effects of inelastic processes. In the red, the light field and AOPs are strongly affected or totally dominated by inelastic processes of Raman scattering by water molecules, and additionally by chlorophyll-a fluorescence within the fluorescence emission band. The strongest effects occur in the chlorophyll-a fluorescence band within the chlorophyll-a maximum layer, where the average cosines of the light field approach the values of uniform light field, irradiance reflectance is exceptionally high approaching 1, and the diffuse attenuation coefficients for various irradiances are exceptionally low, including the negative values for the attenuation of upwelling plane and scalar irradiances. We established the empirical relationships describing the vertical patterns of some AOPs in the red spectral region as well as the relationships between some AOPs which can be useful in common experimental situations when only the downwelling plane irradiance measurements are available. We also demonstrated the applicability of irradiance quartet data in conjunction with Gershun’s equation for estimating the absorption coefficient of seawater in the blue-green spectral region, in which the effects of inelastic processes are weak or negligible.

3D reconstruction by a combined structure tensor and Hough transform light field approach

Abstract Disparity estimation using the structure tensor is a local approach to determine orientation in Epipolar Plane Images. A global extension would lead to more precise and robust estimations. In this work, a novel algorithm for 3D reconstruction from linear light fields is proposed. This method uses a modified version of the Progressive Probabilistic Hough Transform to extract orientations from Epipolar Plane Images, allowing to achieve high quality disparity maps. To this aim, the structure tensor estimates are used to speed up computation and improve the disparity estimation near occlusion boundaries. The new algorithm is evaluated on both synthetic and real light field datasets, and compared with classical local disparity estimation techniques based on the structure tensor.

DEM RECONSTRUCTION USING LIGHT FIELD AND BIDIRECTIONAL REFLECTANCE FUNCTION FROM MULTI-VIEW HIGH RESOLUTION SPATIAL IMAGES

Abstract. This paper presents a method for dense DSM reconstruction from high resolution, mono sensor, passive imagery, spatial panchromatic image sequence. The interest of our approach is four-fold. Firstly, we extend the core of light field approaches using an explicit BRDF model from the Image Synthesis community which is more realistic than the Lambertian model. The chosen model is the Cook-Torrance BRDF which enables us to model rough surfaces with specular effects using specific material parameters. Secondly, we extend light field approaches for non-pinhole sensors and non-rectilinear motion by using a proper geometric transformation on the image sequence. Thirdly, we produce a 3D volume cost embodying all the tested possible heights and filter it using simple methods such as Volume Cost Filtering or variational optimal methods. We have tested our method on a Pleiades image sequence on various locations with dense urban buildings and report encouraging results with respect to classic multi-label methods such as MIC-MAC, or more recent pipelines such as S2P. Last but not least, our method also produces maps of material parameters on the estimated points, allowing us to simplify building classification or road extraction.

DEM RECONSTRUCTION USING LIGHT FIELD AND BIDIRECTIONAL REFLECTANCE FUNCTION FROM MULTI-VIEW HIGH RESOLUTION SPATIAL IMAGES

This paper presents a method for dense DSM reconstruction from high resolution, mono sensor, passive imagery, spatial panchromatic image sequence. The interest of our approach is four-fold. Firstly, we extend the core of light field approaches using an explicit BRDF model from the Image Synthesis community which is more realistic than the Lambertian model. The chosen model is the Cook-Torrance BRDF which enables us to model rough surfaces with specular effects using specific material parameters. Secondly, we extend light field approaches for non-pinhole sensors and non-rectilinear motion by using a proper geometric transformation on the image sequence. Thirdly, we produce a 3D volume cost embodying all the tested possible heights and filter it using simple methods such as Volume Cost Filtering or variational optimal methods. We have tested our method on a Pleiades image sequence on various locations with dense urban buildings and report encouraging results with respect to classic multi-label methods such as MIC-MAC, or more recent pipelines such as S2P. Last but not least, our method also produces maps of material parameters on the estimated points, allowing us to simplify building classification or road extraction.

Imaging through strong turbulence with a light field approach

Under strong turbulence conditions, object's images can be severely distorted and become unrecognizable throughout the observing time. Conventional image restoring algorithms do not perform effectively in these circumstances due to the loss of good references on the object. We propose the use a plenoptic sensor as a light field camera to map a conventional camera image onto a cell image array in the image's sub-angular spaces. Accordingly, each cell image on the plenoptic sensor is equivalent to the image acquired by a sub-aperture of the imaging lens. The wavefront distortion over the lens aperture can be analyzed by comparing cell images in the plenoptic sensor. By using a modified "Laplacian" metric, we can identify a good cell image in a plenoptic image sequence. The good cell image corresponds with the time and sub-aperture area on the imaging lens where wavefront distortion becomes relatively and momentarily "flat". As a result, it will reveal the fundamental truths of the object that would be severely distorted on normal cameras. In this paper, we will introduce the underlying physics principles and mechanisms of our approach and experimentally demonstrate its effectiveness under strong turbulence conditions. In application, our approach can be used to provide a good reference for conventional image restoring approaches under strong turbulence conditions. This approach can also be used as an independent device to perform object recognition tasks through severe turbulence distortions.

Analysis of image distortion based on light ray field by multi-view and horizontal parallax only integral imaging display

Three-dimensional image distortion caused by mismatch between autostereoscopic displays and contents is analyzed. For a given three-dimensional object scene, the original light ray field in the contents and deformed one by the autostereoscopic displays are calculated. From the deformation of the light ray field, the distortion of the resultant three-dimensional image is finally deduced. The light ray field approach enables generalized distortion analysis across different autostereoscopic display techniques. The analysis result is verified experimentally when multi-view contents are applied to a multi-view display of non-matching parameters and a horizontal parallax only integral imaging display.

Real-time view synthesis from a sparse set of views

It is known that the pure light field approach for view synthesis relies on a large number of image samples to produce anti-aliased renderings. Otherwise, the insufficiency of image sampling needs to be compensated for by geometry sampling. Currently, geometry estimation is done either offline or using dedicated hardware. Our solution to this dilemma is based on three key ideas: a formal analysis of the equivalency between light field rendering and plane-based warping, multi focus imaging in a multi camera system by plane sweeping, and the fusion of the multi focus image using multi view stereo. The essence of our method is to perform necessary depth estimation up to the level required by the minimal joint image-geometry sampling rate using off-the-shelf graphics hardware. As a result, real-time anti-aliased light field rendering is achieved even if the image samples are insufficient.

Image-Based Photorealistic 3D Reconstruction Using Hexagonal Representation

An algorithm for reconstructing photorealistic 3D model from multiple-view images is proposed. The idea is based on the surface light field approach. In the algorithm, a geometric model is reconstructed as a visual hull using an image-based multi-pass algorithm we have developed, then the hull is represented as a quadrilateral-meshed surface. Next, colors of input images are assigned onto each vertex according to viewing directions using a new data structure we have developed. The structure is a hexagonal tessellation based on expansion and replacement of a buckyball. Finally, the hexagonal tessellation is represented as a hexagonal image whose pixels represent colors of corresponding input images. Experimental results for real objects show that 3D data can be successfully generated automatically in a short time and that photorealistic data can be viewed from arbitrary viewpoints even for objects with reflective or translucent surfaces.

Critical Phenomenon of a Consistent q-Deformed Squeezed State

Within a self-consistent framework of the q-deformed Heisenberg algebra and its equivalent framework of q-deformed boson commutation relations, which relate to the under-cutting phenomenon of Heisenberg's minimal uncertainty relation, special q-deformed squeezed states are constructed. Besides the similar local maximum squeezing as the one in the undeformed case, new strong squeezing appears when the amplitude of the related coherent state increases to large values. A critical phenomenon appears at a large value of the amplitude: the variance of one component of the quadrature of the light field approaches zero, but the variance of the corresponding conjugate quantity remains finite, which is a surprising deviation from Heisenberg's uncertainty relation. The qualitative character exposed by this q-squeezed state may provide some evidence about q-deformed effects in current experiments.