Super-Resolution for Near-Eye Light Field Display in Fourier Space

We develop a Mellin space approach to boundary correlation functions in anti-de Sitter (AdS) and de Sitter (dS) spaces. Using the Mellin-Barnes representation of correlators in Fourier space, we show that the analytic continuation between AdSd+1 and dSd+1 is encoded in a collection of simple relative phases. This allows us to determine the late-time tree-level three-point correlators of spinning fields in dSd+1 from known results for Witten diagrams in AdSd+1 by multiplication with a simple trigonometric factor. At four point level, we show that Conformal symmetry fixes exchange four-point functions both in AdSd+1 and dSd+1 in terms of the dual Conformal Partial Wave (which in Fourier space is a product of boundary three-point correlators) up to a factor which is determined by the boundary conditions. In this work we focus on late-time four-point correlators with external scalars and an exchanged field of integer spin-ℓ. The Mellin-Barnes representation makes manifest the analytic structure of boundary correlation functions, providing an analytic expression for the exchange four-point function which is valid for general d and generic scaling dimensions, in particular massive, light and (partially-)massless fields. It moreover naturally identifies boundary correlation functions for generic fields with multi-variable Meijer-G functions. When d = 3 we reproduce existing explicit results available in the literature for external conformally coupled and massless scalars. From these results, assuming the weak breaking of the de Sitter isometries, we extract the corresponding correction to the inflationary three-point function of general external scalars induced by a general spin- ℓ field at leading order in slow roll. These results provide a step towards a more systematic understanding of de Sitter observables at tree level and beyond using Mellin space methods.

Fluctuation and dissipation dynamics is examined at all temperature ranges for the general case of a background time evolving scalar field coupled to heavy intermediate quantum fields which in turn are coupled to light quantum fields. The evolution of the background field induces particle production from the light fields through the action of the intermediate catalyzing heavy fields. Such field configurations are generically present in most particle physics models, including Grand Unified and Supersymmetry theories, with application of this mechanism possible in inflation, heavy ion collision and phase transition dynamics. The effective evolution equation for the background field is obtained and a fluctuation-dissipation theorem is derived for this system. The effective evolution in general is nonlocal in time. Appropriate conditions are found for when these time nonlocal effects can be approximated by local terms. Here careful distinction is made between a local expansion and the special case of a derivative expansion to all orders, which requires analytic behavior of the evolution equation in Fourier space.

We experimentally study the interference of dipole scattered light from two optically levitated nanoparticles in vacuum, which present an environment free of particle-substrate interactions. We illuminate the two trapped nanoparticles with a linearly polarized probe beam orthogonal to the propagation of the trapping laser beams. The scattered light from the nanoparticles are collected by a high numerical aperture (NA) objective lens and imaged. The interference fringes from the scattered vector light for the different dipole orientations in image and Fourier space are observed. Especially, the interference fringes of two scattered light fields with polarization vortex show the π shift of the interference fringes between inside and outside the center region of the two nanoparticles in the image space. As far as we know, this is the first experimental observation of the interference of scattered vector light fields from two dipoles in free space. This work also provides a simple and direct method to determine the spatial scales between optically levitated nanoparticles by the interference fringes.

We show that structured light beams can be customized with a differential operator in Fourier space. This operator is represented as an algebraic function that acts on a seed beam for adjusting its shape. If the seed beams are perfect Laguerre-Gauss beams (PLGBs) and Bessel beams (BBs) without orbital angular momentum, we demonstrate that the custom beams generated on the seed-PLG preserve their distribution a longer distance than the propagation-invariant custom-caustic light fields obtained with the seed-Bessel, where both beams have similar initial conditions. In this sense, the custom-PLGBs can be a better option for many applications where the propagation-invariant light fields are used. We show some beam distributions-astroid, deltoid, and parabolic-generated with both seeds.

We consider the flatland or 2D properties of the light field generated when a homogeneous convex curved surface reflects a distant illumination field. Besides being of considerable theoretical interest, this problem has applications in computer vision and graphics - for instance, in determining lighting and bidirectional reflectance distribution functions (BRDFs), in rendering environment maps, and in image-based rendering. We demonstrate that the integral for the reflected light transforms to a simple product of coefficients in Fourier space. Thus, the operation of rendering can be viewed in simple signal processing terms as a filtering operation that convolves the incident illumination with the BRDF. This analysis leads to a number of interesting observations for computer graphics, computer vision, and visual perception.

The spatio-angular resolution of a light field (LF) display is a crucial factor for delivering adequate spatial image quality and eliciting an accommodation response. Previous studies have modelled retinal image formation with an LF display and evaluated whether accommodation would be evoked correctly. The models were mostly based on ray-tracing and a schematic eye model, which pose computational complexity and inaccurately represent the human eye population's behaviour. We propose an efficient wave-optics-based framework to model the human eye and a general LF display. With the model, we simulated the retinal point spread function (PSF) of a point rendered by an LF display at various depths to characterise the retinal image quality. Additionally, accommodation responses to the rendered point were estimated by computing the visual Strehl ratio based on the optical transfer function (VSOTF) from the PSFs. We assumed an ideal LF display that had an infinite spatial resolution and was free from optical aberrations in the simulation. We tested points rendered at 0-4 dioptres of depths having angular resolutions of up to 4x4 viewpoints within a pupil. The simulation predicted small and constant accommodation errors, which contradict the findings of previous studies. An evaluation of the optical resolution on the retina suggested a trade-off between the maximum achievable resolution and the depth range of a rendered point where in-focus resolution is kept high. The proposed framework can be used to evaluate the upper bound of the optical performance of an LF display for realistically aberrated eyes, which may help to find an optimal spatio-angular resolution required to render a high quality 3D scene.

Light field displays depict the real world by emitting light rays corresponding to the 3D space of the scene or object that is to be represented. Therefore, such displays are suitable for utilization in a multitude of use cases, including the cinema. Cinematography is the art encompassing motion picture photography. It describes a great number of methods for working with physical cameras. For example, a camera can be mounted on a car in a typical chase scene, attached to a spring, suspended or mounted on a colliding object and many more. Although these camera rigs may produce exceptional visuals, achieving similar results by using light field cameras will be extremely challenging. In order to capture light fields, two sets of possible solutions can occur. These include capturing light fields by means of an actual light field camera or via an array of cameras, where the latter may be planar or curved. Using one way or the other is rather demanding, as each method has its own limitations and difficulties. When using a light field camera, the captured light field should map to that of the light field display, on which it shall be visualized. Whereas capturing a scene with a camera array, can unfortunately lead to self-capture. Moreover, the portability of the camera arrays is evidently problematic, due to the sheer size and weight. In addition to these challenges, light field rendering on its own is far from being trivial. While rendering to conventional 2D displays from camera arrays may use image-based rendering, where many views of the scene can be set up from pre-captured images, light fields are represented by 5D plenoptic functions that are not easy to capture with conventional camera arrays. Moreover, image-based rendering techniques often fail to produce convincing results for light field displays. For some use cases, they can be reduced into 4D for horizontal- parallax-only light fields, since our eyes are horizontally separated and horizontal motion is more frequent than vertical. In practice, the creation of a light field scene from a set of images requires injecting each 2D image into a 4D light field representation. In this paper, we visualize different simulations for realistic physical camera motions on a real light field display. In order to overcome the aforementioned problems, virtual cameras were used to simulate a set of different physical camera motions used in cinematography for light field displays. In applications, physics simulation libraries include algorithms for the dynamics of soft as well as rigid bodies. Moreover, collision detection is also accounted for. Many tools have been devised to simulate physics, among which is the Bullet Physics library. In our work, we used the Bullet Physics library in order to generate realistic physical camera motions as well as physical environment simulations for light field displays. The limitations and challenges imposed by light field displays when simulating physical camera motions are discussed, along with the results and the produced outputs.

The paradigm theorem of light field optics is different from that of conventional imaging optics. The conditions for rays in a 3D space are reconstructed from the light field data shown on a 2D high definition flat display through the lens array. A light field display consists of a flat display and lens array attached on its screen. The light field display can be regarded as a transformation system for the light field data. Further, the light field data can be considered as 3D space information in another domain. It contains information of rays in a space, and a point light source in space is expressed as information of many rays that go through the point light source. A point light source in space represents a set of many point light sources, and each ray that composes a point light source should be described in the light field data. Therefore, the light field flat display requires a much higher resolution as compared to that required by a conventional imaging device, on which a point light source can be expressed by a single pixel. In this study, we developed various light field displays using high definition flat displays, introduced the theorem of light field display, and reported the examples of our experimental light field displays.

Near-eye light field displays offer natural 3D visual experiences for AR/VR users by projecting light rays onto retina as if the light rays were emanated from a real object. Such displays normally take four-dimensional light field data as input. Given that sizeable existing 3D contents are in the form of stereo images, we propose a practical approach that generates light field data from such contents at minimal computational cost while maintaining a reasonable image quality. The perceptual quality of light field is ensured by making the baseline of light field subviews consistent with that of the micro-projectors of the light field display and by compensating for the optical artifact of the light field display through digital rectification. The effectiveness and efficiency of the proposed approach is verified through both quantitative and qualitative experiments. The results demonstrate that our light field converter works for real-world light field displays.

Fourier-plane optical microscopy is a powerful technique for studying the angularly-resolved optical properties of a plethora of materials and devices. The information about the direction of the emission of light by a sample is extracted by imaging the objective back focal plane on a two-dimensional detector, via a suitable optical system. This imaging technique is able to resolve the angular spectrum of the light over a wide angular field of view, but typically it doesn’t provide any spectral information, since it integrates the light intensity over a broad wavelength range. On the other hand, advanced hyperspectral imaging techniques are able to record the spectrum of the transmitted/reflected/emitted light at each pixel of the detector. In this work, we combine an innovative hyperspectral imaging system with Fourier-space microscopy, and we apply the novel device to the characterization of planar organic microcavities. In our system, hyperspectral imaging is performed by Fourier-transform spectroscopy thanks to an innovative common-path birefringent interferometer: it generates two delayed replicas of the light field, whose interference pattern is recorded as a function of their delay. The Fourier Transform of the resulting interferogram yields the intensity spectrum for each element of the microscope angular field-of-view. This system provides an angle-resolved hyperspectral view of the microcavities. The hyperspectral Fourier-space image clearly evidences the cavity modes both in photoluminescence and reflection, whose energy has a parabolic dependence on the emission angle. From the hyperspectral image, we reconstruct a 3D view of the parabolic cavity dispersion across the whole Fourier space.

We experimentally realize higher-order catastrophic structures in light fields presenting solutions of the paraxial diffraction catastrophe integral. They are determined by potential functions whose singular mapping manifests as caustic hypersurfaces in control parameter space. By addressing different cross-sections in the higher-dimensional control parameter space, we embed swallowtail and butterfly catastrophes with varying caustic structures in the lower-dimensional transverse field distribution. We systematically analyze these caustics analytically and observe their field distributions experimentally in real and Fourier space. Their spectra can be described by polynomials or expressions with rational exponents capable to form a cusp.

We present synthetic Fourier transform light scattering, a method for measuring extended angle-resolved light scattering (ARLS) from individual microscopic samples. By measuring the light fields scattered from the sample plane and numerically synthesizing them in Fourier space, the angle range of the ARLS patterns is extended up to twice the numerical aperture of the imaging system with unprecedented sensitivity and precision. Extended ARLS patterns of individual microscopic polystyrene beads, healthy human red blood cells (RBCs), and Plasmodium falciparum-parasitized RBCs are presented.

An appropriate design of wavefront will enable light fields propagating along arbitrary trajectories thus forming accelerating beams in free space. Previous ways of designing such accelerating beams mainly rely on caustic methods, which start from diffraction integrals and only deal with two-dimensional fields. Here we introduce a new perspective to construct accelerating beams in phase space by designing the corresponding Wigner distribution function (WDF). We find such a WDF-based method is capable of providing both the initial field distribution and the angular spectrum in need by projecting the WDF into the real space and the Fourier space respectively. Moreover, this approach applies to the construction of both two- and three-dimensional fields, greatly generalizing previous caustic methods. It may therefore open up a new route to construct highly-tailored accelerating beams and facilitate applications ranging from particle manipulation and trapping to optical routing as well as material processing.

Natural and comfortable visual experience is critical to the success of metaverse, which has recently drawn worldwide attention. However, due to the vergence-accommodation conflict (VAC), most augmented reality (AR) or virtual reality (VR) displays on the market today can easily cause visual discomfort or eyestrain to users. Being able to resolve the VAC, light field display is commonly believed to be the ultimate display for metaverse. Similar to conventional near-eye AR displays, a near-eye light field AR display consists of three basic components: light source, projection unit, and eyepiece. Although the same light source can be used in both kinds of displays, the projection unit and the eyepiece of a near-eye light field AR display call for a new design to preserve the structure of light field when it reaches user’s retina. The primary focus of this paper is on the eyepiece design for a near-eye light field AR display. In consideration of the compact form factor and wide field of view, the birdbath architecture, which consists of a beam splitter and a combiner, is selected as the basis of our eyepiece design. We optimize the birdbath eyepiece for the light field projection module produced by PetaRay Inc. The birdbath eyepiece receives the light field emitted from the light field projection module and projects it fully into the user’s eye. Our design preserves the structure of light field and hence allows virtual objects at different depths to be properly perceived. In addition, the eyepiece design leads to a compact form factor for the near-eye light field AR display. Specifically, our eyepiece is designed by optimizing the tradeoff between the eyebox and the depth of focus (DOF) of the near-eye light field AR display. The resulting DOF allows the user to have a clear and sharp perception of any virtual object in the working range, which is from 30 cm to infinity. In addition, we optimize the entrance pupil position and the F-number of the eyepiece according to the exit pupil position and the divergence angle of the light field projection module. This way, the eyepiece is able to preserve the structure of light field, meaning that the angular relation between light rays coming from the same object point in space is preserved. To demonstrate the performance of our birdbath eyepiece, we use the Human Eye Model-Liou & Brennan (JOSA A 08/97) to simulate the image formation process.

This chapter discusses the topic of emerging light field displays from a signal processing perspective. Light field displays are defined as devices which deliver continuous parallax along with the focus and binocular visual cues acting together in rivalry-free manner. In order to ensure such functionality, one has to deal with the light field, conceptualized by the plenoptic function and its adequate parametrization, sampling and reconstruction. The light field basics and the corresponding display technologies are overviewed in order to address the fundamental problems of analyzing light field displays as signal processing channels, and of capturing and representing light field visual content for driving such displays. Spectral analysis of multidimensional sampling operators is utilized to profile the displays in question, and modern sparsification approaches are employed to develop methods for high-quality light field reconstruction and rendering.