Light Source Estimation Research Articles

A low-cost IoT multi-spectral acquisition device

Color and lighting quality assessment allow users to reliably examine the quality of light sources and the changes in the lighting conditions. Technological advance has led to an increase in the development of intelligent systems that are permanently connected to the Internet and that allow access to real time-data. Smart lighting systems have become a subject of great interest given their potential in different application areas, such as optimization for object reflectance and damage reduction to artwork. It has become necessary to develop new sensors that are aligned with the technological needs, that are easily integrated into IoT (Internet of Things) or smart lightning systems, and that allow estimating lighting quality measurements in real time. Here, a low-cost IoT multi-spectral acquisition device with wireless communication is presented. Insights into the overall system design and a validation of its operation is provided to allow reproducibility and independent testing of the proposed device. The device can be used for light source spectrum recovery and estimation of light quality and color rendering measures. The total cost of the device is below 130 USD, and customizability provides a great advantage over commercially available devices.

Correction to: DeepLight: light source estimation for augmented reality using deep learning

The spelling of Hannes Kaufmann’s family name was incorrect. The correct wording is given here.

DeepLight: light source estimation for augmented reality using deep learning

This paper presents a novel method for illumination estimation from RGB-D images. The main focus of the proposed method is to enhance visual coherence in augmented reality applications by providing accurate and temporally coherent estimates of real illumination. For this purpose, we designed and trained a deep neural network which calculates a dominant light direction from a single RGB-D image. Additionally, we propose a novel method for real-time outlier detection to achieve temporally coherent estimates. Our method for light source estimation in augmented reality was evaluated on the set of real scenes. Our results demonstrate that the neural network can successfully estimate light sources even in scenes which were not seen by the network during training. Moreover, we compared our results with illumination estimates calculated by the state-of-the-art method for illumination estimation. Finally, we demonstrate the applicability of our method on numerous augmented reality scenes.

SymPS: BRDF Symmetry Guided Photometric Stereo for Shape and Light Source Estimation.

We propose uncalibrated photometric stereo methods that address the problem due to unknown isotropic reflectance. At the core of our methods is the notion of "constrained half-vector symmetry" for general isotropic BRDFs. We show that such symmetry can be observed in various real-world materials, and it leads to new techniques for shape and light source estimation. Based on the 1D and 2D representations of the symmetry, we propose two methods for surface normal estimation; one focuses on accurate elevation angle recovery for surface normals when the light sources only cover the visible hemisphere, and the other for comprehensive surface normal optimization in the case that the light sources are also non-uniformly distributed. The proposed robust light source estimation method also plays an essential role to let our methods work in an uncalibrated manner with good accuracy. Quantitative evaluations are conducted with both synthetic and real-world scenes, which produce the state-of-the-art accuracy for all of the non-Lambertian materials in MERL database and the real-world datasets.

ElasticFusion: Real-time dense SLAM and light source estimation

We present a novel approach to real-time dense visual simultaneous localisation and mapping. Our system is capable of capturing comprehensive dense globally consistent surfel-based maps of room scale environments and beyond explored using an RGB-D camera in an incremental online fashion, without pose graph optimization or any post-processing steps. This is accomplished by using dense frame-to-model camera tracking and windowed surfel-based fusion coupled with frequent model refinement through non-rigid surface deformations. Our approach applies local model-to-model surface loop closure optimizations as often as possible to stay close to the mode of the map distribution, while utilizing global loop closure to recover from arbitrary drift and maintain global consistency. In the spirit of improving map quality as well as tracking accuracy and robustness, we furthermore explore a novel approach to real-time discrete light source detection. This technique is capable of detecting numerous light sources in indoor environments in real-time as a user handheld camera explores the scene. Absolutely no prior information about the scene or number of light sources is required. By making a small set of simple assumptions about the appearance properties of the scene our method can incrementally estimate both the quantity and location of multiple light sources in the environment in an online fashion. Our results demonstrate that our technique functions well in many different environments and lighting configurations. We show that this enables (a) more realistic augmented reality rendering; (b) a richer understanding of the scene beyond pure geometry and; (c) more accurate and robust photometric tracking.

English

A method for light sources estimation is proposed in this paper. The method utilizes feature points in cast shadows to estimate near light source positions from estimated source directions using specular highlights. There are several methods that can be used to estimate light sources from scene images, using either cast shadows or specular highlights. However, most of them are limited to directional light sources. The proposed method can estimate the positions and intensities of multiple near point light sources. Specular highlights on an object of known geometry are first used for light source direction estimation. Then, a discontinuity point in the object shape and the corresponding cast shadow on a ground plane are used for light source position estimation. Feature points obtained from an image of the cast shadow, however, can be inaccurate due to various factors. Information on diffused light reflected from Lambertian ground-plane surface is subsequently used to improve estimation accuracy. Experimental results were used to evaluate the performance of the proposed method.   Key words: Light source estimation, light source recovery, augmented reality.

Fast Light Source Estimation Technique for Effective Synthesis of Mixed Reality Scene

Fast Light Source Estimation Technique for Effective Synthesis of Mixed Reality Scene

Improved Variance Cut Algorithm for Light Source Estimation in Augmented Reality

illumination plays very important role in efficient integration of virtual objects into real world scene. Real time photometric registration requires efficient detection and segmentation of the brighter regions in an image to provide better light source detection and thus efficient illumination of virtual objects is possible. The paper presents a technique for light source sampling of images taken through a fish-eye Camera. These images taken through this Camera are sampled using improved variance-cut algorithm. The technique helps to detect the brighter region efficiently. With deferred rendering, some experimental results demonstrate that the proposed method produces plausible AR visualization Keywordsreality), illumination, median-cut algorithm

Light Source Estimation in Mobile Augmented Reality Scenes by Using Human Face Geometry

Light source estimation and virtual lighting must be believable in terms of appearance and correctness in augmented reality scenes. As a result of illumination complexity in an outdoor scene, realistic lighting for augmented reality is still a challenging problem. In this paper, we propose a framework based on an estimation of environmental lighting from well-defined objects, specifically human faces. The method is tuned for outdoor use, and the algorithm is further enhanced to illuminate virtual objects exposed to direct sunlight. Our model can be integrated into existing mobile augmented reality frameworks to enhance visual perception.

Scene surface light field representation for realistic lighting

Existing light field-based lighting follows the representation on plane, such as the commonly used light slab. A novel light field representation on scene surface is proposed. The illumination is captured by a panorama camera on translation stage with a nonuniform sampling and calibration strategy. Sampled light rays are reprojected on the reconstructed scene to generate the scene surface light field representations for both direct light source and indirect light. This representation is more consistent with the actual emission and reflection of light transport than the existing ones. An accurate light source estimation and a practical indirect light resampling approach are presented based on the representation. The rendering results illustrate the realistic lighting effect, especially in spatially varying illumination.

13-5 人間の顔に着目した単一画像からの光源推定に関する一検討(第13部門 映像表現)

13-5 人間の顔に着目した単一画像からの光源推定に関する一検討(第13部門 映像表現)

Difference sphere: An approach to near light source estimation

We present a novel approach for estimating light sources from a single image of a scene that is illuminated by major near point light sources, a few directional light sources and ambient light. We propose to employ a pair of reference spheres as light probes and introduce the difference sphere that we acquire by differencing the intensities of two image regions of the reference spheres. Because the effect by directional light sources and ambient light is eliminated by differencing, the key advantage of considering the difference sphere is that it enables us to identify near point light sources including their radiant intensities. We also show that analysis of gray level contours on spherical surfaces facilitates separate identification of multiple combined light sources and is well suited to the difference sphere. We demonstrate the effectiveness of the entire algorithm with experimental results while also investigating the applicability to an open scanner in indoor scene.

Light source estimation of outdoor scenes for mixed reality

Illumination consistency is important for photorealistic rendering of mixed reality. However, it is usually difficult to acquire illumination conditions of natural environments. In this paper, we propose a novel method for evaluating the light conditions of a static outdoor scene without knowing its geometry, material, or texture. In our method, we separate respectively the shading effects of the scene due to sunlight and skylight through learning a set of sample images which are captured with the same sun position. A fixed illumination map of the scene under sunlight or skylight is then derived reflecting the scene geometry, surface material properties and shadowing effects. These maps, one for sunlight and the other for skylight, are therefore referred to as basis images of the scene related to the specified sun position. We show that the illumination of the same scene under different weather conditions can be approximated as a linear combination of the two basis images. We further extend this model to estimate the lighting condition of scene images under deviated sun positions, enabling virtual objects to be seamlessly integrated into images of the scene at any time. Our approach can be applied for online video process and deal with both cloudy and sun shine situations. Experiment results successfully verify the effectiveness of our approach.

가상 복사조도 반구와 반사계수에 근거한 얼굴 재조명

본 논문에서는 임의의 알려지지 않은 조명 상황 하에서 3차원 객체의 텍스처 영상 한 장을 이용하여 광원의 위치를 추정하고 이를 이용해 조명에 의해 왜곡되어 있는 얼굴 영상을 재조명하는 기법을 제안한다. 우선 주어진 텍스처 영상으로부터 광원의 위치를 추정하기 위해 법선 벡터와 가중 양선형 보간을 이용하여 가상 복사조도 반구를 만들었다. 이를 이용해 추정된 주변광과 확산광 계수로 재조명 방정식을 도출하였다. 얼굴 텍스처 영상에서의 그림자와 음영을 원색으로 복원하는 알고리즘의 효율성과 정확성을 보이기 위해 광원의 위치 추정, 재조명, 얼굴 인식 등의 다양한 실험 결과를 보였다. 실험결과에 의하여 제안한 알고리즘은 조명에 강인한 얼굴 인식시스템뿐만 아니라 3차원 디스플레이에서의 작업 시 시각적 피로감을 줄이고 작업 효율을 높일 수 있음을 확인하였다. We present a novel method to estimate the light source direction and relight a face texture image of a single 3D model under arbitrary unknown illumination conditions. We create a virtual irradiance sphere to detect the light source direction from a given illuminated texture image using both normal vector mapping and weighted bilinear interpolation. We then induce a relighting equation with estimated ambient and diffuse coefficients. We provide the result of a series of experiments on light source estimation, relighting and face recognition to show the efficiency and accuracy of the proposed method in restoring the shading and shadows areas of a face texture image. Our approach for face relighting can be used for not only illuminant invariant face recognition applications but also reducing visual load and Improving visual performance in tasks using 3D displays.

Face relighting based on virtual irradiance sphere and reflection coefficient

We present a novel method to estimate the light source direction and relight a face texture image of a single 3-D model under arbitrary unknown illumination conditions. We create a virtual irradiance sphere to detect the light source direction from a given illuminated texture image using both normal vector mapping and weighted bilinear interpolation. We then induce a relighting equation with estimated ambient and diffuse intensity. We provide the result of a series of experiments on light source estimation, relighting, and face recognition to show the efficiency and accuracy of the proposed method in restoring the shading and shadow areas of a face texture image. Our approach for face relighting can be used for illuminant-invariant face recognition applications.

Point Light Source Estimation from Two Images and Its Limits

In this paper, the performance of parameter estimation of a single static distant point light source from two video images is analyzed in terms of estimation theory. The illumination parameters are the intensity and direction of the light source. In the first part of this paper, estimators from the literature are reviewed. Most recent estimators evaluate as input data two video images as well as the 3D shape and the 3D motion of the visible moving objects. In the second part of the paper, the performance of these recent methods is analyzed. The input data to estimation as well as the inherent input data errors are described by a stochastic observation model. Based on this model, the performance is analyzed regarding the Cramer-Rao theoretical lower bound of estimation error variances. The bound is derived for a variety of cases of scene illumination, object motion and errors in input data. For simplification purpose, the bound is valid only for object motions with the rotation axis lying in the image plane. The analysis shows in which cases which estimation accuracy can be expected with current methods. Finally, a comparison of the bound with one of the recent estimators shows that recent estimators are suboptimal in case of errors in the 3D shape of the objects. In future work, the stochastic observation model presented in this paper can be used to improve illumination estimation.

Estimating illuminant direction and degree of surface relief

Many algorithms for deriving surface shape from shading require an estimate of the direction of illumination. This paper presents a new estimator for illuminant direction, which also generates an estimate of the degree of surface relief, that is measured by the variance of surface orientation (the partial derivatives of surface depth). Surfaces are considered to be samples of a stochastic process representing depth as a function of position in the image plane. We derive an estimator for illuminant tilt that is based only on some general assumptions about the process. The assumptions are that the process is wide-sense stationary, strictly isotropic; and mean-square differentiable and that the second partial derivatives of surface depth are locally independent of the first partial derivatives. We develop an estimator of illuminant slant and degree of surface relief in two stages. In the first, we develop a general format for an estimator based on the same assumptions that are used for the tilt estimator. The second stage is the actual implementation of the estimator and requires the specification of a functional form for the local probability distribution of surface orientations. This approach contrasts with previous ones, which begin their development with an assumption of a particular distribution for surfaces. The approach has the advantage that it separates the problems of surface modeling and light-source estimation, permiting one to easily implement specific estimators for different surface models. We implement the illuminant slant estimator for surfaces that have a Gaussian distribution of surface orientations and show simulation results. Degraded performance in the presence of self-shadowing is discussed.