Uncontrolled Illumination Research Articles

ControlMat: A Controlled Generative Approach to Material Capture

Material reconstruction from a photograph is a key component of 3D content creation democratization. We propose to formulate this ill-posed problem as a controlled synthesis one, leveraging the recent progress in generative deep networks. We present ControlMat, a method which, given a single photograph with uncontrolled illumination as input, conditions a diffusion model to generate plausible, tileable, high-resolution physically-based digital materials. We carefully analyze the behavior of diffusion models for multi-channel outputs, adapt the sampling process to fuse multi-scale information and introduce rolled diffusion to enable both tileability and patched diffusion for high-resolution outputs. Our generative approach further permits exploration of a variety of materials that could correspond to the input image, mitigating the unknown lighting conditions. We show that our approach outperforms recent inference and latent-space optimization methods, and we carefully validate our diffusion process design choices. 1

A Deeper Analysis of Volumetric Relightable Faces.

Portrait viewpoint and illumination editing is an important problem with several applications in VR/AR, movies, and photography. Comprehensive knowledge of geometry and illumination is critical for obtaining photorealistic results. Current methods are unable to explicitly model in 3D while handling both viewpoint and illumination editing from a single image. In this paper, we propose VoRF, a novel approach that can take even a single portrait image as input and relight human heads under novel illuminations that can be viewed from arbitrary viewpoints. VoRF represents a human head as a continuous volumetric field and learns a prior model of human heads using a coordinate-based MLP with individual latent spaces for identity and illumination. The prior model is learned in an auto-decoder manner over a diverse class of head shapes and appearances, allowing VoRF to generalize to novel test identities from a single input image. Additionally, VoRF has a reflectance MLP that uses the intermediate features of the prior model for rendering One-Light-at-A-Time (OLAT) images under novel views. We synthesize novel illuminations by combining these OLAT images with target environment maps. Qualitative and quantitative evaluations demonstrate the effectiveness of VoRF for relighting and novel view synthesis, even when applied to unseen subjects under uncontrolled illumination. This work is an extension of Rao et al. (VoRF: Volumetric Relightable Faces 2022). We provide extensive evaluation and ablative studies of our model and also provide an application, where any face can be relighted using textual input.

IMPROVEMENT PROCEDURE FOR IMAGE SEGMENTATION OF FRUITS AND VEGETABLES BASED ON THE OTSU METHOD.

Currently, there are significant challenges in the classification, recognition, and detection of fruits and vegetables. An important step to solve this problem is to obtain an accurate segmentation of the object of interest. However, the background and object separation in a gray image shows high errors for some thresholding techniques due to uneven or poorly conditioned lighting. An accepted strategy to decrease the error segmentation is to select the channel of a RGB image with high contrast. This paper presents the results of an experimental procedure based on a binary segmentation enhancement by using the Otsu method. The procedure was carried out with images of real agricultural products with and without additional noise to corroborate the robustness of the proposed strategy. The experimental tests were performed by using our database of RGB images of agricultural products under uncontrolled illumination. The results exhibit that the best segmentation is based on the selection of the Blue channel of the RGB test images due to its better contrast. Here, the quantitative results are measured by applying the Jaccard and Dice metrics based on the ground-truth images as optimal reference. Most of the results show an average percentage improvement difference greater than 45.5% in two experimental tests.

INTEGRATED 3D ACQUISITION OF COMPLEX WOODEN ARTEFACTS: THE PIFFETTI'S LIBRARY IN QUIRINALE PALACE (ROME)

Abstract. Integrated surveying with active and passive systems is a well-established process for understanding artifacts at different scales and levels of complexity. But some acquisition problems still exist, related to the presence of non-optically cooperating materials, reflective surfaces, and non-homogeneous and uncontrolled illumination conditions. Especially in these cases, the integration between different 3D acquisition techniques allows us to test the response of different instruments concerning the boundary conditions, going to compare the results. This activity highlights the pros and cons of each technique, identifying the right balance for future applications with the same conditions. The case study analyzed in the article is the Piffetti Library, a wooden work from the first half of the 1800s present at the Quirinale Palace (Rome). The artifact shows multiple levels of environmental complexity (small space with movement constraints and uncontrollable light), and it is composed of several wood pieces with a very glossy finish. The presence of shelving, free-form surfaces, and sculptural details requires careful planning to survey the artifact with consistent resolution. The use of different active and passive acquisition methods is tested, defining an integrated methodology coherent with the complexity of the artifact and the context in which it is positioned. The dual purpose is to define a possible replicable protocol, arriving at the definition of reliable 3D data for subsequent analysis and virtual reconstructions.

Analysis of Image Preprocessing and Binarization Methods for OCR-Based Detection and Classification of Electronic Integrated Circuit Labeling

Automatic recognition and classification of electronic integrated circuits based on optical character recognition combined with the analysis of the shape of their housings are essential to machine vision methods supporting the production of electronic parts, especially small-volume ones in the through-hole technology, characteristic of printed circuit boards. Since such methods utilize binary images, applying appropriate image preprocessing and thresholding methods significantly influences the obtained results, particularly in uncontrolled illumination conditions. Therefore, the examination of various adaptive image binarization algorithms for this purpose is conducted in this paper, together with the experimental verification of the proposed method based on the pixel voting approach.

Skin detection in video under uncontrolled illumination

Many vision-based human-computer interaction (HCI) applications require skin detection. However, their performance relies on accuracy in detecting skin regions in video, which is difficult under uncontrolled illumination. The chromatic appearance of skin changes because of shading, often caused by body movement. To address this, we propose a dynamic adaptation method to detect skin regions affected by local color deformations. Static and dynamic skin regions are detected by a corresponding module. The static module includes a facial skin distribution model (FSDM) and a fusion-based background distribution model (FBDM). The FBDM is obtained from a local background distribution model (LBDM) and a global background distribution model (GBDM). The LBDM is obtained by comparing a frame pixel distribution model with the FSDM and GBDM. Next, the FBDM is derived from the LBDM and the GBDM. The dynamic module includes a moving skin distribution model (MSDM), derived from a set of moving skin samples. Initially, moving skin regions are detected using a modified double frame-difference method and then modeled using a Gaussian mixture model. To avoid misidentifying background regions as skin, the final MSDM is obtained by comparing the initial moving skin model to the FSDM and FBDM. Finally, the static and the dynamic models are fused by applying a maximization rule. Experimental results shows that the proposed method can detect skin regions more accurately than state-of-the-art methods.

An Autonomous Illumination System for Vehicle Documentation Based on Deep Reinforcement Learning

A common problem for machine vision applications is uncontrolled illumination conditions that cause undesired artifacts on sensorial data. For instance, quality inspection using color cameras, while having wide industrial application, requires manual illumination adjustment and is severely affected by external lighting sources and the physical properties of the inspected object. To overcome this problem, we propose an autonomous illumination solution, that adjusts illumination via a Deep Reinforcement Learning (DRL) agent following a goal-oriented reward that takes into account image entropy and specularity. The system is validated in a challenging vehicle documentation use case where vehicle images are captured under various lighting conditions using a camera and an in-house built illumination system. The DRL agent learns to control illumination levels directly from high-dimensional visual inputs by mapping the interactions from the environment to the reward-driven control actions of the illumination system, targeting an optimal illumination zone even under the appearance of abrupt illumination changes in the environment.

Deep softmax collaborative representation for robust degraded face recognition

Deep convolutional neural networks (DCNN) have attracted much attention in the field of face recognition because they have achieved high performance than other approaches in the so-called in-the-wild datasets. However, in many real-world applications of face recognition, the performance of CNN-based algorithms is significantly decreased when images contain various kinds of degradations caused by random noise, motion blur, compression artifacts, uncontrolled illumination, and occlusion. Moreover, this is because the main weakness of existing DCNN models is the overfitting problem. To boost the recognition performance of state-of-the-art deep learning networks, we propose a deep softmax collaborative representation-based network, which can be used as a divide-and-conquer algorithm to help multiple DCCNs work together more effectively to solve multiple sub-problems of face reconstruction and classification. We demonstrated several experiments with challenging face recognition datasets. Our extensive experiments demonstrate that our proposed method is more robust and efficient in dealing with the challenging real-world problems in face recognition compared to related state-of-the-art methods.

A Hybrid Framework for Matching Printing Design Files to Product Photos

We propose a real-time image matching framework, which is hybrid in the sense that it uses both hand-crafted features and deep features obtained from a well-tuned deep convolutional network. The matching problem, which we concentrate on, is specific to a certain application, that is, printing design to product photo matching. Printing designs are any kind of template image files, created using a design tool, thus are perfect image signals. However, photographs of a printed product suffer many unwanted effects, such as uncontrolled shooting angle, uncontrolled illumination, occlusions, printing deficiencies in color, camera noise, optic blur, et cetera. For this purpose, we create an image set that includes printing design and corresponding product photo pairs with collaboration of an actual printing facility. Using this image set, we benchmark various hand-crafted and deep features for matching performance and propose a framework in which deep learning is utilized with highest contribution, but without disabling real-time operation using an ordinary desktop computer.

An improved Weber-face-based method for face recognition under uncontrolled illumination conditions

The paper presents a new face recognition system robust to illumination variations and moderate occlusion. Two main contributions are discussed. First, we introduce an approach based on contrast equalisation (CE) to improve the traditional Weber-face (WF) technique and make it more robust. Second, we use the local binary patterns (LBP) and local phase quantisation (LPQ) descriptors to make the Weber-face method more resilient to variations in illumination by exploiting both spatial and frequency domains information. By combining the two descriptors, enhanced facial features are obtained showing more discriminating power for variable lighting conditions as well as occlusion. The concept of complementing the WF model with spatial-frequency descriptors is novel and shown to result in a robust system resilient to changing lighting conditions, variations in pose, and occlusion. The method was compared to a number of existing techniques over three public databases. The proposed algorithm outperformed existing techniques under challenging environments.

An improved Weber-face-based method for face recognition under uncontrolled illumination conditions

The paper presents a new face recognition system robust to illumination variations and moderate occlusion. Two main contributions are discussed. First, we introduce an approach based on contrast equalisation (CE) to improve the traditional Weber-face (WF) technique and make it more robust. Second, we use the local binary patterns (LBP) and local phase quantisation (LPQ) descriptors to make the Weber-face method more resilient to variations in illumination by exploiting both spatial and frequency domains information. By combining the two descriptors, enhanced facial features are obtained showing more discriminating power for variable lighting conditions as well as occlusion. The concept of complementing the WF model with spatial-frequency descriptors is novel and shown to result in a robust system resilient to changing lighting conditions, variations in pose, and occlusion. The method was compared to a number of existing techniques over three public databases. The proposed algorithm outperformed existing techniques under challenging environments.

A Robust Geometric Skin Colour Face Detection Method under Unconstrained Environment of Smartphone Database

Face detection is the primary task in building a vision-based human-computer interaction system and in special applications such as face recognition, face tracking, face identification, expression recognition and also content-based image retrieval. A potent face detection system must be able to detect faces irrespective of illuminations, shadows, cluttered backgrounds, orientation and facial expressions. In previous literature, many approaches for face detection had been proposed. However, face detection in outdoor images with uncontrolled illumination and images with complex background are still a serious problem. Hence, in this paper, we had proposed a Geometric Skin Colour (GSC) method for detecting faces accurately in real world image, under capturing conditions of both indoor and outdoor, and with a variety of illuminations and also in cluttered backgrounds. The selected method was evaluated on two different face video smartphone databases and the obtained results proved the outperformance of the proposed method under the unconstrained environment of these databases.

Multifocus HDR VIS/NIR hyperspectral imaging and its application to works of art.

This paper presents a complete framework for capturing and processing hyperspectral reflectance images of artworks in situ, using a hyperspectral line scanner. These capturing systems are commonly used in laboratory conditions synchronized with scanning stages specifically designed for planar surfaces. However, when the intended application domain does not allow for image capture in these controlled conditions, achieving useful spectral reflectance image data can be a very challenging task (due to uncontrolled illumination, high-dynamic range (HDR) conditions in the scene, and the influence of chromatic aberration on the image quality, among other factors). We show, for the first time, all the necessary steps in the image capturing and post-processing in order to obtain high-quality HDR-based reflectance in the visible and near infrared, directly from the data captured by using a hyperspectral line scanner coupled to a rotating tripod. Our results show that the proposed method outperforms the normal capturing process in terms of dynamic range, color and spectral accuracy. To demonstrate the potential interest of this processing strategy for on-site analysis of artworks, we applied it to the study of a vintage copy of the famous painting "Transfiguration" by Raphael, as well as a facsimile of "The Golden Haggadah" from the British Library of London. The second piece has been studied for the identification of highly reflective gold-foil covered areas.

Human iris feature extraction under pupil size variation using local texture descriptors

The human iris texture is one of the most reliable biometric traits because it is unique, and the iris pattern remains stable for years. However, iris images acquired under uncontrolled illumination is one source of difficulties for iris recognition systems, mainly in applications at a distance and in non-cooperative environments. Different levels of light cause iris texture modifications due to pupil size variation. The iris contains 02 groups of muscles: the sphincter pupillae and the dilator pupillae. When the sphincter pupillae contracts the iris reduces the size of the pupil and its texture changes. It is well known in the biometric literature that pupil dilation degrades iris biometric performance. We propose in this paper to evaluate some local texture descriptors for iris recognition, considering pupil contraction and dilation. Furthermore, we propose 02 new texture descriptors called Median-Local-Mapped-Pattern (Median-LMP) and Modified Median-Local-Mapped-Pattern (MM-LMP) and compare their performances to the original Local Mapped Pattern (LMP), the Completed Modeling of Local Binary Pattern (CLBP), the Median Binary Pattern (MBP), the Weber Local Descriptor (WLD) and the Daugman’s method. Our results show that our methodology is more robust when we compare iris samples with different levels of pupil sizes (dilated vs contracted). Besides this, our descriptor performs better than all the compared methods, primarily if one iris with a contracted pupil is used for searching another iris with a dilated pupil.

On the use of ear and profile faces for distinguishing identical twins and nontwins

AbstractThis study aims to measure the efficiency of ear and profile face in distinguishing identical twins under identification and verification modes. In addition, to distinguish identical twins by ear and profile face separately, we propose to fuse these traits with all possible binary combinations of left ear, left profile face, right ear, and right profile face. Fusion is implemented by score‐level fusion and decision‐level fusion techniques in the proposed method. Additionally, feature‐level fusion is used for comparison. All experiments in this paper are also implemented on nontwins individuals, and the recognition performance of twins and nontwins are compared. Local binary patterns, local phase quantization, and binarized statistical image features approaches are used as texture‐based descriptors for feature extraction process. Images under controlled and uncontrolled lighting are tested. Ear and profile images from ND‐TWINS‐2009‐2010 dataset are used in the experiments. The experimental results show that the proposed method is more accurate and reliable than using ear or profile face images separately. The performance of the proposed method for recognizing identical twins as recognition rate is 100% and 99.45%, and equal error rates are 0.54% and 1.63% in controlled and uncontrolled illumination conditions, respectively.

Multispectral camera as spatio-spectrophotometer under uncontrolled illumination.

Multispectral constancy enables the illuminant invariant representation of multi-spectral data. This article proposes an experimental investigation of multispectral constancy through the use of multispectral camera as a spectrophotometer for the reconstruction of surface reflectance. Three images with varying illuminations are captured and the spectra of material surfaces is reconstructed. The acquired images are transformed into canonical representation through the use of diagonal transform and spectral adaptation transform. Experimental results show that use of multispectral constancy is beneficial for both filter-wheel and snapshot multi-spectral cameras. The proposed concept is robust to errors in illuminant estimation and is able to perform well with linear spectral reconstruction method. This work makes us one step closer to the use of multispectral imaging for computer vision.

Optimized superpixel and AdaBoost classifier for human thermal face recognition

Infrared spectrum-based human recognition systems offer straightforward and robust solutions for achieving an excellent performance in uncontrolled illumination. In this paper, a human thermal face recognition model is proposed. The model consists of four main steps. Firstly, the grey wolf optimization algorithm is used to find optimal superpixel parameters of the quick-shift segmentation method. Then, segmentation-based fractal texture analysis algorithm is used for extracting features and the rough set-based methods are used to select the most discriminative features. Finally, the AdaBoost classifier is employed for the classification process. For evaluating our proposed approach, thermal images from the Terravic Facial infrared dataset were used. The experimental results showed that the proposed approach achieved (1) reasonable segmentation results for the indoor and outdoor thermal images, (2) accuracy of the segmented images better than the non-segmented ones, and (3) the entropy-based feature selection method obtained the best classification accuracy. Generally, the classification accuracy of the proposed model reached to 99% which is better than some of the related work with around 5%.

Automatic Torso Detection in Images of Preterm Infants

Imaging systems have applications in patient respiratory monitoring but with limited application in neonatal intensive care units (NICU). In this paper we propose an algorithm to automatically detect the torso in an image of a preterm infant during non-invasive respiratory monitoring. The algorithm uses normalised cut to segment each image into clusters, followed by two fuzzy inference systems to detect the nappy and torso. Our dataset comprised overhead images of 16 preterm infants in a NICU, with uncontrolled illumination, and encompassing variations in poses, presence of medical equipment and clutter in the background. The algorithm successfully identified the torso region for 15 of the 16 images, with a high agreement between the detected torso and the torso identified by clinical experts.

VITRAIL: Acquisition, Modeling, and Rendering of Stained Glass.

Stained glass windows are designed to reveal their powerful artistry under diverse and time-varying lighting conditions; virtual relighting of stained glass, therefore, represents an exceptional tool for the appreciation of this age old art form. However, as opposed to most other artifacts, stained glass windows are extremely difficult if not impossible to analyze using controlled illumination because of their size and position. In this paper, we present novel methods built upon image based priors to perform virtual relighting of stained glass artwork by acquiring the actual light transport properties of a given artifact. In a preprocessing step, we build a material-dependent dictionary for light transport by studying the scattering properties of glass samples in a laboratory setup. We can now use the dictionary to recover a light transport matrix in two ways: under controlled illuminations the dictionary constitutes a sparsifying basis for a compressive sensing acquisition, while in the case of uncontrolled illuminations the dictionary is used to perform sparse regularization. The proposed basis preserves volume impurities and we show that the retrieved light transport matrix is heterogeneous, as in the case of real world objects. We present the rendering results of several stained glass artifacts, including the Rose Window of the Cathedral of Lausanne, digitized using the presented methods.

Addressing the illumination challenge in two‐dimensional face recognition: a survey

Uncontrolled illumination is one of the most widely researched and most encountered face recognition challenges in recent years. In this study, the authors propose the division of algorithms into two categories: (i) relighting and (ii) unlighting. Relighting methods try to match the probe's illumination conditions using a subset of representative gallery images, while unlighting methods seek to suppress the variations. A total of 64 state‐of‐the‐art methods are summarised and categorised in each of the groups. To make this work concise and easy to follow, they restricted themselves to selected conferences/journals and they limited the number of approaches reviewed. Also, eight past state‐of‐the‐art approaches are used in both identification and verification experiments. However, only significant reported results from all methods were compared and organised in tables. The author's main objective is not to provide an exhaustive analysis of each category, but to present a collection of papers that can be useful in identifying research directions. Results indicate that unlighting methods are a better and a practical solution to address illumination challenges.