Abstract
Reflectance transformation imaging (RTI) is a computational photography technique widely used in the cultural heritage and material science domains to characterize relieved surfaces. It basically consists of capturing multiple images from a fixed viewpoint with varying lights. Handling the potentially huge amount of information stored in an RTI acquisition that consists typically of 50–100 RGB values per pixel, allowing data exchange, interactive visualization, and material analysis, is not easy. The solution used in practical applications consists of creating “relightable images” by approximating the pixel information with a function of the light direction, encoded with a small number of parameters. This encoding allows the estimation of images relighted from novel, arbitrary lights, with a quality that, however, is not always satisfactory. In this paper, we present NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. Using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, especially in the case of challenging glossy materials. We also address the problem of validating the relight quality on different surfaces, proposing a specific benchmark, SynthRTI, including image collections synthetically created with physical-based rendering and featuring objects with different materials and geometric complexity. On this dataset and as well on a collection of real acquisitions performed on heterogeneous surfaces, we demonstrate the advantages of the proposed relightable image encoding.
Highlights
Reflectance transformation imaging [4,9,11] is a popular computational photography technique, allowing to capture the rich representations of surfaces including geometric details and local reflective behavior of materials
According to a recent survey [14] on the state of the art of Reflectance transformation imaging (RTI) processing and relighting of multi-light image collections, practical applications, especially in the cultural heritage domain, rely on the classic polynomial texture mapping (PTM) [9] or on Hemispherical Harmonics (HSH) [10] coefficients to store a compact representation of the original data and interactively relightable images
On the novel SynthRTI and real calibrated MLICs (RealRTI) datasets, we performed several tests to evaluate the advantages of the proposed neural encoding and relighting approach with respect to standard RTI encoding
Summary
Reflectance transformation imaging [4,9,11] is a popular computational photography technique, allowing to capture the rich representations of surfaces including geometric details and local reflective behavior of materials. A good RTI encoding should be compact and allow interactive relighting from an arbitrary direction, rendering the correct diffuse and specular behaviors of the imaged materials and limiting interpolation artifacts These requirements are not always satisfied with the methods currently employed in practical applications [13]. Using a fully connected autoencoder architecture and storing per-pixel codes and the decoder coefficients as our processed RTI data, we obtain a relevant compression still enabling accurate relighting quality and limiting interpolation artifacts Another relevant issue in the RTI community is the lack of benchmarks to evaluate data processing tools on the kind of surfaces typically captured in real-world applications (e.g., quasi-planar surfaces made of heterogeneous materials with a wide range of metallic and specular behaviors).
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
