Bidirectional Texture Function Data Research Articles

Neural BTF Compression and Interpolation

AbstractThe Bidirectional Texture Function (BTF) is a data‐driven solution to render materials with complex appearance. A typical capture contains tens of thousands of images of a material sample under varying viewing and lighting conditions. While capable of faithfully recording complex light interactions in the material, the main drawback is the massive memory requirement, both for storing and rendering, making effective compression of BTF data a critical component in practical applications. Common compression schemes used in practice are based on matrix factorization techniques, which preserve the discrete format of the original dataset. While this approach generalizes well to different materials, rendering with the compressed dataset still relies on interpolating between the closest samples. Depending on the material and the angular resolution of the BTF, this can lead to blurring and ghosting artefacts. An alternative approach uses analytic model fitting to approximate the BTF data, using continuous functions that naturally interpolate well, but whose expressive range is often not wide enough to faithfully recreate materials with complex non‐local lighting effects (subsurface scattering, inter‐reflections, shadowing and masking…). In light of these observations, we propose a neural network‐based BTF representation inspired by autoencoders: our encoder compresses each texel to a small set of latent coefficients, while our decoder additionally takes in a light and view direction and outputs a single RGB vector at a time. This allows us to continuously query reflectance values in the light and view hemispheres, eliminating the need for linear interpolation between discrete samples. We train our architecture on fabric BTFs with a challenging appearance and compare to standard PCA as a baseline. We achieve competitive compression ratios and high‐quality interpolation/extrapolation without blurring or ghosting artifacts.

BTF data Generation based on Deep Learning

Many applications, such as computer-aided design and game rendering, need to reproduce realistic material appearance in complex light environment and different visual conditions. The authenticity of the three-dimensional object or the scene is heavily depended on the representation and rendering of textures, where the Bidirectional Texture Function (BTF) is one of the most widely-used texture models. In this paper, we proposed a neural network to learn the representation of the BTF data for predicting new texture images under novel conditions. The proposed method was tested on a public BTF dataset and was shown to produce satisfactory synthetic results.

An efficient method for specular-enhanced BTF compression

The Bidirectional Texture Function (BTF) has been used in the data-driven model to recover the reflectance of complex surfaces. The high-dimensionality of the BTF causes huge measurement data, but that needs to be compressed when real-time rendering is required. However, the distribution of BTF data is highly non-uniform due to non-lambertian reflections. Projecting the measured data onto a linear space constructed by a statistical analysis such as PCA results in low-quality of data compression. In this paper, we propose a computationally efficient and robust data compression method. Based on the assumption that there exist many overlapping measurements, we seek to find reflectance holes caused by the non-lambertian reflection with a series of linear algorithms. Before performing the standard compression, our method separates diffuse reflections which can be well approximated with a linear model and compress them using a standard factorization. Specular components detected in the reflectance hole are separately stored using a sparse matrix representation to prevent the loss of the range of BTF data. Experimental results show that our method effectively improves the compression accuracy as well as the visual quality with a competitive computation time.

Optomechanical design of rotary kaleidoscope for bidirectional texture function acquisition.

Optical systems are traditionally used for accurate recording and measurement of the real world's appearance. Present techniques allow us to form a computer-based virtual world, which is used in a variety of technical fields. The crucial issue for future applications of virtual reality is the fidelity of rendered images to real-world objects. This is strongly affected by the appearance of the rendered object's surfaces. Currently, the most applied method of describing a surface's visual appearance of spatially nonuniform surfaces is bidirectional texture function (BTF). We have designed, optimized, built, and tested a unique portable instrument based on a rotary kaleidoscope principle for BTF acquisition in situ. To the best of our knowledge, such an instrument has never been used before to measure BTF of a surface. We enhanced a common static kaleidoscope by adding rotation, which allows us to get a larger number of images of the sample for more combinations of illumination directions and viewing directions. This results in a higher directional and spatial resolution of measured BTF data. In this paper, we focus on the optomechanical design of the rotary BTF measurement instrument and issues related to its alignment to keep the desired mechanical precision.

Simulating painted appearance of BTF materials

Bidirectional texture function (BTF) provides a realistic representation for real-world materials; however, the representation does not allow users to append additional rendering effects to the BTF data in a simple and effective way. In this paper, we present an approach for generating the painted appearance on a material that is represented by a BTF. We first convert a BTF into a physical representation composed of a height-field and a spatially varying bidirectional reflectance distribution function (SVBRDF). This representation allows us to simulate the appearance of paints on the surface of a material by computing the reflectance and the height field of the painted material. During rendering, we combine an existing BTF rendering approach and the modified material data to simulate the painted appearance of the BTF materials. Our experiments show that the proposed approach can simulate the painted appearance of a material while preserving the characteristic appearance of the original BTF. Our approach is mainly suitable for less specular BTF materials that allow depth reconstruction.

Bidirectional texture function image super-resolution using singular value decomposition.

The bidirectional texture function (BTF) is widely employed to achieve realistic digital reproduction of real-world material appearance. In practice, a BTF measurement device usually does not use high-resolution (HR) cameras in data collection, considering the high equipment cost and huge data space required. The limited image resolution consequently leads to the loss of texture details in BTF data. This paper proposes a fast BTF image super-resolution (SR) algorithm to deal with this issue. The algorithm uses singular value decomposition (SVD) to separate the collected low-resolution (LR) BTF data into intrinsic textures and eigen-apparent bidirectional reflectance distribution functions (eigen-ABRDFs) and then improves the resolution of the intrinsic textures via image SR. The HR BTFs can be finally obtained by fusing the reconstructed HR intrinsic textures with the LR eigen-ABRDFs. Experimental results show that the proposed algorithm outperforms the state-of-the-art single-image SR algorithms in terms of reconstruction accuracy. In addition, thanks to the employment of SVD, the proposed algorithm is computationally efficient and robust to noise corruption.

Extensions to bidirectional texture function compression with multi-level vector quantization in OpenCL

Bidirectional texture function (BTF) as an effective visual fidelity representation of surface appearance is becoming more and more widely used. In this paper we report on three contributions to BTF data compression. First, we propose novel decompositions for multi-level vector quantization in BTF data compression that improve the compression ratio by 12% in comparison with the original method, without loss of visual quality. This is achieved by a highly configurable software architecture that allows for different compression pipelines. Second, we show how for offline storage the compression ratio can be increased by 30% in total by Huffman coding. Third, we show how to efficiently parallelize this vector quantization algorithm in OpenCL for massively parallel architectures such as GPUs with a new modular design of the compression pipeline. The results for the new compression algorithm are shown on twelve low dynamic range BTFs and four high dynamic range publicly available BTF samples. Our method allows for real time synthesis on a GPU.

Rapid Material Appearance Acquisition Using Consumer Hardware

A photo-realistic representation of material appearance can be achieved by means of bidirectional texture function (BTF) capturing a material’s appearance for varying illumination, viewing directions, and spatial pixel coordinates. BTF captures many non-local effects in material structure such as inter-reflections, occlusions, shadowing, or scattering. The acquisition of BTF data is usually time and resource-intensive due to the high dimensionality of BTF data. This results in expensive, complex measurement setups and/or excessively long measurement times. We propose an approximate BTF acquisition setup based on a simple, affordable mechanical gantry containing a consumer camera and two LED lights. It captures a very limited subset of material surface images by shooting several video sequences. A psychophysical study comparing captured and reconstructed data with the reference BTFs of seven tested materials revealed that results of our method show a promising visual quality. Speed of the setup has been demonstrated on measurement of human skin and measurement and modeling of a glue dessication time-varying process. As it allows for fast, inexpensive, acquisition of approximate BTFs, this method can be beneficial to visualization applications demanding less accuracy, where BTF utilization has previously been limited.

Level-of-Detail Streaming and Rendering using Bidirectional Sparse Virtual Texture Functions

Bidirectional Texture Functions (BTFs) are among the highest quality material representations available today and thus well suited whenever an exact reproduction of the appearance of a material or complete object is required. In recent years, BTFs have started to find application in various industrial settings and there is also a growing interest in the cultural heritage domain. BTFs are usually measured from real-world samples and easily consist of tens or hundreds of gigabytes. By using data-driven compression schemes, such as matrix or tensor factorization, a more compact but still faithful representation can be derived. This way, BTFs can be employed for real-time rendering of photo-realistic materials on the GPU. However, scenes containing multiple BTFs or even single objects with high-resolution BTFs easily exceed available GPU memory on today’s consumer graphics cards unless quality is drastically reduced by the compression. In this paper, we propose the Bidirectional Sparse Virtual Texture Function, a hierarchical level-of-detail approach for the real-time rendering of large BTFs that requires only a small amount of GPU memory. More importantly, for larger numbers or higher resolutions, the GPU and CPU memory demand grows only marginally and the GPU workload remains constant. For this, we extend the concept of sparse virtual textures by choosing an appropriate prioritization, finding a trade off between factorization components and spatial resolution. Besides GPU memory, the high demand on bandwidth poses a serious limitation for the deployment of conventional BTFs. We show that our proposed representation can be combined with an additional transmission compression and then be employed for streaming the BTF data to the GPU from from local storage media or over the Internet. In combination with the introduced prioritization this allows for the fast visualization of relevant content in the users field of view and a consecutive progressive refinement.

Bidirectional Texture Function Compression Based on Multi‐Level Vector Quantization

Abstract The Bidirectional Texture Function (BTF) is becoming widely used for accurate representation of real‐world material appearance. In this paper a novel BTF compression model is proposed. The model resamples input BTF data into a parametrization, allowing decomposition of individual view and illumination dependent texels into a set of multi‐dimensional conditional probability density functions. These functions are compressed in turn using a novel multi‐level vector quantization algorithm. The result of this algorithm is a set of index and scale code‐books for individual dimensions. BTF reconstruction from the model is then based on fast chained indexing into the nested stored code‐books. In the proposed model, luminance and chromaticity are treated separately to achieve further compression. The proposed model achieves low distortion and compression ratios 1:233–1:2040, depending on BTF sample variability. These results compare well with several other BTF compression methods with predefined compression ratios, usually smaller than 1:200. We carried out a psychophysical experiment comparing our method with LPCA method. BTF synthesis from the model was implemented on a standard GPU, yielded interactive framerates. The proposed method allows the fast importance sampling required by eye‐path tracing algorithms in image synthesis.

G‐BRDFs: An Intuitive and Editable BTF Representation

Abstract Measured reflection data such as the bidirectional texture function (BTF) represent spatial variation under the full hemisphere of view and light directions and offer a very realistic visual appearance. Despite its high‐dimensional nature, recent compression techniques allow rendering of BTFs in real time. Nevertheless, a still unsolved problem is that there is no representation suited for real‐time rendering that can be used by designers to modify the BTF's appearance. For intuitive editing, a set of low‐dimensional comprehensible parameters, stored as scalars, colour values or texture maps, is required. In this paper we present a novel way to represent BTF data by introducing the geometric BRDF (g‐BRDF), which describes both the underlying meso‐ and micro‐scale structure in a very compact way. Both are stored in texture maps with only a few additional scalar parameters that can all be modified at runtime and thus give the designer full control over the material's appearance in the final real‐time application. The g‐BRDF does not only allow intuitive editing, but also reduces the measured data into a small set of textures, yielding a very effective compression method. In contrast to common material representation combining heightfields and BRDFs, our g‐BRDF is physically based and derived from direct measurement, thus representing real‐world surface appearance. In addition, we propose an algorithm for fully automatic decomposition of a given measured BTF into the g‐BRDF representation.

Edit Propagation on Bidirectional Texture Functions

AbstractWe propose an efficient method for editing bidirectional texture functions (BTFs) based on edit propagation scheme. In our approach, users specify sparse edits on a certain slice of BTF. An edit propagation scheme is then applied to propagate edits to the whole BTF data. The consistency of the BTF data is maintained by propagating similar edits to points with similar underlying geometry/reflectance. For this purpose, we propose to use view independent features including normals and reflectance features reconstructed from each view to guide the propagation process. We also propose an adaptive sampling scheme for speeding up the propagation process. Since our method needn't any accurate geometry and reflectance information, it allows users to edit complex BTFs with interactive feedback.

A psychophysically validated metric for bidirectional texture data reduction

Bidirectional Texture Functions (BTF) are commonly thought to provide the most realistic perceptual experience of materials from rendered images. The key to providing efficient compression of BTFs is the decision as to how much of the data should be preserved. We use psychophysical experiments to show that this decision depends critically upon the material concerned. Furthermore, we develop a BTF derived metric that enables us to automatically set a material's compression parameters in such a way as to provide users with a predefined perceptual quality. We investigate the correlation of three different BTF metrics with psychophysically derived data. Eight materials were presented to eleven naive observers who were asked to judge the perceived quality of BTF renderings as the amount of preserved data was varied. The metric showing the highest correlation with the thresholds set by the observers was the mean variance of individual BTF images. This metric was then used to automatically determine the material-specific compression parameters used in a vector quantisation scheme. The results were successfully validated in an experiment with six additional materials and eighteen observers. We show that using the psychophysically reduced BTF data significantly improves performance of a PCA-based compression method. On average, we were able to increase the compression ratios, and decrease processing times, by a factor of four without any differences being perceived.

Physically-Based Analysis and Rendering of Bidirectional Texture Functions Data

In pursuit of photorealistic rendering of surface material, Bidirectional Texture Function (BTF) has been used frequently in recent years. Its main drawback is its massive data size. To solve this, the Spatial Bidirectional Reflectance Function (SBRDF) techniques compress BTFs into reflectance model parameters. However, SBRDF cannot produce the self-shadowing and self-occlusion mesostructure effects in many real-world surfaces. Aiming at this drawback, we investigate how self-shadowing and self-occlusion affect the surface appearance by additional physically-based analysis. We mainly rely on two physical phenomena to separate self-shadowing and self-occlusion into two independent effects. First, the self-shadowing is view independent. Second, the self-occlusion is independent of changes in lighting direction. After those analyses, we are able to add self-shadowing and self-occlusion effects to SBRDF to achieve rendering quality that is much closer to the original uncompressed BTF data than SBRDF.

Extreme Compression and Modeling of Bidirectional Texture Function

The recent advanced representation for realistic real-world materials in virtual reality applications is the Bidirectional Texture Function (BTF) which describes rough texture appearance for varying illumination and viewing conditions. Such a function can be represented by thousands of measurements (images) per material sample. The resulting BTF size excludes its direct rendering in graphical applications and some compression of these huge BTF data spaces is obviously inevitable. In this paper we present a novel, fast probabilistic model-based algorithm for realistic BTF modeling allowing an extreme compression with the possibility of a fast hardware implementation. Its ultimate aim is to create a visual impression of the same material without a pixel-wise correspondence to the original measurements. The analytical step of the algorithm starts with a BTF space segmentation and a range map estimation by photometric stereo of the BTF surface, followed by the spectral and spatial factorization of selected sub-space color texture images. Single mono-spectral band-limited factors are independently modeled by their dedicated spatial probabilistic model. During rendering, the sub-space images of arbitrary size are synthesized and both color (possibly multi-spectral) and range information is combined in a bump-mapping filter according to the view and illumination directions. The presented model offers a huge BTF compression ratio unattainable by any alternative sampling-based BTF synthesis method. Simultaneously this model can be used to reconstruct missing parts of the BTF measurement space.

BTF modelling using BRDF texels

The highest fidelity representations of realistic real-world materials currently comprise Bidirectional Texture Functions (BTF). The BTF is a six-dimensional function depending on view and illumination directions as well as on planar texture coordinates. The huge size of such measurements, typically in the form of thousands of images covering all possible combinations of illumination and viewing angles, has prohibited their practical exploitation, and obviously some compression and modelling method of these enormous BTF data spaces is inevitable. The two proposed approaches combine BTF spatial clustering with cluster index modelling by means of efficient Markov random field models. The methods allow the generation of a seamless cluster index of arbitrary size to cover large virtual 3D object surfaces. Both methods represent original BTF data using a set of local spatially dependent Bidirectional Reflectance Distribution Function (BRDF) values which are combined according to the synthesized cluster index and illumination/viewing directions by means of two types of Markov random field models. BTF data compression using both methods is about 1:200 and their synthesis is very fast.

Tileable BTF

This paper presents a modular framework to efficiently apply the bidirectional texture functions (BTF) onto object surfaces. The basic building blocks are the BTF tiles. By constructing one set of BTF tiles, a wide variety of objects can be textured seamlessly without re-synthesizing the BTF. The proposed framework nicely decouples the surface appearance from the geometry. With this appearance-geometry decoupling, one can build a library of BTF tile sets to instantaneously dress and render various objects under variable lighting and viewing conditions. The core of our framework is a novel method for synthesizing seamless high-dimensional BTF tiles, that are difficult for existing synthesis techniques. Its key is to shorten the cutting paths and broaden the choices of samples so as to increase the chance of synthesizing seamless BTF tiles. To tackle the enormous data, the tile synthesis process is performed in compressed domain. This not just allows the handling of large BTF data during the synthesis, but also facilitates compact storage of the BTF in GPU memory during the rendering.

Reflectance field based real-time, high-quality rendering of bidirectional texture functions

The bidirectional texture function (BTF) is a suitable representation for the appearance of highly detailed surface structures under varying illumination and viewing conditions. Since real-time rendering of the full BTF data is currently not feasible, approximations of the six-dimensional BTF are used such that the amount of data is reduced and current graphics hardware can be exploited. While existing methods work well for materials with low-depth variation, realism is lost if the depth variation grows. In this article we analyze this problem and devise a new real-time rendering paradigm based on linear interpolation of reflection fields, which provides significant improvements with respect to realism for such highly structured materials without sacrificing the general applicability and speed of previous algorithms. We propose real-time rendering algorithms for this new method supporting either point light sources or image-based lighting and demonstrate the capabilities of our new approach with several examples.