Bidirectional Texture Function Research Articles

Optical and Electromechanical Design and Implementation of an Advanced Multispectral Device to Capture Material Appearance.

The application of materials with changing visual properties with lighting and observation directions has found broad utility across diverse industries, from architecture and fashion to automotive and film production. The expanding array of applications and appearance reproduction requirements emphasizes the critical role of material appearance measurement and surface characterization. Such measurements offer twofold benefits in soft proofing and product quality control, reducing errors and material waste while providing objective quality assessment. Some image-based setups have been proposed to capture the appearance of material surfaces with spatial variations in visual properties in terms of Spatially Varying Bidirectional Reflectance Distribution Functions (SVBRDF) and Bidirectional Texture Functions (BTF). However, comprehensive exploration of optical design concerning spectral channels and per-pixel incident-reflection direction calculations, along with measurement validation, remains an unexplored domain within these systems. Therefore, we developed a novel advanced multispectral image-based device designed to measure SVBRDF and BTF, addressing these gaps in the existing literature. Central to this device is a novel rotation table as sample holder and passive multispectral imaging. In this paper, we present our compact multispectral image-based appearance measurement device, detailing its design, assembly, and optical considerations. Preliminary measurements showcase the device's potential in capturing angular and spectral data, promising valuable insights into material appearance properties.

Acquisition and Application of Reflectance for Computer-Generated Images

In the field of computer graphics, accurate representation of material properties is crucial for rendering realistic imagery. This paper focuses on the bidirectional reflectance distribution function (BRDF) and its role in determining how materials interact with light. The authors review the state of the art in reflectance measurement systems, with a focus on BRDF and bidirectional texture function (BTF) measurement. They discuss practical limitations in measuring multi-dimensional functions and provide examples of how researchers have addressed these challenges. Additionally, they analyse various approaches to converting measured data into practical analytical functions for use in commercial rendering software, including data-driven methods such as neural networks and hybridized approaches.

Low-Rank High-Order Tensor Completion With Applications in Visual Data.

Recently, tensor Singular Value Decomposition (t-SVD)-based low-rank tensor completion (LRTC) has achieved unprecedented success in addressing various pattern analysis issues. However, existing studies mostly focus on third-order tensors while order- d ( d ≥ 4 ) tensors are commonly encountered in real-world applications, like fourth-order color videos, fourth-order hyper-spectral videos, fifth-order light-field images, and sixth-order bidirectional texture functions. Aiming at addressing this critical issue, this paper establishes an order- d tensor recovery framework including the model, algorithm and theories by innovatively developing a novel algebraic foundation for order- d t-SVD, thereby achieving exact completion for any order- d low t-SVD rank tensors with missing values with an overwhelming probability. Emperical studies on synthetic data and real-world visual data illustrate that compared with other state-of-the-art recovery frameworks, the proposed one achieves highly competitive performance in terms of both qualitative and quantitative metrics. In particular, as the observed data density becomes low, i.e., about 10%, the proposed recovery framework is still significantly better than its peers. The code of our algorithm is released at https://github.com/Qinwenjinswu/TIP-Code.

Robust low-rank tensor reconstruction using high-order t-SVD

Currently, robust low-rank tensor reconstruction based on tensor singular value decomposition (t-SVD) has made remarkable achievements in the fields of computer vision, image processing, etc. However, existing works mainly concentrate on third-order tensors while order-d (d ≥ 4) tensors are commonly encountered in practical applications, such as fourth-order color videos, fourth-order facial images, fifth-order light-field images and sixth-order bidirectional texture functions. Aiming at this critical issue, we establish a robust order-d tensor reconstruction framework including the model, algorithm and theory by developing a novel algebraic foundation of order-d t-SVD with any invertible linear transforms. Equipped with the newly developed framework, any order-d low-rank tensors can be accurately reconstructed from a sample of its entries corrupted by arbitrary outliers. The superiority and effectiveness of the proposed framework are substantiated in comparison with state-of-the-art alternatives by conducting experiments on both synthetic and real-world tensor data.

Texture Segmentation Benchmark.

The Prague texture segmentation data-generator and benchmark (mosaic.utia.cas.cz) is a web-based service designed to mutually compare and rank (recently nearly 200) different static and dynamic texture and image segmenters, to find optimal parametrization of a segmenter and support the development of new segmentation and classification methods. The benchmark verifies segmenter performance characteristics on potentially unlimited monospectral, multispectral, satellite, and bidirectional texture function (BTF) data using an extensive set of over forty prevalent criteria. It also enables us to test for noise robustness and scale, rotation, or illumination invariance. It can be used in other applications, such as feature selection, image compression, query by pictorial example, etc. The benchmark's functionalities are demonstrated in evaluating several examples of leading previously published unsupervised and supervised image segmentation algorithms. However, they are used to illustrate the benchmark functionality and not review the recent image segmentation state-of-the-art.

Texture spectral similarity criteria

New similarity criteria capable of assessing spectral modelling plausibility of colour, bidirectional texture functions (BTF), and hyperspectral textures are presented. The criteria credibly compare the multi-spectral pixel values of the textures. They simultaneously consider the pixels of similar values and their mutual ratios. It allows support of the optimal modelling or acquisition setup development by comparing the original data with its synthetic simulations. Analytical applications of the criteria can be spectral based texture retrieval or classification. The suggested criteria together with existing alternatives are extensively tested and compared on a wide variety of colour, BTF, and hyperspectral textures. The performance quality of the criteria is examined in a long series of thousands specially designed monotonically degrading experiments, where proposed ones outperform all tested alternatives.

Neural BTF Compression and Interpolation

AbstractThe Bidirectional Texture Function (BTF) is a data‐driven solution to render materials with complex appearance. A typical capture contains tens of thousands of images of a material sample under varying viewing and lighting conditions. While capable of faithfully recording complex light interactions in the material, the main drawback is the massive memory requirement, both for storing and rendering, making effective compression of BTF data a critical component in practical applications. Common compression schemes used in practice are based on matrix factorization techniques, which preserve the discrete format of the original dataset. While this approach generalizes well to different materials, rendering with the compressed dataset still relies on interpolating between the closest samples. Depending on the material and the angular resolution of the BTF, this can lead to blurring and ghosting artefacts. An alternative approach uses analytic model fitting to approximate the BTF data, using continuous functions that naturally interpolate well, but whose expressive range is often not wide enough to faithfully recreate materials with complex non‐local lighting effects (subsurface scattering, inter‐reflections, shadowing and masking…). In light of these observations, we propose a neural network‐based BTF representation inspired by autoencoders: our encoder compresses each texel to a small set of latent coefficients, while our decoder additionally takes in a light and view direction and outputs a single RGB vector at a time. This allows us to continuously query reflectance values in the light and view hemispheres, eliminating the need for linear interpolation between discrete samples. We train our architecture on fabric BTFs with a challenging appearance and compare to standard PCA as a baseline. We achieve competitive compression ratios and high‐quality interpolation/extrapolation without blurring or ghosting artifacts.

Image-based appearance acquisition of effect coatings

Paint manufacturers strive to introduce unique visual effects to coatings in order to visually communicate functional properties of products using value-added, customized design. However, these effects often feature complex, angularly dependent, spatially-varying behavior, thus representing a challenge in digital reproduction. In this paper we analyze several approaches to capturing spatially-varying appearances of effect coatings. We compare a baseline approach based on a bidirectional texture function (BTF) with four variants of half-difference parameterization. Through a psychophysical study, we determine minimal sampling along individual dimensions of this parameterization. We conclude that, compared to BTF, bivariate representations better preserve visual fidelity of effect coatings, better characterizing near-specular behavior and significantly the restricting number of images which must be captured.

Increasing the Spatial Resolution of BTF Measurement with Scheimpflug Imaging

AbstractWe present an improved way of acquiring spatially varying surface reflectance represented by a bidirectional texture function (BTF). Planar BTF samples are measured as images at several inclination angles which puts constraints on the minimum depth of field of cameras used in the measurement instrument. For standard perspective imaging, we show that the size of a sample measured and the achievable spatial resolution are strongly interdependent and limited by diffraction at the lens' aperture. We provide a formula for this relationship. We overcome the issue of the required depth of field by using Scheimpflug imaging further enhanced by an anamorphic attachment. The proposed optics doubles the spatial resolution of images compared to standard perspective imaging optics. We built an instrument prototype with the proposed optics that is portable and allows for measurement on site. We show rendered images using the visual appearance measured by the instrument prototype.

BTF data Generation based on Deep Learning

Many applications, such as computer-aided design and game rendering, need to reproduce realistic material appearance in complex light environment and different visual conditions. The authenticity of the three-dimensional object or the scene is heavily depended on the representation and rendering of textures, where the Bidirectional Texture Function (BTF) is one of the most widely-used texture models. In this paper, we proposed a neural network to learn the representation of the BTF data for predicting new texture images under novel conditions. The proposed method was tested on a public BTF dataset and was shown to produce satisfactory synthetic results.

Optical analysis of coatings including diffractive pigments using a high-resolution gonioreflectometer

The aim of this article is to demonstrate a new way of measuring and understanding the appearance of pigment flake orientation and texture in special effect pigments for use in industrial coatings. We have used diffractive pigments and analyzed the relative orientation of the particles in the coating layers by evaluating their behavior in two common industry applications: solventborne and powder coatings. We have measured the interference color by taking readings with a high-resolution gonioreflectometer, in order to test the viability of automatic diffractive pigment evaluation. The results were analyzed using both psychophysical (i.e., human) and computational (i.e., mechanical) methods. Our later psychophysical and computational analysis of the visual differences that diffractive pigments present in both solventborne (1) and powder coating (2) systems for in-plane and out-of-plane geometries revealed that solventborne liquid paint systems better preserve the appearance of original diffraction gratings. This is due to enhanced orientation of the anisotropic pigment particles. The powder coating surfaces investigated, on the other hand, preserved higher intensity and thus visibility in randomly oriented solitary flakes, creating a greater sparkle contrast. We confirmed our findings by capturing and visualizing coating appearance by means of a bidirectional texture function. We then compared the diffractive pigment evaluation results with other state-of-the-art measuring device readings. We believe that our work provides valuable information on flake orientation and also compares pigment performance in a range of industrial coating systems, which may enable industrial companies to improve paint spraying processes.

An efficient method for specular-enhanced BTF compression

The Bidirectional Texture Function (BTF) has been used in the data-driven model to recover the reflectance of complex surfaces. The high-dimensionality of the BTF causes huge measurement data, but that needs to be compressed when real-time rendering is required. However, the distribution of BTF data is highly non-uniform due to non-lambertian reflections. Projecting the measured data onto a linear space constructed by a statistical analysis such as PCA results in low-quality of data compression. In this paper, we propose a computationally efficient and robust data compression method. Based on the assumption that there exist many overlapping measurements, we seek to find reflectance holes caused by the non-lambertian reflection with a series of linear algorithms. Before performing the standard compression, our method separates diffuse reflections which can be well approximated with a linear model and compress them using a standard factorization. Specular components detected in the reflectance hole are separately stored using a sparse matrix representation to prevent the loss of the range of BTF data. Experimental results show that our method effectively improves the compression accuracy as well as the visual quality with a competitive computation time.

Rapid material capture through sparse and multiplexed measurements

Among the many models for material appearance, data-driven representations like bidirectional texture functions (BTFs) play an important role as they provide accurate real-time reproduction of complex light transport effects such as interreflections. However, their acquisition involves time-consuming capturing of many thousands of bidirectional samples in order to avoid interpolation artifacts. Furthermore, high dynamic range imaging including many and long exposure steps is necessary in the presence of low albedo or self-shadowing. So far, these problems have been dealt with separately by means of sparse reconstruction and multiplexed illumination techniques, respectively. Existing methods rely on data-driven models learned on data that has been range-reduced in a way that made their simultaneous application impossible. In this paper, we address both problems at once through a novel method for learning data-driven appearance models, based on moving the dynamic range reduction from the data to the metric. Specifically, we learn models by minimizing the relative L2 error on the original data instead of the absolute L2 error on range-reduced data. We demonstrate that the models thus obtained allow for faithful reconstruction of material appearance from sparse and illumination-multiplexed measurements, greatly reducing both the number of images and the shutter times required. As a result, we are able to reduce acquisition times down to the order of minutes from what used to be the order of hours.

Optomechanical design of rotary kaleidoscope for bidirectional texture function acquisition.

Optical systems are traditionally used for accurate recording and measurement of the real world's appearance. Present techniques allow us to form a computer-based virtual world, which is used in a variety of technical fields. The crucial issue for future applications of virtual reality is the fidelity of rendered images to real-world objects. This is strongly affected by the appearance of the rendered object's surfaces. Currently, the most applied method of describing a surface's visual appearance of spatially nonuniform surfaces is bidirectional texture function (BTF). We have designed, optimized, built, and tested a unique portable instrument based on a rotary kaleidoscope principle for BTF acquisition in situ. To the best of our knowledge, such an instrument has never been used before to measure BTF of a surface. We enhanced a common static kaleidoscope by adding rotation, which allows us to get a larger number of images of the sample for more combinations of illumination directions and viewing directions. This results in a higher directional and spatial resolution of measured BTF data. In this paper, we focus on the optomechanical design of the rotary BTF measurement instrument and issues related to its alignment to keep the desired mechanical precision.

Simulating painted appearance of BTF materials

Bidirectional texture function (BTF) provides a realistic representation for real-world materials; however, the representation does not allow users to append additional rendering effects to the BTF data in a simple and effective way. In this paper, we present an approach for generating the painted appearance on a material that is represented by a BTF. We first convert a BTF into a physical representation composed of a height-field and a spatially varying bidirectional reflectance distribution function (SVBRDF). This representation allows us to simulate the appearance of paints on the surface of a material by computing the reflectance and the height field of the painted material. During rendering, we combine an existing BTF rendering approach and the modified material data to simulate the painted appearance of the BTF materials. Our experiments show that the proposed approach can simulate the painted appearance of a material while preserving the characteristic appearance of the original BTF. Our approach is mainly suitable for less specular BTF materials that allow depth reconstruction.

Bidirectional texture function image super-resolution using singular value decomposition.

The bidirectional texture function (BTF) is widely employed to achieve realistic digital reproduction of real-world material appearance. In practice, a BTF measurement device usually does not use high-resolution (HR) cameras in data collection, considering the high equipment cost and huge data space required. The limited image resolution consequently leads to the loss of texture details in BTF data. This paper proposes a fast BTF image super-resolution (SR) algorithm to deal with this issue. The algorithm uses singular value decomposition (SVD) to separate the collected low-resolution (LR) BTF data into intrinsic textures and eigen-apparent bidirectional reflectance distribution functions (eigen-ABRDFs) and then improves the resolution of the intrinsic textures via image SR. The HR BTFs can be finally obtained by fusing the reconstructed HR intrinsic textures with the LR eigen-ABRDFs. Experimental results show that the proposed algorithm outperforms the state-of-the-art single-image SR algorithms in terms of reconstruction accuracy. In addition, thanks to the employment of SVD, the proposed algorithm is computationally efficient and robust to noise corruption.

Lightdrum-Portable Light Stage for Accurate BTF Measurement on Site.

We propose a miniaturised light stage for measuring the bidirectional reflectance distribution function (BRDF) and the bidirectional texture function (BTF) of surfaces on site in real world application scenarios. The main principle of our lightweight BTF acquisition gantry is a compact hemispherical skeleton with cameras along the meridian and with light emitting diode (LED) modules shining light onto a sample surface. The proposed device is portable and achieves a high speed of measurement while maintaining high degree of accuracy. While the positions of the LEDs are fixed on the hemisphere, the cameras allow us to cover the range of the zenith angle from 0 to 75 and by rotating the cameras along the axis of the hemisphere we can cover all possible camera directions. This allows us to take measurements with almost the same quality as existing stationary BTF gantries. Two degrees of freedom can be set arbitrarily for measurements and the other two degrees of freedom are fixed, which provides a tradeoff between accuracy of measurements and practical applicability. Assuming that a measured sample is locally flat and spatially accessible, we can set the correct perpendicular direction against the measured sample by means of an auto-collimator prior to measuring. Further, we have designed and used a marker sticker method to allow for the easy rectification and alignment of acquired images during data processing. We show the results of our approach by images rendered for 36 measured material samples.

Optomechanical design of a portable compact bidirectional texture function measurement instrument.

Recent developments in optoelectronics and material processing techniques make it possible to design and produce a portable and compact measurement instrument for bidirectional texture function (BTF). Parallelized optics, on-board data processing, rapid prototyping, and other nonconventional production techniques and materials were the key to building an instrument capable of in situ measurements with fast data acquisition. We designed, built, and tested a prototype of a unique portable and compact multi-camera system for BTF measurement which is capable of in situ measurement of temporally unstable samples. In this paper, we present its optomechanical design.

A Normal Vector and BTF Profile Measurement System Using a Correlation Camera and Scanning Dome Illumination

A system for simultaneously measuring surface normal vectors and BTF (bidirectional texture function) profiles of an object is proposed. This system consists of a correlation camera and a spherical scanning light source. The system can compress BTF measurement in one dimension of incidence direction to a single image frame, as well as measure a profile of the BTF under the scanning light source just from a single image frame. Experimental results on an implemented system demonstrate the feasibility of the proposed system.

Spectral bidirectional texture function reconstruction by fusing multiple-color and spectral images.

Spectral bidirectional texture function (BTF) is essential for accurate reproduction of material appearance due to its nature of conveying both spatial and spectral information. A practical issue is that the acquisition of raw spectral BTFs is time-consuming. To resolve the limitation, this paper proposes a novel framework for efficient spectral BTF acquisition and reconstruction. The framework acquires red-green-blue (RGB) BTF images and just one spectral image. The full spectral BTFs are reconstructed by fusing the RGB and spectral images based on nonnegative matrix factorization (NMF). Experimental results indicate that the accuracy of spectral reflectance reconstruction is higher than that of existing algorithms. With the reconstructed spectral BTFs, the material appearance can be reproduced with high fidelity under various illumination conditions.