Advanced textural representation of materials appearance

Abstract

Multidimensional visual texture is the appropriate paradigm for physically correct material visual properties representation. The course will present recent advances in texture modelling methodology as applied in computer vision, pattern recognition, computer graphics, and virtual/augmented reality applications. Contrary to previous courses on material appearance, we will focus on materials whose nature allows the exploitation of texture modeling approaches. This topic is introduced in the wider and complete context of pattern recognition and image processing. It comprehends modeling of multi-spectral images and videos which can be accomplished either with multi-dimensional mathematical models or sophisticated sampling methods from the original measurements. The key aspects of the topic, i.e., different multi-dimensional data models with their corresponding benefits and drawbacks, optimal model selection, parameter estimation and model synthesis techniques, are discussed. These methods produce compact parametric sets that not only faithfully reproduce material appearance, but are also vital for visual scene analysis, e.g. texture segmentation, classification, and retrieval. Special attention is devoted to recent advanced trends towards Bidirectional Texture Function (BTF) modeling, used for materials that do not obey Lambertian law, and whose reflectance has non-trivial illumination and viewing direction dependency. BTFs represent the best known effectively applicable textural representation of most real-world materials' visual properties. The techniques covered include efficient Markov random field-based algorithms, intelligent sampling algorithms, spatially-varying reflectance models and challenges with their possible implementation on GPU. The course also deals with proper data measurement, visualization of texture models in virtual scenes, visual quality evaluation feedback, as well as description of key industrial and research applications. We will discuss options as to which type of material representation is appropriate for required application, what are its limits and possible modelling options, and what the biggest challenges in realistic modelling of materials are.