Statistical Texture Model Research Articles

Multi-view self-supervised learning for 3D facial texture reconstruction from single image

Recent years witnessed that deep learning based methods have achieved significant progresses in recovering 3D face shape from single image. However, reconstructing realistic 3D facial texture from single image is still a challenging task due to the unavailability of large-scale training datasets and the low expression ability of previous statistical texture models (e.g. 3DMM). In this paper, we introduce a novel deep architecture trained by self-supervision with multi-view setup, to reconstruct 3D facial texture. Specifically, we first obtain incomplete UV texture map from input facial image, and then introduce a Texture Completion Network (TC-Net) to inpaint missing areas. To train TC-Net, firstly, we collect 50,000 triplets of facial images from in-the-wild videos, each triplet consists of a nearly frontal, a left-side, and a right-side facial images. With this dataset, we propose a novel multi-view consistency loss that ensures consistent photometric, face identity, 3DMM identity, and UV texture among multi-view facial images. This loss allows to optimize TC-Net in a self-supervision way without using ground-truth texture map as supervision. In addition, multi-view input images are only required in training to provide self-supervision, and our method only needs single input image in inference. Extensive experiments show that our method achieves state-of-the-art performance in both qualitative and quantitative comparisons.

3D Reconstruction of "In-the-Wild" Faces in Images and Videos.

3D Morphable Models (3DMMs) are powerful statistical models of 3D facial shape and texture, and are among the state-of-the-art methods for reconstructing facial shape from single images. With the advent of new 3D sensors, many 3D facial datasets have been collected containing both neutral as well as expressive faces. However, all datasets are captured under controlled conditions. Thus, even though powerful 3D facial shape models can be learnt from such data, it is difficult to build statistical texture models that are sufficient to reconstruct faces captured in unconstrained conditions ("in-the-wild"). In this paper, we propose the first "in-the-wild" 3DMM by combining a statistical model of facial identity and expression shape with an "in-the-wild" texture model. We show that such an approach allows for the development of a greatly simplified fitting procedure for images and videos, as there is no need to optimise with regards to the illumination parameters. We have collected three new benchmarks that combine "in-the-wild" images and video with ground truth 3D facial geometry, the first of their kind, and report extensive quantitative evaluations using them that demonstrate our method is state-of-the-art.

Neutral face classification using personalized appearance models for fast and robust emotion detection.

Facial expression recognition is one of the open problems in computer vision. Robust neutral face recognition in real time is a major challenge for various supervised learning-based facial expression recognition methods. This is due to the fact that supervised methods cannot accommodate all appearance variability across the faces with respect to race, pose, lighting, facial biases, and so on, in the limited amount of training data. Moreover, processing each and every frame to classify emotions is not required, as user stays neutral for majority of the time in usual applications like video chat or photo album/web browsing. Detecting neutral state at an early stage, thereby bypassing those frames from emotion classification would save the computational power. In this paper, we propose a light-weight neutral versus emotion classification engine, which acts as a pre-processer to the traditional supervised emotion classification approaches. It dynamically learns neutral appearance at key emotion (KE) points using a statistical texture model, constructed by a set of reference neutral frames for each user. The proposed method is made robust to various types of user head motions by accounting for affine distortions based on a statistical texture model. Robustness to dynamic shift of KE points is achieved by evaluating the similarities on a subset of neighborhood patches around each KE point using the prior information regarding the directionality of specific facial action units acting on the respective KE point. The proposed method, as a result, improves emotion recognition (ER) accuracy and simultaneously reduces computational complexity of the ER system, as validated on multiple databases.

Maximum-likelihood based synthesis of volumetric textures from a 2D sample.

We propose a genuine 3D texture synthesis algorithm based on a probabilistic 2D Markov random field conceptualization, capable of capturing the visual characteristics of a texture into a unique statistical texture model. We intend to reproduce, in the volumetric texture, the interactions between pixels learned in an input 2D image. The learning is done by nonparametric Parzen-windowing. Optimization is handled voxel by a relaxation algorithm, aiming at maximizing the likelihood of each voxel in terms of its local conditional probability function. Variants are proposed regarding the relaxation algorithm and the heuristic strategies used for the simultaneous handling of the orthogonal slices containing the voxel. The procedures are materialized on various textures through a comparative study and a sensitivity analysis, highlighting the variants strengths and weaknesses. Finally, the probabilistic model is compared objectively with a nonparametric neighborhood-search-based algorithm.

Statistical Texture Model for mass Detection in Mammography

En el contexto de procesamiento de imagenes para deteccion de masas en mamografia, la textura es una caracteristica clave para distinguir tejido anormal del normal. Recientemente un modelo de textura basado en mezcla de gaussianas multivariadas fue propuesto. El modelo produce imagenes 􀁔ue son mapas probabilisticos de la normalidad de la textura y fue propuesto como una ayuda de visualizacion para diagnostico por medicos expertos. En este articulo la usabilidad de dicho modelo es estudiada para deteccion automatica de masas. Una estrategia de segmentacion es propuesta y evaluada en 79 casos de mamografia

Computer-Aided Detection and Diagnosis in Medical Imaging

Computer-Aided Detection and Diagnosis in Medical Imaging

A Statistical Texture Model of the Liver Based on Generalized N-Dimensional Principal Component Analysis (GND-PCA) and 3D Shape Normalization

We present a method based on generalized N-dimensional principal component analysis (GND-PCA) and a 3D shape normalization technique for statistical texture modeling of the liver. The 3D shape normalization technique is used for normalizing liver shapes in order to remove the liver shape variability and capture pure texture variations. The GND-PCA is used to overcome overfitting problems when the training samples are too much fewer than the dimension of the data. The preliminary results of leave-one-out experiments show that the statistical texture model of the liver built by our method can represent an untrained liver volume well, even though the mode is trained by fewer samples. We also demonstrate its potential application to classification of normal and abnormal (with tumors) livers.

Linking statistical texture models to population coding in the ventral stream

Linking statistical texture models to population coding in the ventral stream

Development and testing of a statistical texture model for land cover classification of the Black Sea region with MODIS imagery

A statistical model is proposed for analysis of the texture of land cover types for global and regional land cover classification by using texture features extracted by multiresolution image analysis techniques. It consists of four novel indices representing second-order texture, which are calculated after wavelet decomposition of an image and after texture extraction by a new approach that makes use of a four-pixel texture unit. The model was applied to four satellite images of the Black Sea region, obtained by Terra/MODIS and Aqua/MODIS at different spatial resolution. In single texture classification experiments, we used 15 subimages (50 × 50 pixels) of the selected classes of land covers that are present in the satellite images studied. These subimages were subjected to one-level and two-level decompositions by using orthonormal spline and Gabor-like spline wavelets. The texture indices were calculated and used as feature vectors in the supervised classification system with neural networks. The testing of the model was based on the use of two kinds of widely accepted statistical texture quantities: five texture features determined by the co-occurrence matrix (angular second moment, contrast, correlation, inverse difference moment, entropy), and four statistical texture features determined after the wavelet transformation (mean, standard deviation, energy, entropy). The supervised neural network classification was performed and the discrimination ability of the proposed texture indices was found comparable with that for the sets of five GLCM texture features and four wavelet-based texture features. The results obtained from the neural network classifier showed that the proposed texture model yielded an accuracy of 92.86% on average after orthonormal wavelet decomposition and 100% after Gabor-like wavelet decomposition for texture classification of the examined land cover types on satellite images.

Computer-Aided Evaluation of Screening Mammograms Based on Local Texture Models

We propose a new approach to diagnostic evaluation of screening mammograms based on local statistical texture models. The local evaluation tool has the form of a multivariate probability density of gray levels in a suitably chosen search window. First, the density function in the form of Gaussian mixture is estimated from data obtained by scanning of the mammogram with the search window. Then we evaluate the estimated mixture at each position and display the corresponding log-likelihood value as a gray level at the window center. The resulting log-likelihood image closely correlates with the structural details of the original mammogram and emphasizes unusual places. We assume that, in parallel use, the log-likelihood image may provide additional information to facilitate the identification of malignant lesions as atypical locations of high novelty.