Texture Representation Research Articles

Compact Hybrid Multi-Color Space Descriptor Using Clustering-Based Feature Selection for Texture Classification.

Color texture classification aims to recognize patterns by the analysis of their colors and their textures. This process requires using descriptors to represent and discriminate the different texture classes. In most traditional approaches, these descriptors are used with a predefined setting of their parameters and computed from images coded in a chosen color space. The prior choice of a color space, a descriptor and its setting suited to a given application is a crucial but difficult problem that strongly impacts the classification results. To overcome this problem, this paper proposes a color texture representation that simultaneously takes into account the properties of several settings from different descriptors computed from images coded in multiple color spaces. Since the number of color texture features generated from this representation is high, a dimensionality reduction scheme by clustering-based sequential feature selection is applied to provide a compact hybrid multi-color space (CHMCS) descriptor. The experimental results carried out on five benchmark color texture databases with five color spaces and manifold settings of two texture descriptors show that combining different configurations always improves the accuracy compared to a predetermined configuration. On average, the CHMCS representation achieves 94.16% accuracy and outperforms deep learning networks and handcrafted color texture descriptors by over 5%, especially when the dataset is small.

DFAEN: Double-order knowledge fusion and attentional encoding network for texture recognition

Recent studies have shown that deep convolutional neural networks (CNNs) have been successfully used for texture representation and recognition. One of the most successful texture recognition methods is the deep texture encoding network (DeepTEN), which has been shown to be effective. However, this network directly uses redundant CNN features with generality and ignores the role of multiorder information during the encoding and learning processes. To address these issues, this paper proposes a double-order knowledge fusion and attentional encoding network for texture recognition (DFAEN). First, crucial texture features are encoded by an embedded attention mechanism. Second, double-order modeling is implemented in the encoding and learning stage to make full use of convolution feature information with different orders, enabling the network to focus on and learn more texture domain information. Our method can stably and effectively perform end-to-end optimization. Evaluation experiments conducted on several widely used benchmark datasets (e.g., the FMD, MINC-2500, the DTD, KTH-TISP-2b, and GTOS-mobile) show that our method clearly demonstrates superior performance to that of competing approaches.

POp: Parameter Optimization of Differentiable Vector Patterns

AbstractProcedural materials are extensively used in computer graphics, since they provide editable, resolution‐independent representation of textures. However, tuning the parameters of procedural generators to achieve a desired result remains time‐consuming for users. Recently, inverse procedural material algorithms have been developed, exploiting differentiable rendering methods to find the parameters of a procedural model that match a target image. These approaches focus on raster textures. We propose pOp, a practical method for estimating the parameters of vector patterns, that are formed by collections of vector shapes arranged by an arbitrary procedural program. In our approach, patterns are defined as arbitrary programs, that control the translation, rotation and scale or vector graphics elements. We support elements typical of vector graphics, namely points, lines, circle, rounded rectangles, and quadratic Bèzier drawings, in multiple colors. We optimize the program parameters by automatically differentiating the signed distance field of the drawing, which we found to be significantly more reliable than using differentiable rendering of the final image. We demonstrate our method on a variety of cases, representing the variations found in structured vector patterns.

Text2Human

Generating high-quality and diverse human images is an important yet challenging task in vision and graphics. However, existing generative models often fall short under the high diversity of clothing shapes and textures. Furthermore, the generation process is even desired to be intuitively controllable for layman users. In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation. We synthesize full-body human images starting from a given human pose with two dedicated steps. 1) With some texts describing the shapes of clothes, the given human pose is first translated to a human parsing map. 2) The final human image is then generated by providing the system with more attributes about the textures of clothes. Specifically, to model the diversity of clothing textures, we build a hierarchical texture-aware codebook that stores multi-scale neural representations for each type of texture. The codebook at the coarse level includes the structural representations of textures, while the codebook at the fine level focuses on the details of textures. To make use of the learned hierarchical codebook to synthesize desired images, a diffusion-based transformer sampler with mixture of experts is firstly employed to sample indices from the coarsest level of the codebook, which then is used to predict the indices of the codebook at finer levels. The predicted indices at different levels are translated to human images by the decoder learned accompanied with hierarchical codebooks. The use of mixture-of-experts allows for the generated image conditioned on the fine-grained text input. The prediction for finer level indices refines the quality of clothing textures. Extensive quantitative and qualitative evaluations demonstrate that our proposed Text2Human framework can generate more diverse and realistic human images compared to state-of-the-art methods. Our project page is https://yumingj.github.io/projects/Text2Human.html. Code and pretrained models are available at https://github.com/yumingj/Text2Human.

Application of a Texture Appearance Model for Segmentation of Lung Nodules on Computed Tomography of the Chest

Introduction. Lung cancer is one of the most critical diseases globally, with more than 1.6 million new cases registered every year. Early detection of lung cancer is essential; therefore, particular attention should be paid to the development of effective diagnostic and therapeutic procedures. Computer processing of CT scans in the course of lung cancer diagnostics involves the following stages: medical image acquisition, pre-processing of medical images, segmentation, and false-positive reduction. Since segmentation is an essential stage in the process of medical image analysis, the development of novel segmentation approaches is attracting much research interest. Model-based segmentation approaches have recently gained in popularity, largely due to their potential to restore lost information.Aim. To apply a texture appearance model for the segmentation of pulmonary nodules on computed tomography of the chest.Materials and methods. A novel model-based Texture Appearance Model (TAM) is proposed for precise and effective segmentation of all sorts of nodule regions. We taught the TAM for segmentation of a lung nodule in lung CT images using a combination of extracted texture characteristics from CT scans and Texture Representation of Image (TRI).Results. The results of applying the described TAM method to normal and noisy CT images are presented and compared to those obtained using the Region Growing and Active Contour algorithms, as well as the combination of Active Contour and Watershed algorithms. The TAM was tested in 85 nodules from a dataset, yielding an average dice similarity coefficient (DSC) of 84.75 percent.Conclusion. A novel method for segmenting nodules in the lung, which is capable of segmenting all forms of nodules with excellent accuracy, is proposed. This model-based technique, when used with the active loop algorithm, can enhance accuracy and decrease false positives by selecting the initial mask. The precision, dice, accuracy, and specificity of lung nodule segmentation on a normal CT scan are 85.5, 85, 96, and 98, which levels are superior to those produced by the Active Contour, Region Growing and the combination of Active Contour and Watershed algorithms.

Dimensionality of the intermediate-level representation of shape and texture in monkey V4

The visual area V4 has been considered to play a crucial role in the intermediate representation of objects, where low-level image features are transformed into object-level representations. We estimated the intrinsic dimensionality in V4 for the representation of local patches generated from natural scenes. The dimensionality was approximately 40, which is approximately half of that reported in IT for the representation of whole natural objects. The analyses of the estimated dimensionality suggest both common and independent representations that code contour shapes and/or surfaces with textures, implying a relatively complex and mixed representation in the intermediate-level area.

The representation of shape and texture in category-selective regions of ventral-temporal cortex.

Neuroimaging studies using univariate and multivariate approaches have shown that the fusiform face area (FFA) and parahippocampal place area (PPA) respond selectively to images of faces and places. The aim of this study was to determine the extent to which this selectivity to faces or places is based on the shape or texture properties of the images. Faces and houses were filtered to manipulate their texture properties, while preserving the shape properties (spatial envelope) of the images. In Experiment 1, multivariate pattern analysis (MVPA) showed that patterns of fMRI response to faces and houses in FFA and PPA were predicted by the shape properties, but not by the texture properties of the image. In Experiment 2, a univariate analysis (fMR‐adaptation) showed that responses in the FFA and PPA were sensitive to changes in both the shape and texture properties of the image. These findings can be explained by the spatial scale of the representation of images in the FFA and PPA. At a coarser scale (revealed by MVPA), the neural selectivity to faces and houses is sensitive to variation in the shape properties of the image. However, at a finer scale (revealed by fMR‐adaptation), the neural selectivity is sensitive to the texture properties of the image. By combining these neuroimaging paradigms, our results provide insights into the spatial scale of the neural representation of faces and places in the ventral‐temporal cortex.

Using global information to refine local patterns for texture representation and classification

Local binary pattern (LBP) and its variants have been successfully applied in texture feature extraction. However, it is hard for most LBP-based methods to effectively describe and distinguish the local neighborhoods with similar structures (that is, the calculated feature patterns are identical) but different contrasts or grayscales. To alleviate such problems, we propose a novel global refined local binary pattern (GRLBP) by analyzing the nature of pixel intensity distribution in local neighborhoods. GRLBP consists of two descriptors called magnitude refined local sign binary pattern (MRLBP_S) and center refined local magnitude binary pattern (CRLBP_M). MRLBP_S distinguishes local neighborhoods with contrast differences by using global magnitude anchors to refine local sign patterns. And CRLBP_M identifies local neighborhoods with grayscale differences by employing global central grayscale anchors to refine local magnitude patterns. Finally, frequency histograms of MRLBP_S and CRLBP_M from each image are cascaded to generate the GRLBP. Extensive experimental results on seven benchmark texture databases: Outex, CUReT, KTH-TIPS, UMD, UIUC, KTH-T2b, and DTD demonstrate that the proposed GRLBP can represent the detailed information of texture images. Furthermore, compared with state-of-the-art LBP variants, GRLBP has competitive advantages in classification accuracy, feature dimension, and computational complexity, respectively.

Local ternary pattern based multi-directional guided mixed mask (MDGMM-LTP) for texture and material classification

Local feature descriptors (LFDs) have been researched and developed for decades, and their applications are still being examined nowadays. In addition, the fact that LFDs are significant in a variety of computer vision applications and have excellent performances that should be improved, especially in challenging conditions, is the key reason for the ongoing work in this area. Therefore, we design in this paper a novel model of local feature descriptor for texture representation. The essence of the proposed method, referred to as multi directional guided mixed mask based local ternary pattern (MDGMM-LTP), is to progressively extract comprehensive micro structure features by analyzing the differential excitation and directional information according to relationships between pixels sampled in a variety of spatial arrangements within each 3 × 3 neighborhood. The process of MDGMM-LTP consists first in constructing a new mixed hybrid directional kernel based on a combination of several edge directional kernels in eight directions, including Kirsch, Robinson, Prewitt and Frei–Chen masks and then both local and non-local pixel interactions are encoded according to a new compact pattern encoding scheme integrating both LTP’s and LDP’s concepts. MDGMM-LTP operator has good ability to capture stable and more discriminant micro structure information through complementary information resulting from both LTP’s and LDP’s encoders’ combination. With the help of the K-Nearest Neighbors (KNN) classifier, the experiments conducted on sixteen challenging texture datasets revealed that the proposed MDGMM-LTP descriptor outperforms the other commonly used LFDs-based and deep learning-based techniques in terms of overall performance. To further show the effectiveness of the proposed method, a statistical experiment applied over all the tested datasets through the Wilcoxon signed rank test is performed, which statistically validates the efficacy of the proposed method.

A shallow extraction of texture features for classification of abnormal video endoscopy frames

Automated analysis of the gastric lesions in endoscopy videos is a challenging task and dynamics of the gastrointestinal environment make it even more difficult. In computer-aided diagnosis, gastric images are analyzed by visual descriptors. Various Deep Convolutional Neural Network (DCNN) models are available for representation learning and classification. In this paper, a computer aided diagnosis system is presented for the classification of abnormalities in Videos Endoscopy (VE) images based on Deep Gray-Level Co-occurrence Matrix (DeepGLCM) texture features. In our scheme, the convolutional layers of an already trained model are employed for acquisition of the statistical features from responses of filters to estimate the texture representation of VE frames. A learning model is trained on these features for gastric frames classification. The results obtained by using public datasets of endoscopy images to calculate the performance of the proposed method. In addition, we also use a private endoscopy dataset which is acquired from the University of Aveiro. The DeepGLCM outperforms by achieving the average accuracy of ≈92% and 0.96 area under the curve (AUC) for the chromoendoscopy (CH) dataset and ≈85% accuracy for Confocal Laser Endomicroscopy (CLE) and white light video endoscopy datasets. It is evident that the DeepGLCM texture features provide a better representation than the traditional texture extraction methods by efficiently dealing with variance in images due to different imaging technologies.

Selective representations of texture and motion in mouse higher visual areas

The mouse visual cortex contains interconnected higher visual areas, but their functional specializations are unclear. Here, we used a data-driven approach to examine the representations of complex visual stimuli by L2/3 neurons across mouse higher visual areas, measured using large-field-of-view two-photon calcium imaging. Using specialized stimuli, we found higher fidelity representations of texture in area LM, compared to area AL. Complementarily, we found higher fidelity representations of motion in area AL, compared to area LM. We also observed this segregation of information in response to naturalistic videos. Finally, we explored how receptive field models of visual cortical neurons could produce the segregated representations of texture and motion we observed. These selective representations could aid in behaviors such as visually guided navigation.

Weakly supervised learning of multi-object 3D scene decompositions using deep shape priors

Representing scenes at the granularity of objects is a prerequisite for scene understanding and decision making. We propose PriSMONet, a novel approach based on Prior Shape knowledge for learning Multi-Object 3D scene decomposition and representations from single images. Our approach learns to decompose images of synthetic scenes with multiple objects on a planar surface into its constituent scene objects and to infer their 3D properties from a single view. A recurrent encoder regresses a latent representation of 3D shape, pose and texture of each object from an input RGB image. By differentiable rendering, we train our model to decompose scenes from RGB-D images in a self-supervised way. The 3D shapes are represented continuously in function-space as signed distance functions which we pre-train from example shapes in a supervised way. These shape priors provide weak supervision signals to better condition the challenging overall learning task. We evaluate the accuracy of our model in inferring 3D scene layout, demonstrate its generative capabilities, assess its generalization to real images, and point out benefits of the learned representation.

Artificial SA-I, RA-I and RA-II/vibrotactile afferents for tactile sensing of texture.

Robot touch can benefit from how humans perceive tactile textural information, from the stimulation mode to which tactile channels respond, then the tactile cues and encoding. Using a soft biomimetic tactile sensor (the TacTip) based on the physiology of the dermal-epidermal boundary, we construct two biomimetic tactile channels based on slowly adapting SA-I and rapidly adapting RA-I afferents, and introduce an additional sub-modality for vibrotactile information with an embedded microphone interpreted as an artificial RA-II channel. These artificial tactile channels are stimulated dynamically with a set of 13 artificial rigid textures comprising raised-bump patterns on a rotating drum that vary systematically in roughness. Methods employing spatial, spatio-temporal and temporal codes are assessed for texture classification insensitive to stimulation speed. We find: (i) spatially encoded frictional cues provide a salient representation of texture; (ii) a simple transformation of spatial tactile features to model natural afferent responses improves the temporal coding; and (iii) the harmonic structure of induced vibrations provides a pertinent code for speed-invariant texture classification. Just as human touch relies on an interplay between slowly adapting (SA-I), rapidly adapting (RA-I) and vibrotactile (RA-II) channels, this tripartite structure may be needed for future robot applications with human-like dexterity, from prosthetics to materials testing, handling and manipulation.

Renal Cancer Detection: Fusing Deep and Texture Features from Histopathology Images.

Histopathological images contain morphological markers of disease progression that have diagnostic and predictive values, with many computer-aided diagnosis systems using common deep learning methods that have been proposed to save time and labour. Even though deep learning methods are an end-to-end method, they perform exceptionally well given a large dataset and often show relatively inferior results for a small dataset. In contrast, traditional feature extraction methods have greater robustness and perform well with a small/medium dataset. Moreover, a texture representation-based global approach is commonly used to classify histological tissue images expect in explicit segmentation to extract the structure properties. Considering the scarcity of medical datasets and the usefulness of texture representation, we would like to integrate both the advantages of deep learning and traditional machine learning, i.e., texture representation. To accomplish this task, we proposed a classification model to detect renal cancer using a histopathology dataset by fusing the features from a deep learning model with the extracted texture feature descriptors. Here, five texture feature descriptors from three texture feature families were applied to complement Alex-Net for the extensive validation of the fusion between the deep features and texture features. The texture features are from (1) statistic feature family: histogram of gradient, gray-level cooccurrence matrix, and local binary pattern; (2) transform-based texture feature family: Gabor filters; and (3) model-based texture feature family: Markov random field. The final experimental results for classification outperformed both Alex-Net and a singular texture descriptor, showing the effectiveness of combining the deep features and texture features in renal cancer detection.

Universal encoding of pan-cancer histology by deep texture representations.

Cancer histological images contain rich biological and clinical information, but quantitative representation can be problematic and has prevented the direct comparison and accumulation of large-scale datasets. Here, we show successful universal encoding of cancer histology by deep texture representations (DTRs) produced by a bilinear convolutional neural network. DTR-based, unsupervised histological profiling, which captures the morphological diversity, is applied to cancer biopsies and reveals relationships between histologic characteristics and the response to immune checkpoint inhibitors (ICIs). Content-based image retrieval based on DTRs enables the quick retrieval of histologically similar images using The Cancer Genome Atlas (TCGA) dataset. Furthermore, via comprehensive comparisons with driver and clinically actionable gene mutations, we successfully predict 309 combinations of genomic features and cancer types from hematoxylin-and-eosin-stained images. With its mounting capabilities on accessible devices, such as smartphones, universal encoding for cancer histology has a strong impact on global equalization for cancer diagnosis and therapies.

Fabric Classification Using a Finger-Shaped Tactile Sensor via Robotic Sliding.

Tactile sensing endows the robots to perceive certain physical properties of the object in contact. Robots with tactile perception can classify textures by touching. Interestingly, textures of fine micro-geometry beyond the nominal resolution of the tactile sensors can also be identified through exploratory robotic movements like sliding. To study the problem of fine texture classification, we design a robotic sliding experiment using a finger-shaped multi-channel capacitive tactile sensor. A feature extraction process is presented to encode the acquired tactile signals (in the form of time series) into a low dimensional (≤7D) feature vector. The feature vector captures the frequency signature of a fabric texture such that fabrics can be classified directly. The experiment includes multiple combinations of sliding parameters, i.e., speed and pressure, to investigate the correlation between sliding parameters and the generated feature space. Results show that changing the contact pressure can greatly affect the significance of the extracted feature vectors. Instead, variation of sliding speed shows no apparent effects. In summary, this paper presents a study of texture classification on fabrics by training a simple k-NN classifier, using only one modality and one type of exploratory motion (sliding). The classification accuracy can reach up to 96%. The analysis of the feature space also implies a potential parametric representation of textures for tactile perception, which could be used for the adaption of motion to reach better classification performance.

Conceptual Compression via Deep Structure and Texture Synthesis.

Existing compression methods typically focus on the removal of signal-level redundancies, while the potential and versatility of decomposing visual data into compact conceptual components still lack further study. To this end, we propose a novel conceptual compression framework that encodes visual data into compact structure and texture representations, then decodes in a deep synthesis fashion, aiming to achieve better visual reconstruction quality, flexible content manipulation, and potential support for various vision tasks. In particular, we propose to compress images by a dual-layered model consisting of two complementary visual features: 1) structure layer represented by structural maps and 2) texture layer characterized by low-dimensional deep representations. At the encoder side, the structural maps and texture representations are individually extracted and compressed, generating the compact, interpretable, inter-operable bitstreams. During the decoding stage, a hierarchical fusion GAN (HF-GAN) is proposed to learn the synthesis paradigm where the textures are rendered into the decoded structural maps, leading to high-quality reconstruction with remarkable visual realism. Extensive experiments on diverse images have demonstrated the superiority of our framework with lower bitrates, higher reconstruction quality, and increased versatility towards visual analysis and content manipulation tasks.

C2CL: Contact to Contactless Fingerprint Matching

Matching contactless fingerprints or finger photos to contact-based fingerprint impressions has received increased attention in the wake of COVID-19 due to the superior hygiene of the contactless acquisition and the widespread availability of low cost mobile phones capable of capturing photos of fingerprints with sufficient resolution for verification purposes. This paper presents an end-to-end automated system, called C2CL, comprised of a mobile finger photo capture app, preprocessing, and matching algorithms to handle the challenges inhibiting previous cross-matching methods; namely i) low ridge-valley contrast of contactless fingerprints, ii) varying roll, pitch, yaw, and distance of the finger to the camera, iii) non-linear distortion of contact-based fingerprints, and vi) different image qualities of smartphone cameras. Our preprocessing algorithm segments, enhances, scales, and unwarps contactless fingerprints, while our matching algorithm extracts both minutiae and texture representations. A sequestered dataset of 9, 888 contactless 2D fingerprints and corresponding contact-based fingerprints from 206 subjects (2 thumbs and 2 index fingers for each subject) acquired using our mobile capture app is used to evaluate the cross-database performance of our proposed algorithm. Furthermore, additional experimental results on 3 publicly available datasets show substantial improvement in the state-of-the-art for contact to contactless fingerprint matching (TAR in the range of 96.67% to 98.30% at FAR=0.01%).

Deep Learning in Computer Real-time Graphics and Image Using the Visual Effects of Non-photorealistic Rendering of Ink Painting

This study is to explore the effect of deep learning algorithms applied to the non-photorealistic rendering (NPR) of ink painting. The powerful processing capabilities of NPR technology and convolutional neural networks (CNN) in computer vision are analyzed. Aiming at the features of ink texture images, a new sample-based ink texture synthesis method is proposed. The feature representation of ink texture is calculated using the CNN model, and then an ink texture with the same feature representation is generated on a random image. In addition, a synthesis method of ink painting picture is proposed based on the CNN. The results show that the ink texture generated by the ink texture synthesis algorithm is very good, and the Visual Geometry Group Network (VGG-Net) model shows the best generation effect. For images generated by high-level networks lack a lot of pixel information, but retain the main feature information of the image. When the learning rate is 10, the effect of the ink image synthesized based on the content image is slightly worse; and the image generated by the fusion of multiple styles confirms the feasibility of synthesizing ink images of different "sizes and styles". Therefore, this ink image synthesized on the basis of the CNN model shows a better effect, which extends the possibility of computer creation of richer ink painting works.

Реконструкція 3D-моделі обличчя з 2D-зображень за допомогою нейронних мереж

The most common methods of reconstruction of 3D-models of the face are considered, their quantitative estimates are analyzed and determined, the most promising approach is highlighted - 3D Morphable Model. The necessity of its modification in order to improve the results of reconstruction based on the analysis of the main components and the use of generative-competitive neural network is substantiated. One of the advantages of using the 3D Morphable Model with principal component analysis is to present only a plausible solution when the solution space is limited, which simplifies the problem to be solved. Whereas the original approach involves manual initialization. It is planned to use the generative-competitive neural network on high-resolution UV maps as a statistical representation of facial texture. In this way, you can reconstruct textures with high-frequency details. The main result is an approach to creating three-dimensional models of faces from their two-dimensional images that have the least time and a satisfactory standard error. The tasks of further research are determined.