Color Texture Images Research Articles

Perception of Tactile Attributes of Colored Texture Images

Perception of Tactile Attributes of Colored Texture Images

A light-weight skeleton human action recognition model with knowledge distillation for edge intelligent surveillance applications

Skeleton based human action recognition has evolved as one of the most important applications in multimedia IoT system. However, it requires extensive computation resource including high performance computing unites and large memory to train a deep mode with large number of parameters, which seriously limits it effectiveness and efficiency for edge intelligence multimedia IoT applications. In this paper, a knowledge distillation based light-weight deep model is proposed for skeleton human action recognition to meet the edge multimedia IoT applications. It can get competitive recognition performance in terms of learning accuracy for combination of AI model and edge surveillance equipment. On the one hand, to achieve desirable accuracy, we propose a deep pose-transition image representation method based on two-stream spatial–temporal architecture, which can mine the hidden features of color texture images in spatial and temporal domain, and fuse them for comprehensive discrimination before final classification. On the other hand, to increase the transfer learning ability to the student model on the edge device, we use tucker decomposition to weak the teacher model during knowledge transfer learning process. Finally, in order to validate the effectiveness of our proposal, we conducted extensive experiments to evaluate the proposed approach. The experimental results demonstrate that our proposal can realize deep model miniaturization to meet the requirement of edge multimedia IoT system and achieve the competitive performance.

RGB pixel n-grams: A texture descriptor

This article proposes the “RGB Pixel N-grams” descriptor, which uses a sequence of n pixels to represent RGB color texture images. We conducted classification experiments with three different classifiers and five color texture image databases to evaluate the descriptor’s performance, using accuracy as the evaluation metric. These databases include various textures from different surfaces, sometimes under different lighting, scale, or rotation conditions. The proposed descriptor proved to be robust and competitive compared to other state-of-the-art descriptors, as it has better accuracy in classification results in most databases and classifiers.

Exploring multivariate generalized gamma manifold for color texture retrieval

This work proposes a novel method for color-textured image retrieval on a Multivariate Generalized Gamma Distribution manifold (MGΓD). Thanks to the Gaussian copula theory, we define the expression of MGΓD, which efficiently models the statistical dependence structure between dual-tree complex wavelet transform (DTCWT) of the color components. The major contribution of this paper is to provide a geometric perspective to the MGΓD by treating it as a Riemannian manifold while proposing the geodesic distance (GD) as a measure of Riemannian similarity on it. Based on information geometry tools, we conduct a geometrical study of the MGΓD manifold, allowing us to derive two suitable approximations of the GD. The experiments are performed on five well-known color texture databases, considering the content-based image retrieval (CBIR) framework and using the RGB color space. The obtained results demonstrate the efficiency of the geometric interpretation through the proposed GD as a natural and intuitive similarity measure on the studied statistical manifold.

Mixture of multivariate generalized Gaussians for multi-band texture modeling and representation

We present a unified statistical model for multivariate and multi-modal texture representation. This model is based on the formalism of finite mixtures of multivariate generalized Gaussians (MoMGG) which enables to build a compact and accurate representation of texture images using multi-resolution texture transforms. The MoMGG model enables to describe the joint statistics of subbands in different scales and orientations, as well as between adjacent locations within the same subband, providing a precise description of the texture layout. It can also combine different multi-scale transforms to build a richer and more representative texture signature for image similarity measurement. We tested our model on both traditional texture transforms (e.g., wavelets, contourlets, maximum response filter) and convolution neural networks (CNNs) features (e.g., ResNet, SqueezeNet). Experiments on color-texture image retrieval have demonstrated the performance of our approach comparatively to state-of-the-art methods.

Seamless Texture Optimization for RGB-D Reconstruction.

Restoring high-fidelity textures for 3D reconstructed models are an increasing demand in AR/VR, cultural heritage protection, entertainment, and other relevant fields. Due to geometric errors and camera pose drifting, existing texture mapping algorithms are either plagued by blurring and ghosting or suffer from undesirable visual seams. In this paper, we propose a novel tri-directional similarity texture synthesis method to eliminate the texture inconsistency in RGB-D 3D reconstruction and generate visually realistic texture mapping results. In addition to RGB color information, we incorporate a novel color image texture detail layer serving as an additional context to improve the effectiveness and robustness of the proposed method. First, we select an optimal texture image for each triangle face of the reconstructed model to avoid texture blurring and ghosting. During the selection procedure, the texture details are weighted to avoid generating texture chart partitions across high-frequency areas. Then, we optimize the camera pose of each texture image to align with the reconstructed 3D shape. Next, we propose a tri-directional similarity function to resynthesize the image context within the boundary stripe of texture charts, which can significantly diminish the occurrence of texture seams. Finally, we introduce a global color harmonization method to address the color inconsistency between texture images captured from different viewpoints. The experimental results demonstrate that the proposed method outperforms state-of-the-art texture mapping methods and effectively overcomes texture tearing, blurring, and ghosting artifacts.

Multiscale Analysis for Improving Texture Classification

Information from an image occurs over multiple and distinct spatial scales. Image pyramid multiresolution representations are a useful data structure for image analysis and manipulation over a spectrum of spatial scales. This paper employs the Gaussian–Laplacian pyramid to separately treat different spatial frequency bands of a texture. First, we generate three images corresponding to three levels of the Gaussian–Laplacian pyramid for an input image to capture intrinsic details. Then, we aggregate features extracted from gray and color texture images using bioinspired texture descriptors, information-theoretic measures, gray-level co-occurrence matrix feature descriptors, and Haralick statistical feature descriptors into a single feature vector. Such an aggregation aims at producing features that characterize textures to their maximum extent, unlike employing each descriptor separately, which may lose some relevant textural information and reduce the classification performance. The experimental results on texture and histopathologic image datasets have shown the advantages of the proposed method compared to state-of-the-art approaches. Such findings emphasize the importance of multiscale image analysis and corroborate that the descriptors mentioned above are complementary.

Fuzzy Color Aura Matrices for Texture Image Segmentation.

Fuzzy gray-level aura matrices have been developed from fuzzy set theory and the aura concept to characterize texture images. They have proven to be powerful descriptors for color texture classification. However, using them for color texture segmentation is difficult because of their high memory and computation requirements. To overcome this problem, we propose to extend fuzzy gray-level aura matrices to fuzzy color aura matrices, which would allow us to apply them to color texture image segmentation. Unlike the marginal approach that requires one fuzzy gray-level aura matrix for each color channel, a single fuzzy color aura matrix is required to locally characterize the interactions between colors of neighboring pixels. Furthermore, all works about fuzzy gray-level aura matrices consider the same neighborhood function for each site. Another contribution of this paper is to define an adaptive neighborhood function based on information about neighboring sites provided by a pre-segmentation method. For this purpose, we propose a modified simple linear iterative clustering algorithm that incorporates a regional feature in order to partition the image into superpixels. All in all, the proposed color texture image segmentation boils down to a superpixel classification using a simple supervised classifier, each superpixel being characterized by a fuzzy color aura matrix. Experimental results on the Prague texture segmentation benchmark show that our method outperforms the classical state-of-the-art supervised segmentation methods and is similar to recent methods based on deep learning.

Evolutionary Algorithm with Machine Learning Enabled Color Texture Image Segmentation and Classification Model

Evolutionary Algorithm with Machine Learning Enabled Color Texture Image Segmentation and Classification Model

Color Texture Image Complexity—EEG-Sensed Human Brain Perception vs. Computed Measures

In practical applications, such as patient brain signals monitoring, a non-invasive recording system with fewer channels for an easy setup and a wireless connection for remotely monitor physiological signals will be beneficial. In this paper, we investigate the feasibility of using such a system in a visual perception scenario. We investigate the complexity perception of color natural and synthetic fractal texture images, by studying the correlations between four types of data: image complexity that is expressed by computed color entropy and color fractal dimension, human subjective evaluation by scoring, and the measured brain EEG responses via Event-Related Potentials. We report on the considerable correlation experimentally observed between the recorded EEG signals and image complexity while considering three complexity levels, as well on the use of an EEG wireless system with few channels for practical applications, with the corresponding electrodes placement in accordance with the type of neural activity recorded.

Color texture image retrieval based on Copula multivariate modeling in the Shearlet domain

Due to the increasing number of images on the Internet, image retrieval framework is needed for image-based search. Shearlet transform is a sparse, multiscale, and multidimensional representation used to extract anisotropic features in image processing applications such as image fusion and image denoising. In this paper, we proposed a color texture image retrieval framework based on Shearlet domain modeling using Copula multivariate model. In the proposed framework, Gaussian Copula is used to model the dependencies between different sub-bands of the Non-Subsample Shearlet Transform (NSST) and non-Gaussian models are used for marginal modeling of the coefficients. Six different schemes are proposed for modeling NSST coefficients based on the four types of neighboring defined; moreover, Kullback–Leibler Divergence(KLD) close form is calculated in different situations for the two Gaussian Copula and non-Gaussian functions in order to investigate the similarities in the proposed retrieval framework. The Jeffery divergence (JD) criterion, which is a symmetrical version of KLD, is used for investigating similarities in the proposed framework. We have implemented our experiments on four texture image retrieval benchmark datasets, the results of which show the superiority of the proposed framework over the existing state-of-the-art methods. In addition, the retrieval time of the proposed framework is also analyzed in the two steps of feature extraction and similarity matching, which also shows that the proposed framework enjoys an appropriate retrieval time.

Using Direct Acyclic Graphs to Enhance Skeleton-Based Action Recognition with a Linear-Map Convolution Neural Network.

Research on the human activity recognition could be utilized for the monitoring of elderly people living alone to reduce the cost of home care. Video sensors can be easily deployed in the different zones of houses to achieve monitoring. The goal of this study is to employ a linear-map convolutional neural network (CNN) to perform action recognition with RGB videos. To reduce the amount of the training data, the posture information is represented by skeleton data extracted from the 300 frames of one film. The two-stream method was applied to increase the accuracy of recognition by using the spatial and motion features of skeleton sequences. The relations of adjacent skeletal joints were employed to build the direct acyclic graph (DAG) matrices, source matrix, and target matrix. Two features were transferred by DAG matrices and expanded as color texture images. The linear-map CNN had a two-dimensional linear map at the beginning of each layer to adjust the number of channels. A two-dimensional CNN was used to recognize the actions. We applied the RGB videos from the action recognition datasets of the NTU RGB+D database, which was established by the Rapid-Rich Object Search Lab, to execute model training and performance evaluation. The experimental results show that the obtained precision, recall, specificity, F1-score, and accuracy were 86.9%, 86.1%, 99.9%, 86.3%, and 99.5%, respectively, in the cross-subject source, and 94.8%, 94.7%, 99.9%, 94.7%, and 99.9%, respectively, in the cross-view source. An important contribution of this work is that by using the skeleton sequences to produce the spatial and motion features and the DAG matrix to enhance the relation of adjacent skeletal joints, the computation speed was faster than the traditional schemes that utilize single frame image convolution. Therefore, this work exhibits the practical potential of real-life action recognition.

Force Display and Tactile Display of Color Image Texture

In haptic interaction technology, texture haptic display is an important part. In order to perceive color image texture better, the haptic display methods of color texture based on force feedback and tactile feedback are proposed in this work. On the one hand, through the study of the physiological and psychological perception characteristics of color information, a new force rendering method of color image texture based on force feedback device is presented. The experimental results of color texture force perception show that the color texture force rendering algorithm in this paper works well. On the other hand, a color texture vibration tactile model is established based on the designed vibrotactile device. Its effectiveness is verified by vibration tactile perception experiment. Finally, for the color texture image of the real object surface, the force and vibrotactile display methods of color texture in this paper are used to conduct perception experiments and compared.

Constrained feature selection for semisupervised color-texture image segmentation using spectral clustering

Color-texture image segmentation remains a challenging problem due to extensive color-texture variability. Thus, the limited prior knowledge that is expressed by pairwise constraints can be exploited to guide the segmentation process. We propose a new semisupervised method by combining constrained feature selection and spectral clustering (SC) to perform color-texture image segmentation. The pairwise constraints are used by the constraint feature selection to choose the most relevant features among an available set of color and texture features. For this purpose, an innovative constraint score is developed to evaluate a subset of features at one time. A specific constrained SC algorithm involving the pairwise constraints is then applied to regroup the pixels into clusters. Experimental results on four benchmark datasets show that the proposed constraint score outperforms the main state-of-the-art constraint scores and that our semisupervised segmentation method is competitive compared with supervised, semisupervised, and unsupervised state-of-the-art segmentation methods.

Texture Filtering With Filtering Scale Map

In this paper, we propose a novel texture filtering method. Starting with a texture boundaries extraction, we obtain the possibility of texture boundary by the statistics of the proportion of the pixels in different colors. The possibility of texture boundary can be obtained by calculating the Bhattacharyya distance of the color proportion on each side of each pixel. Further, we build a filtering scale map to guide the parameters of the filter. This filtering scale map is based on the texture boundary. Finally, to obtain the texture filtering result, we design an adaptive shape edge-preserving filter which is simple and effective. By counting the color information of all pixel neighborhoods the filter can select the pixels in a similar color to filter. Experiments are performed on different color-texture images, and the results show that our proposed method performs much better compared with state-of-the-art methods on texture filtering.

Predicting the Perceptual Quality of Point Cloud: A 3D-to-2D Projection-Based Exploration

Point cloud is emerged as a promising media format to represent realistic 3D objects or scenes in applications, such as virtual reality, teleportation, etc. How to accurately quantify the subjective point cloud quality for application-driven optimization, however, is still a challenging and open problem. In this paper, we attempt to tackle this problem in a systematic means. First, we produce a fairly large point cloud dataset where ten popular point clouds are augmented with seven types of impairments (e.g., compression, photometry/color noise, geometry noise, scaling) at six different distortion levels, and organize a formal subjective assessment with tens of subjects to collect mean opinion scores (MOS) for all 420 processed point cloud samples (PPCS). We then try to develop an objective metric that can accurately estimate the subjective quality. Towards this goal, we choose to project the 3D point cloud onto six perpendicular image planes of a cube for the color texture image and corresponding depth image, and aggregate image-based global (e.g., Jensen-Shannon (JS) divergence) and local features (e.g., edge, depth, pixel-wise similarity, complexity) among all projected planes for a final objective index. Model parameters are fixed constants after performing the regression using a small and independent dataset previously published. The proposed metric has demonstrated the state-of-the-art performance for predicting the subjective point cloud quality compared with multiple full-reference and no-reference models, e.g., the weighted peak signal-to-noise ratio (PSNR), structural similarity (SSIM), feature similarity (FSIM) and natural image quality evaluator (NIQE). The dataset is made publicly accessible at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://smt.sjtu.edu.cn</uri> or <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">http://vision.nju.edu.cn</uri> for all interested audiences.

New local binary pattern approaches based on color channels in texture classification

In this paper, four novel, simple and robust approaches, which are left to right local binary patterns (LBPLL2R), top to down local binary patterns (LBPT2D), cube surface local binary pattern (LBPSurfaces), and cube diagonal local binary pattern (LBPDiagonal), were proposed in order to exact texture features in color images. These approaches were based on the local binary pattern (LBP), which is an effective statistical texture descriptor and can be employed in gray images. Proposed approaches were evaluated and validated in four datasets, which are Outex, KTH_TIPS, KTH_TIPS2, and USPtex datasets. The images in these datasets are in RGB, HSV, YIQ, and YCbCr color formats. Achieved results by these approaches were compared with the obtained results by the classical LBP and literature findings. As a result, the proposed approaches performed better than the traditional LBP method and they found effective in the classification of color texture images, especially in images, which are in RGB and HSV formats. Furthermore, noise robustness and time complexity of the proposed approaches were validated.

Compact Interchannel Sampling Difference Descriptor for Color Texture Classification

Many representation methods were built for gray image textures. However, they are not effective for color textures in general. To alleviate this problem, in this paper we propose a novel Compact Interchannel Sampling Difference Descriptor (CISDD) for color texture classification. In particular, considering sampling-based method can capture more directional information, we first use a heavy-tailed distribution, t-distribution to generate sample points in the image patch to calculate the micro-block difference. Then we model the interchannel relationship of color texture image by using dense micro-block differences. Furthermore, we utilize principal component analysis (PCA) to reduce the dimensions of the features encoded by the Fisher vector, and construct a Compact Interchannel Sampling Difference Descriptor (CISDD) for representing color texture image. Finally, experimental results on five published standard texture datasets (KTH-TIPS, VisTex, CUReT, USPTex and Colored Brodatz) reveal that CISDD is effective and outperforms thirteen representative color texture classification methods.

Partial Order Rank Features in Colour Space

Partial orders are the natural mathematical structure for comparing multivariate data that, like colours, lack a natural order. We introduce a novel, general approach to defining rank features in colour spaces based on partial orders, and show that it is possible to generalise existing rank based descriptors by replacing the order relation over intensity values by suitable partial orders in colour space. In particular, we extend a classical descriptor (the Texture Spectrum) to work with partial orders. The effectiveness of the generalised descriptor is demonstrated through a set of image classification experiments on 10 datasets of colour texture images. The results show that the partial-order version in colour space outperforms the grey-scale classic descriptor while maintaining the same number of features.

Completed Extremely Nonnegative DMD for Color Texture Classification

Dense micro-block difference (DMD) has achieved good performance in gray texture representation and classification. However, its performance is not satisfactory when representing color texture. To alleviate this problem, we propose a novel color texture representation method based on Completed Extremely Nonnegative DMD (CEN-DMD) in this paper. In particular, we first use DMD to model interchannel features and interchannel features of color texture images. Considering that negative value is meaningless in a digital image, we perform a nonnegative operation during the difference process. Due to that the maximum value in a nonnegative difference patch represents a significant difference, we construct the Extremely Nonnegative DMD (EN-DMD) by fusing the maximum values of the intrachannel features and the maximum of interchannel features, and further build Completed Extremely Nonnegative DMD (CEN-DMD) by fusing EN-DMDs at five scales and the global feature of the color texture images. Finally, the Fisher Vector is used to encode the CEN-DMD to obtain a color texture descriptor. Experimental results on five published standard color texture datasets (CUReT, Colored Brodatz, VisTex, USPTex and KTH-TIPS) reveal that CEN-DMD is effective when compared to the thirteen representative color texture classification methods.