Discriminative Texture Features Research Articles

A multi-channel framework based Local Binary Pattern with two novel local feature descriptors for texture classification

Local binary pattern (LBP) has been widely used in various application fields, including object detection, texture analysis, and remote sensing. To improve discrimination performance, many LBP-based methods have been proposed for extracting different local feature information for texture classification. However, current LBP-based algorithms typically describe local features at a single sampling scale, but some significant and discriminative texture feature information is contained between different scales. Therefore, the lack of capability to extract cross-scale features can lead to losing critical texture features. Moreover, low-frequency texture information should be accorded great importance in texture classification. Additionally, if the diversity and validity of local feature extraction are lacking, the effectiveness of classification capability suffers. To address these issues, (1) we propose a completed cross-scale Local binary pattern (ccsLBP) operator to extract cross-scale texture features. (2) A mean-filtered Local binary pattern (LBPmf) operator is presented to highlight the important low-frequency texture information of texture images. (3) We build a high-performing multi-channel framework based Local binary pattern (MC-LBP) for texture classification, which combines complementary features extracted by LBP, ccsLBP, and LBPmf hybridly to form a final feature vector of the texture image. The effectiveness of the proposed MC-LBP framework is verified on 6 representative texture databases, and the experimental results demonstrate its state-of-art texture classification performance.

Machine learning predicts histologic type and grade of canine gliomas based on MRI texture analysis.

Conventional MRI features of canine gliomas subtypes and grades significantly overlap. Texture analysis (TA) quantifies image texture based on spatial arrangement of pixel intensities. Machine learning (ML) models based on MRI-TA demonstrate high accuracy in predicting brain tumor types and grades in human medicine. The aim of this retrospective, diagnostic accuracy study was to investigate the accuracy of ML-based MRI-TA in predicting canine gliomas histologic types and grades. Dogs with histopathological diagnosis of intracranial glioma and available brain MRI were included. Tumors were manually segmented across their entire volume in enhancing part, non-enhancing part, and peri-tumoral vasogenic edema in T2-weighted (T2w), T1-weighted (T1w), FLAIR, and T1w postcontrast sequences. Texture features were extracted and fed into three ML classifiers. Classifiers' performance was assessed using a leave-one-out cross-validation approach. Multiclass and binary models were built to predict histologic types (oligodendroglioma vs. astrocytoma vs. oligoastrocytoma) and grades (high vs. low), respectively. Thirty-eight dogs with a total of 40 masses were included. Machine learning classifiers had an average accuracy of 77% for discriminating tumor types and of 75.6% for predicting high-grade gliomas. The support vector machine classifier had an accuracy of up to 94% for predicting tumor types and up to 87% for predicting high-grade gliomas. The most discriminative texture features of tumor types and grades appeared related to the peri-tumoral edema in T1w images and to the non-enhancing part of the tumor in T2w images, respectively. In conclusion, ML-based MRI-TA has the potential to discriminate intracranial canine gliomas types and grades.

A completed parted region local neighborhood energy pattern for texture classification

Binary pattern family has received unprecedented attention due to its primary role in texture representation. Texture images usually have strong region attributes and show dramatic changes from coarse regions to flats regions. However, most methods build texture descriptors by applying the same strategy to all pixels in the whole image, which ignore the difference between coarse regions and flat regions and fail to precisely represent the corresponding region attribute. To solve this problem, this paper proposes a novel completed parted region local neighborhood energy pattern which adaptively extracted discriminative texture information from coarse regions and flat regions. Firstly, we design an adaptive parted region division strategy for feature extraction. Secondly, to provide more rich and discriminative texture features, this paper further develops two novel operators based on the proposed strategy, parted region local difference energy pattern and parted region local pixel energy pattern. Finally, a completed parted region local neighborhood energy pattern is built by combining local sign pattern, parted region local difference energy pattern, and parted region local pixel energy pattern. Extensive experimental results on three popular texture databases demonstrate that the proposed descriptor outperforms recent state-of-the-art methods for texture classification tasks.

Multi-scale pathology image texture signature is a prognostic factor for resectable lung adenocarcinoma: a multi-center, retrospective study

BackgroundTumor histomorphology analysis plays a crucial role in predicting the prognosis of resectable lung adenocarcinoma (LUAD). Computer-extracted image texture features have been previously shown to be correlated with outcome. However, a comprehensive, quantitative, and interpretable predictor remains to be developed.MethodsIn this multi-center study, we included patients with resectable LUAD from four independent cohorts. An automated pipeline was designed for extracting texture features from the tumor region in hematoxylin and eosin (H&E)-stained whole slide images (WSIs) at multiple magnifications. A multi-scale pathology image texture signature (MPIS) was constructed with the discriminative texture features in terms of overall survival (OS) selected by the LASSO method. The prognostic value of MPIS for OS was evaluated through univariable and multivariable analysis in the discovery set (n = 111) and the three external validation sets (V1, n = 115; V2, n = 116; and V3, n = 246). We constructed a Cox proportional hazards model incorporating clinicopathological variables and MPIS to assess whether MPIS could improve prognostic stratification. We also performed histo-genomics analysis to explore the associations between texture features and biological pathways.ResultsA set of eight texture features was selected to construct MPIS. In multivariable analysis, a higher MPIS was associated with significantly worse OS in the discovery set (HR 5.32, 95%CI 1.72–16.44; P = 0.0037) and the three external validation sets (V1: HR 2.63, 95%CI 1.10–6.29, P = 0.0292; V2: HR 2.99, 95%CI 1.34–6.66, P = 0.0075; V3: HR 1.93, 95%CI 1.15–3.23, P = 0.0125). The model that integrated clinicopathological variables and MPIS had better discrimination for OS compared to the clinicopathological variables-based model in the discovery set (C-index, 0.837 vs. 0.798) and the three external validation sets (V1: 0.704 vs. 0.679; V2: 0.728 vs. 0.666; V3: 0.696 vs. 0.669). Furthermore, the identified texture features were associated with biological pathways, such as cytokine activity, structural constituent of cytoskeleton, and extracellular matrix structural constituent.ConclusionsMPIS was an independent prognostic biomarker that was robust and interpretable. Integration of MPIS with clinicopathological variables improved prognostic stratification in resectable LUAD and might help enhance the quality of individualized postoperative care.

A novel dual-channel brain tumor detection system for MR images using dynamic and static features with conventional machine learning techniques

Brain tumor detection, at the early stages of its development for the timely cure of the patient, is a challenging task due to its complex nonlinear nature. We propose a dual-channel brain tumor detection (DC-BTD) framework for magnetic resonance imaging scans with optimum false negatives, based on the idea of using D-channel for extremely discriminant dynamic features and S-channel for static features using data normalization, augmentation, and different machine learning (ML) classifiers, namely, support vector machine, k-nearest neighbor, naïve Bayes, and XGBoost. The D-channel features are extracted using a proposed Fine-Tuned Convolutional Neural Network (FT-CNN), while the S-channel features are extracted using the histogram of oriented gradients (HOG)- and local binary patterns (LBP)-based operators. The dual-channel data form the hybrid feature space (HFS) gives improved performance using two types of features. Computer experiments have been conducted on publically available brain tumors dataset obtained from Nanfang Hospital, Guangzhou, and Tianjin Medical University General Hospital, China. The finding of the current study shows that the proposed framework for brain-tumor prediction outperforms other contemporary existing methods with the highest generalization performance as 98.70% (accuracy) and 98.56% (F score) on the same dataset that can lead to better brain tumor detection at an early stage. Highlights A novel DC-BTD system is proposed using a competitive ML algorithm The dynamic channel is based on a new fine-tuned deep learning architecture The static channel is based on extremely discriminative textural features using HOG and LBP operators The HFS is formed by the dynamic and static channels

RF-HoDRF: High-Order Hybrid Discriminative Random Field Improved by Two-Layer Random Forest for SAR Image Change Detection

For better exploiting discriminative texture features and encoding high-level structures, this letter presents a high- order hybrid discriminative random field improved by two-layer random forest, abbreviated as RF-HoDRF, for synthetic aperture radar (SAR) image change detection. First, RF-HoDRF constructs a two-layer random forest (TL-RF) model to realize the selection of high-dimensional texture features, and then provides the class probabilities for constructing the unary potential in RF-HoDRF. Second, it defines a high-order potential on high-order cliques generated by superpixels to encode the high-level structures and maintain the region consistency. Finally, considering the pairwise potential by improved generalized Ising model and the statistics by generalized Gamma distribution (GΓD), the RF-HoDRF model is derived under the discriminative model framework. Then, by iteratively maximizing the local posterior probabilities, the class labels and the parameters are optimally estimated until they converge. Extensive comparisons and ablation experiments on measured SAR images verify the effectiveness of our method, and demonstrate that discriminative features selection and high-order structures maintenance have great contributions to improving change detection performances.

OL92 - Radiomics features of 11[C]-MET PET/CT in primary brain tumors: preliminary results on grading discrimination using a machine learning model

1096 Background: Nowadays Artificial Intelligence (AI) algorithms have demonstrated remarkable progress in image-recognition tasks and could be useful for the new concept of precision medicine in order to help physicians in the choice of therapeutic strategies for brain tumors. Previous data suggest that, in primary brain tumor, amino acid PET may demarcate more accurately the active disease than paramagnetic enhanced MRI that is the standard method of the evaluation in brain tumors and help in the evaluation of disease grading, as a fundamental basis for the correct clinical management of these patients. The aim of this study is to propose a radiomics workflow in order to create a predictive model on 11[C]-MET PET/CT scan images capable to discriminate between low-grade and high-grade primary brain tumors. Materials and Methods: in this retrospective study we selected fifty-six patients affected by primary brain tumor who underwent 11[C]-MET PET/CT from January 2016 to December 2019 at the Nuclear Medicine department of the Cannizzaro Hospital of Catania. Histological specimen the analysis was available in all patients in order to confirm the diagnosis and grading disease. PET/CT acquisition was performed after 10 minutes from the administration of 11C-Methionine (401-610 MBq) for a time acquisition of 15 minutes. 11[C]-MET PET/CT images were acquired using two scanners (24 patients on a Siemens scan and 32 patients on a GE scan). Then, LIFEx software was used to perform the segmentation and the extraction of features in order to identify the volume of interest (VOI) and to extract features of first, second, and third-order. LIFEx calculates 44 RF reflecting the VOI shape, the VOI voxel values, the histogram of VOI values, or the VOI textural content. A novel mixed descriptive-inferential sequential approach was used to identify a subset of relevant features that correlate with the outcome. A machine learning model based on discriminant analysis was tested in the evaluation of grading prediction (low vs high-grade) of 11[C]-MET PET/CT. Results: The machine learning model applicated in this study to stratify brain tumors as “low” and “high” grade showed discriminant texture features as follow: two features in fixed ROI respectively NGLDM Busyness (p-value: 0.1615) and GLZLM LZLGE (p-value 0.3207); three features in total volume respectively SHAPE Sphericity (p-value 0.0314), SHAPE Compacity (p-value 0.0215) and HISTO Kurtosis (p-value 0.0232). The results of performances, using a semi-automatic and user-independent segmentation process and an innovative feature extraction process on fixed ROI, showed a good performance in prediction of disease grade with an accuracy of 70.31% (AUC 64.13%) in all patients, of 80.51% (AUC 65.73%) in GE tomograph, of 84.98%(AUC 78.91%) in Siemens Tomograph. Using our model of machine learning with discriminant analysis the results of the performances were suboptimal in all patients group( 24 Siemens plus 32 GE) with an accuracy of 57.25 % and AUC of 58.51%. However in subgroup analysis, despite the low number of patients and different scanner, our model demonstrated good performances in the prediction of disease grade as follow: GE Scanner Accuracy 71.64 and AUC 62.8%; Siemens Scanner Accuracy 72,88% and AUC 78.91%. Conclusions: This preliminary study on radiomics features analysis of 11[C]-MET PET/CT suggests a possible value of Artificial Intelligence to assess primary brain tumors and to predict disease outcome in terms of grading discrimination at diagnosis. Further studies are needed to improve radiomics algorithms to personalize predictive and prognostic models and potentially support the medical decision process.

A deep learning model for burn depth classification using ultrasound imaging

Identification of burn depth with sufficient accuracy is a challenging problem. This paper presents a deep convolutional neural network to classify burn depth based on altered tissue morphology of burned skin manifested as texture patterns in the ultrasound images. The network first learns a low-dimensional manifold of the unburned skin images using an encoder-decoder architecture that reconstructs it from ultrasound images of burned skin. The encoder is then re-trained to classify burn depths. The encoder-decoder network is trained using a dataset comprised of B-mode ultrasound images of unburned and burned ex vivo porcine skin samples. The classifier is developed using B-mode images of burned in situ skin samples obtained from freshly euthanized postmortem pigs. The performance metrics obtained from 20-fold cross-validation show that the model can identify deep-partial thickness burns, which is the most difficult to diagnose clinically, with 99% accuracy, 98% sensitivity, and 100% specificity. The diagnostic accuracy of the classifier is further illustrated by the high area under the curve values of 0.99 and 0.95, respectively, for the receiver operating characteristic and precision-recall curves. A post hoc explanation indicates that the classifier activates the discriminative textural features in the B-mode images for burn classification. The proposed model has the potential for clinical utility in assisting the clinical assessment of burn depths using a widely available clinical imaging device.

Differentiation of Endometriomas from Ovarian Hemorrhagic Cysts at Magnetic Resonance: The Role of Texture Analysis.

Background and Objectives: To assess ovarian cysts with texture analysis (TA) in magnetic resonance (MRI) images for establishing a differentiation criterion for endometriomas and functional hemorrhagic cysts (HCs) that could potentially outperform their classic MRI diagnostic features. Materials and Methods: Forty-three patients with known ovarian cysts who underwent MRI were retrospectively included (endometriomas, n = 29; HCs, n = 14). TA was performed using dedicated software based on T2-weighted images, by incorporating the whole lesions in a three-dimensional region of interest. The most discriminative texture features were highlighted by three selection methods (Fisher, probability of classification error and average correlation coefficients, and mutual information). The absolute values of these parameters were compared through univariate, multivariate, and receiver operating characteristic analyses. The ability of the two classic diagnostic signs (“T2 shading” and “T2 dark spots”) to diagnose endometriomas was assessed by quantifying their sensitivity (Se) and specificity (Sp), following their conventional assessment on T1-and T2-weighted images by two radiologists. Results: The diagnostic power of the one texture parameter that was an independent predictor of endometriomas (entropy, 75% Se and 100% Sp) and of the predictive model composed of all parameters that showed statistically significant results at the univariate analysis (100% Se, 100% Sp) outperformed the ones shown by the classic MRI endometrioma features (“T2 shading”, 75.86% Se and 35.71% Sp; “T2 dark spots”, 55.17% Se and 64.29% Sp). Conclusion: Whole-lesion MRI TA has the potential to offer a superior discrimination criterion between endometriomas and HCs compared to the classic evaluation of the two lesions’ MRI signal behaviors.

Iris presentation attack detection based on best-k feature selection from YOLO inspired RoI

Obfuscating an iris recognition system through forged iris samples has been a major security threat in iris-based authentication. Therefore, a detection mechanism is essential that may explicitly discriminate between the live iris and forged (attack) patterns. The majority of existing methods analyze the eye image as a whole to find discriminatory features for fake and real iris. However, many attacks do not alter the entire eye image, instead merely the iris region is affected. It infers that the iris embodies the region of interest (RoI) for an exhaustive search towards identifying forged iris patterns. This paper introduces a novel framework that locates RoI using the YOLO approach and performs selective image enhancement to enrich the core textural details. The YOLO approach tightly bounds the iris region without any pattern loss, where the textural analysis through local and global descriptors is expected to be efficacious. Afterward, various handcrafted and CNN based methods are employed to extract the discriminative textural features from the RoI. Later, the best-k features are identified through the Friedman test as the optimal feature set and combined using score-level fusion. Further, the proposed approach is assessed on six different iris databases using predefined intra-dataset, cross-dataset, and combined-dataset validation protocols. The experimental outcomes exhibit that the proposed method results in significant error reduction with the state of the arts.

Differentiating High-Grade Gliomas from Brain Metastases at Magnetic Resonance: The Role of Texture Analysis of the Peritumoral Zone.

High-grade gliomas (HGGs) and solitary brain metastases (BMs) have similar imaging appearances, which often leads to misclassification. In HGGs, the surrounding tissues show malignant invasion, while BMs tend to displace the adjacent area. The surrounding edema produced by the two cannot be differentiated by conventional magnetic resonance (MRI) examinations. Forty-two patients with pathology-proven brain tumors who underwent conventional pretreatment MRIs were retrospectively included (HGGs, n = 16; BMs, n = 26). Texture analysis of the peritumoral zone was performed on the T2-weighted sequence using dedicated software. The most discriminative texture features were selected using the Fisher and the probability of classification error and average correlation coefficients. The ability of texture parameters to distinguish between HGGs and BMs was evaluated through univariate, receiver operating, and multivariate analyses. The first percentile and wavelet energy texture parameters were independent predictors of HGGs (75–87.5% sensitivity, 53.85–88.46% specificity). The prediction model consisting of all parameters that showed statistically significant results at the univariate analysis was able to identify HGGs with 100% sensitivity and 66.7% specificity. Texture analysis can provide a quantitative description of the peritumoral zone encountered in solitary brain tumors, that can provide adequate differentiation between HGGs and BMs.

Multi-pose facial expression recognition using rectangular HOG feature extractor and label-consistent KSVD classifier

In this paper, a new approach to the classification of facial expressions from multiple pose images is proposed. In this approach, a rectangular histogram of oriented gradient (R-HOG) algorithm is first designed to extract features of face images. The parameters of the R-HOG algorithm, which is a modification of the original HOG algorithm include cell shape, cell size, block size, and the number of orientation bins. The R-HOG is capable of capturing more discriminative texture features of different facial expressions. In addition, a supervised dictionary learning classifier, the label-consistent K-SVD (LC-KSVD) algorithm, is adopted to recognise the facial expression of the subject. To investigate its effectiveness, the proposed technique was applied to classify emotional states of the face images in the two public available facial expression datasets: KDFE and RafD. The experiment study showed that the new method outperformed in many aspects those methods reported in the literature tested with the same datasets. First, the new method handles pose variations better. Second, it is more robust in cases where the size of a training dataset is small. Finally, it's accuracy performance is more consistent measured by standard deviations.

Detection of Sigatoka Disease in Plantain Using IoT and Machine Learning Techniques

Achieving United Nations Sustainable Development Goal 2 (UN SDG2) infers an imperative to urgently increase food production by up to 70%. However, concerns have risen that increases in food production have not kept pace with increase in world population, which is estimated to reach 10 billion people by the year 2050. In this paper, an IoT with machine learning based system was developed to acquire and process significant indicators such as temperature, moisture, humidity and leave images for the detection of Sigatoka disease in plantain. Appropriate sensors for detecting the stated disease indicators were interfaced with Raspberry Pi3 microcontroller module to collate and transmit the sensor data wirelessly to ThingSpeak, which is the selected cloud based IoT platform. The acquired leave images were further processed using two image descriptors, namely: Scalable Color Descriptor (SCD) and Histogram of Oriented Gradient (HOG) to extract discriminative color and texture features respectively. The features were then classified to detect the diseased or non-diseased class using Multilayer PerceptronArtificial Neural Network (MLP-ANN). The best accuracy of 98% was produced using the HOG descriptor.

Texture Analysis of Multi-Shot Echo-planar Diffusion-Weighted Imaging in Head and Neck Squamous Cell Carcinoma: The Diagnostic Value for Nodal Metastasis.

Accurate assessment of nodal metastasis in head and neck squamous cell carcinoma (SCC) is important, and diffusion-weighted imaging (DWI) has emerged as a potential technique in differentiating benign from malignant lymph nodes (LNs). This study aims to evaluate the diagnostic performance of texture analysis using apparent diffusion coefficient (ADC) data of multi-shot echo-planar imaging-based DWI (msEPI-DWI) in predicting metastatic LNs of head and neck SCC. 36 patients with pathologically proven head and neck SCC were included in this study. A total of 204 MRI-detected LNs, including 176 subcentimeter-sized LNs, were assigned to metastatic or benign groups. Texture features of LNs were compared using independent t-test. Hierarchical cluster analysis was performed to exclude redundant features. Multivariate logistic regression and receiver operating characteristic analysis were performed to assess the diagnostic performance. The discriminative texture features for predicting metastatic LNs were complexity, energy and roundness. Areas under the curves (AUCs) for diagnosing metastasis in all/subcentimeter-sized LNs were 0.829/0.767 using complexity, 0.699/0.685 using energy and 0.671/0.638 using roundness, respectively. The combination of three features resulted in higher AUC values of 0.836/0.781. In conclusion, texture analysis of ADC data using msEPI-DWI could be a useful tool for nodal staging in head and neck SCC.

Differentiating Cancerous and Non-cancerous Prostate Tissue Using Multi-scale Texture Analysis on MRI.

Prostate cancer (PCa) diagnosis is established by pathological examination via biopsies, which are associated with significant complications and false negatives. Using MRIs to identify locations with high probability of containing cancer could instead be used to guide the biopsy procedure. The present investigation aims to identify target regions within different prostatic zones on MRI with high probability of being cancerous for assisting in the decision of where and how to perform biopsy. Our approach involved extracting multi-scale texture features for capturing local patterns to distinguish cancer and healthy tissue in different T2W-MRI prostate zones. Three different classification models were fed by the proposed strategy, namely support vector machine (SVM), Adaboost, and Random Forest. SVM with a linear kernel showed the best classification performance, with AUC scores of 0.91 in the anterior fibromuscular stroma area, 0.85 in the peripheral zone, and 0.87 when classification is performed independently of the prostate zone. The proposed method demonstrated that discriminant multi-scale texture features can accurately identify regions of prostate cancer in a zone-specific fashion, via MRI.

A cluster-based feature selection method for image texture classification

<p>Computer vision and pattern recognition applications have been counted serious research trends in engineering technology and scientific research content. These applications such as texture image analysis and its texture feature extraction. Several studies have been done to obtain accurate results in image feature extraction and classifications, but most of the extraction and classification studies have some shortcomings. Thus, it is substantial to amend the accuracy of the classification via minify the dimension of feature sets. In this paper, presents a cluster-based feature selection approach to adopt more discriminative subset texture features based on three different texture image datasets. Multi-step are conducted to implement the proposed approach. These steps involve texture feature extraction via Gray Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP) and Gabor filter. The second step is feature selection by using K-means clustering algorithm based on five feature evaluation metrics which are infogain, Gain ratio, oneR, ReliefF, and symmetric. Finally, K-Nearest Neighbor (KNN), Naive Bayes (NB) and Support Vector Machine (SVM) classifiers are used to evaluate the proposed classification performance and accuracy. Research achieved better classification accuracy and performance using KNN and NB classifiers that were 99.9554% for Kelberg dataset and 99.0625% for SVM in Brodatz-1 and Brodatz-2 datasets consecutively. Conduct a comparison to other studies to give a unified view of the quality of the results and identify the future research directions.</p>

A Comparative Texture Analysis Based on NECT and CECT Images to Differentiate Lung Adenocarcinoma from Squamous Cell Carcinoma.

The purpose of the study was to compare the texture based discriminative performances between non-contrast enhanced computed tomography (NECT) and contrast-enhanced computed tomography (CECT) images in differentiating lung adenocarcinoma (ADC) from squamous cell carcinoma (SCC) patients. Eighty-seven lung cancer subjects were enrolled in the study, including pathologically proved 47 ADC patients and 40 SCC patients, and 261 texture features were extracted from the manually delineated region of interests on CECT and NECT images respectively. Fisher score was then used to select the effective discriminative texture features between groups, and the selected texture features were adopted to differentiate ADC from SCC using Support Vector Machine and Leave-one-out cross-validation. Both NECT and CECT images could achieve the same best classification accuracy of 95.4%, and most of the informative features were from the gray-level co-occurrence matrix. In addition, CECT images were found with enhanced texture features compared with NECT images, and combining texture features of CECT and NECT images together could further improve the prediction accuracy. Besides the texture feature, the tumor location information also contributed to the differential diagnosis between ADC and SCC.

Multi-feature fusion method for medical image retrieval using wavelet and bag-of-features

Color, texture, and shape are the common features used for the retrieval systems. However, many medical images have a spot of color information. Therefore, the discriminative texture and shape features should be extracted to obtain a satisfied retrieval result. In order to increase the credibility of the retrieval process, many features can be combined to be used for medical image retrieval. Meanwhile, more features require more processing time, which will decrease the retrieval speed. In this paper, wavelet decomposition is adopted to generate different resolution images. Bag-of-feature, texture, and LBP feature are extracted from three different-level wavelet images. Finally, the similarity measure function is obtained by fusing these three types of features. Experimental results show that the proposed multi-feature fusion method can achieve a higher retrieval accuracy with an acceptable retrieval time.

Trajectory-aware three-stream CNN for video action recognition

Video-based human action recognition is a challenging task in computer vision. In recent years, the convolution neural network (CNN) and its extended versions have shown promising results for video action recognition. However, most of the existing methods cannot deal with the global motion information effectively, especially for long-term motion which is crucial to represent complex none-periodic actions. To address this issue, a stacked trajectory energy image (STEI) is proposed by extracting trajectories from motion saliency regions and stacked them onto one grayscale image. This will result in an STEI with discriminative texture feature which can effectively characterize the global motion from multiple consecutive frames. Then, a three-stream CNN framework is proposed to simultaneously capture spatial, temporal, and global motion information of the action from RGB frames, optical flow, and STEI. Moreover, a trajectory-aware convolution strategy is introduced by incorporating local and long-term motion information so as to learn the motion features directly and effectively from three complementary action-related regions. Finally, the learned features are aggregated and categorized by a linear support vector machine. The experimental results on two challenging datasets (i.e., HMDB51 and UCF101) demonstrate that our approach statistically outperforms a number of state-of-the-art methods.

Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification

Designing discriminative powerful texture features robust to realistic imaging conditions is a challenging computer vision problem with many applications, including material recognition and analysis of satellite or aerial imagery. In the past, most texture description approaches were based on dense orderless statistical distribution of local features. However, most recent approaches to texture recognition and remote sensing scene classification are based on Convolutional Neural Networks (CNNs). The de facto practice when learning these CNN models is to use RGB patches as input with training performed on large amounts of labeled data (ImageNet). In this paper, we show that Local Binary Patterns (LBP) encoded CNN models, codenamed TEX-Nets, trained using mapped coded images with explicit LBP based texture information provide complementary information to the standard RGB deep models. Additionally, two deep architectures, namely early and late fusion, are investigated to combine the texture and color information. To the best of our knowledge, we are the first to investigate Binary Patterns encoded CNNs and different deep network fusion architectures for texture recognition and remote sensing scene classification. We perform comprehensive experiments on four texture recognition datasets and four remote sensing scene classification benchmarks: UC-Merced with 21 scene categories, WHU-RS19 with 19 scene classes, RSSCN7 with 7 categories and the recently introduced large scale aerial image dataset (AID) with 30 aerial scene types. We demonstrate that TEX-Nets provide complementary information to standard RGB deep model of the same network architecture. Our late fusion TEX-Net architecture always improves the overall performance compared to the standard RGB network on both recognition problems. Furthermore, our final combination leads to consistent improvement over the state-of-the-art for remote sensing scene classification.