Final Feature Vector Research Articles

Multilayer Convolutional Feature Aggregation Algorithm for Image Retrieval

In image retrieval tasks, the single-layer convolutional feature has insufficient image semantic representation ability. A new image description algorithm ML-RCroW based on multilayer multiregion cross-weighted aggregational deep convolutional features is proposed. First, the ML-RCroW algorithm inputs an image into the VGG16 (a deep convolutional neural network developed by researchers at Visual Geometry Group and Google DeepMind) network model in which the fully connected layer is discarded. The visual feature information in the convolutional neural network (CNN) is extracted, and the target response weight map is generated by combining with the spatial weighting algorithm of the target fuzzy marker. Then, visual features in the CNN are divided into multiple regions, and the pixels of each region are weighted by regional spatial weight, regional channel weight, and regional weight. The image global vector is generated by aggregating and encoding every region in the weighted feature map. Finally, features of each layer of the VGG16 network model are extracted and then aggregated and dimensionally reduced to obtain the final feature vector of the image. The experiments are carried out on the Oxford5k and Paris6k datasets provided by Oxford VGG. The experimental results show that the average accuracy of image retrieval based on the image feature description algorithm ML-RCroW is better than that achieved by the other commonly used algorithms such as SPoC, R-MAC, and CroW.

Action recognition in poor-quality spectator crowd videos using head distribution-based person segmentation

Despite a big volume of research on action recognition, little attention has been given to individual action recognition in poor-quality spectator crowd scenes. It is an important scenario, because most of the surveillance systems generate poor-quality videos, though current state-of-the-art methods may not be effectively applicable. Therefore recognizing actions performed by individuals in poor-quality spectator crowd scenes is an unsolved problem. In such cases, the main challenge is localizing person proposals for each actor in the crowd. This challenge becomes more difficult when occlusion is severe. In this work, we propose a novel approach to find person proposals in poor-quality spectator crowds using crowd-based constraints. First, we define persons in the crowd by using efficient person head detectors. We exploit person head size to estimate the person bounding box using linear regression. Then, we use distribution of heads in the crowd image to estimate more accurate person proposals. Motion trajectories are independently computed in the video without considering persons and then assigned to each person based on a novel distance measure computed between the trajectory and the person proposal. The set of trajectories and associated motion and texture-based features in overlapped time windows are used to compute the final feature vector. For each time window using early information fusion in the bag of visual-words framework, cumulative feature vectors are computed encoding action information. Experiments are performed on a publicly available real-world spectator crowd dataset containing as many as 150 actors performing multiple actions at the same time. Our experiments have demonstrated excellent performance of the proposed technique.

Visual information retrieval from historical document images

In the recent decades, preserving and publicizing historical documents in digital format has gotten considerable attention. Although modern digitizing techniques have mostly solved the problem of protecting and accessing these documents, the task of visual information retrieval and interpretation is still an arduous issue. This is due to historical documents’ complex and unusual structures beside their degraded nature. For information retrieval from historical documents, an appropriate approach is required to characterize the document content in a coherent way. Printed documents contain not only text characters and their formattings but also some associated typographical elements. Finding and pursuing the existing visual typographical objects that shape the content of historical documents, helps us retrieve and convey more information about the various methods of representing these documents. These elements can be footnotes that connect the authority and demonstrate the relationship between manuscripts and sources, or tables that summarize different sort of information into geometric forms. This research focuses on the problem of detecting footnotes and tables in historical documents and establishes a framework for each of the driven objectives. These frameworks must efficiently handle complex structures of historical documents and at the same time possess the generalization power to be applied to large-scale document image collections. To the best of our knowledge, up to this date, footnote detection has rarely been addressed in the literature. Therefore, our first goal is to present a novel framework for footnote-based document image classification in historical documents. The basic idea behind this framework is to utilize the most prominent visual features of a footnote to create a feature vector. The three most notable visual features of a footnote in a page are the smaller font size of the footnote respect to the body text, the footnote location at the bottom of the page and the relatively greater gap between the footnote and the body text compared to the standard line space. Three methods are proposed according to each of these observations. We define a set of rules using these observations to create our final feature vector. Our framework for footnote-based document image classification in the historical documents is completed by feeding these feature vectors to a support vector machine (SVM) classifier. The proposed framework is applied to more than 32 million images from 18th century. The evaluation results prove the efficiency, generalization power, and robustness of our presented framework for detecting page containing footnote despite their layout and structure type. The state-of-the-art methods for table detection in documents mostly use markup documents (e.g., pdf, HTML, etc.) and do not cover all types of the tables within one framework. However, for historical documents, which are our main target for this thesis, we only have access to the scanned image and need to deal with all types of tables at the same time. The proposed framework is based on the hypothesis that texts in tables occur in a harmonic column-wise manner. This fact suggests the idea of using a spectral method for developing our framework. We propose an approach based on using Mel frequency cepstral coefficients (MFCC) to classify document images according to the presence or not presence of tables on the page. MFCCs are well-known speech processing features, which emphasize lower frequency components rather than higher ones. An SVM classifier is used as the final step of our framework for detecting pages containing tables. We test the introduced framework on our datasets and the results confirm the efficiency of the proposed method in comparison to both a state-of-the-art method and our benchmark dataset from the 18th century printed documents.

Computational identification of microbial phosphorylation sites by the enhanced characteristics of sequence information

Protein phosphorylation on serine (S) and threonine (T) has emerged as a key device in the control of many biological processes. Recently phosphorylation in microbial organisms has attracted much attention for its critical roles in various cellular processes such as cell growth and cell division. Here a novel machine learning predictor, MPSite (Microbial Phosphorylation Site predictor), was developed to identify microbial phosphorylation sites using the enhanced characteristics of sequence features. The final feature vectors optimized via a Wilcoxon rank sum test. A random forest classifier was then trained using the optimum features to build the predictor. Benchmarking investigation using the 5-fold cross-validation and independent datasets test showed that the MPSite is able to achieve robust performance on the S- and T-phosphorylation site prediction. It also outperformed other existing methods on the comprehensive independent datasets. We anticipate that the MPSite is a powerful tool for proteome-wide prediction of microbial phosphorylation sites and facilitates hypothesis-driven functional interrogation of phosphorylation proteins. A web application with the curated datasets is freely available at http://kurata14.bio.kyutech.ac.jp/MPSite/.

Improving the Bag-of-Words model with Spatial Pyramid matching using data augmentation for fine-grained arbitrary-oriented ship classification

ABSTRACTIn this letter, we investigate fine-grained classification of arbitrary-oriented ships in very high resolution optical imagery using Bag of Word model with Spatial Pyramid (SP-BoW). Given that based on ‘spatial pyramid’ of the histogram of local features, the final feature vectors not only count the multiplicity of ‘words’ but also represent their spatial topology. We attempt to improve the performance of this model by introducing augmented data for training phase. Our aim is to make the dataset big enough to be able to capture holistic variation of ship orientation. Three data augmentation operations are used including random rotate by an angle of modulo 90°, random flip-left-right, and random flip-top-bottom. Through this procedure, our trained SP-BoW model is able to get better generalization. The proposed approach is validated on the High-Resolution Ship Collections 2016 (HRSC2016) ship dataset. The results indicate that training on augmented data can significantly improve the performance of SP-BoW. Beside, compared to other state-of-the-art convolutional neural network-based approaches, the approach proposed in this research has yielded competitive results and could make it a good baseline for evaluating more sophisticated CNN architecture in the future.

Augmented features to detect image splicing on SWT domain

Nowadays, image forgery is a widespread problem in our lives. Image splicing is one method of forging images, and it has become a common tool for malicious users to modify images. Human eyes cannot perceive forgery operations on forged images; therefore, designing an expert system for detecting image authenticity has become a necessity. The splicing method causes some distortions that can be used to detect forgery on a tampered image. In this work, an expert system is designed that uses the statistical and textural properties of an image in a hybrid manner to detect forgery. The method extracts properties (statistical and textural) from high-level sub-bands of stationary wavelet transform (SWT) domain. SWT is used to obtain image's details in a multi scale manner in frequency domain. The method extracts statistical features from three sub-bands via Markov model, which generates transition probability matrices and then obtains Haralic's textural features based on gray level co-occurrence matrices. Features from each sub-bands are concatenated to form final feature vector. This expert system is the first in the literature to be designed using textural and statistical features together. Finally, the designed expert system classifies images as authentic or forged using SVM. The system's performance was evaluated using three publicly available image databases and tested for different attack types. The experimental results show that using statistical and textural features in a hybrid manner improves detection accuracy when compared to similar works in the literature. The main impact of the proposed system is its high accuracy and robust detection of different attack types.

Local Force Pattern (LFP): Descriptor for Heterogeneous Face Recognition

This paper proposes a novel local-appearance based feature descriptor namely Local Force Pattern (LFP), for robust heterogeneous face recognition. LFP features encode the directional information of the textures of the given face images compactly, producing a more discriminating code than the state of the art methods. We compute the structure of each micro-pattern with the aid of Van der Waal’s force between a pair of pixels, which extracts the directional information, and encodes it using a 2-D vector. This directional information helps us to discriminate images having similar structural patterns with different modality based intensity transitions. We divide the face image into several small pixel boxes called ‘cell’ and extract the LFP features from them. Several cell forms ‘block’. The resultant force obtained from the block is vector-summed to get the final feature vector. Then, we concatenate these features into a vector and use it as the face descriptor. Experiments show that the proposed descriptor consistently performs well over state-of-the-art methods for different heterogeneous face datasets like CUSF, IIIT-D and CASIA.

Novel Classification Algorithm for Ballistic Target Based on HRRP Frame

Nowadays, the identification of ballistic missile warheads in a cloud of decoys and debris is essential for defense systems in order to optimize the use of ammunition resources, avoiding to run out of all the available interceptors in vain. This paper introduces a novel solution for the classification of ballistic targets based on the computation of the inverse Radon transform of the target signatures, represented by a high-resolution range profile frame acquired within an entire period of the main rotation of the target. Namely, the precession for warheads and the tumbling for decoys are taken into account. The pseudo-Zernike moments of the resulting transformation are evaluated as the final feature vector for the classifier. The extracted features guarantee robustness against target's dimensions and rotation velocity, and the initial phase of the target's motion. The classification results on simulated data are shown for different polarizations of the electromagnetic radar waveform and for various operational conditions, confirming the validity of the algorithm.

An Improved Illumination Normalization and Robust Feature Extraction Technique for Face Recognition Under Varying Illuminations

Invariant feature extraction under diverse illuminations is challenging for face recognition. Light intensity variations in face images are predominant in large-scale features. In existing techniques, such features are truncated to achieve illumination-invariant features. However, salient features are lost during small-scale feature extraction that affect performance. Thus, objective in proposed work is to attain improved illumination normalization where large-scale and small-scale feature information is efficiently extracted for face recognition. First, reflectance ratio and histogram equalization (RRHE)-based new illumination normalization framework is proposed in which illumination deviations are annulled. Then, histogram equalization is employed to enhance contrast of reflectance ratio image by adjusting pixel intensities. Then, robust feature extraction in discrete wavelet packet transform domain (RFDWPT) from RRHE images is performed using various orthogonal wavelets with distinct vanishing moments. Here, small-scale features (noise effect) of RRHE images are discarded and final feature vector is formed by appropriate small-scale and large-scale features. This results in illumination-normalized salient features for eigen space analysis where nearest neighbor classification is performed on compact size training and test features. Compared with other face recognition techniques under different illumination conditions, significant enhancement in recognition accuracy is achieved on benchmark databases such as Yale B, CMU-PIE, Yale and extended Yale B.

BLIQUE-TMI: Blind Quality Evaluator for Tone-Mapped Images Based on Local and Global Feature Analyses

High dynamic range (HDR) image, which has a powerful capacity to represent the wide dynamic range of real-world scenes, has been receiving attention from both academic and industrial communities. Although HDR imaging devices have become prevalent, the display devices for HDR images are still limited. To facilitate the visualization of HDR images in standard low dynamic range displays, many different tone mapping operators (TMOs) have been developed. To create a fair comparison of different TMOs, this paper proposes a BLInd QUality Evaluator to blindly predict the quality of Tone-Mapped Images (BLIQUE-TMI) without accessing the corresponding HDR versions. BLIQUE-TMI measures the quality of TMIs by considering the following aspects: 1) visual information; 2) local structure; and 3) naturalness. To be specific, quality-aware features related to the former two aspects are extracted in a local manner based on sparse representation, while quality-aware features related to the third aspect are derived based on global statistics modeling in both intensity and color domains. All the extracted local and global quality-aware features constitute a final feature vector. An emergent machine learning technique, i.e., extreme learning machine, is adopted to learn a quality predictor from feature space to quality space. The superiority of BLIQUE-TMI to several leading blind IQA metrics is well demonstrated on two benchmark databases.

An architecture of deep learning network based on ensemble empirical mode decomposition in precise identification of bearing vibration signal

This paper proposes a deep learning network (DLN) as the basis for a bearing fault diagnosis technique, which is constructed by the autoencoders and softmax classifier for the purpose of identifying the various multi-degree bearing fault. Firstly, the ensemble empirical mode decomposition (EEMD) method is used to decompose the original vibration signal into intrinsic mode functions (IMFs). A high-dimensionality feature vector is formed by analyzing the statistical parameters in the time domain and the frequency domain of the first several IMFs. Then, this feature vector serves as the input for DLN to classify the bearing fault pattern. In a DLN, an autoencoder performs the unsupervised feature self-learning phase to generate a final significant feature vector for training the softmax classifier. Finally, the parameters of a complete DLN based on stacking auto-encoders and the softmax classifier together is fine-tuned with respect to supervised learning criterion aiming to optimize the classification error. Experimental results have shown a great effect for bearing fault diagnosis based on the proposed DLN. The identification accuracy result has been achieved in the bearing fault status even with the unpredictable defects tests on the inner race and roller element of bearing. Methodologies in this study offer confidence for complex data classification.

Robustness-Driven Hybrid Descriptor for Noise-Deterrent Texture Classification

A robustness-driven hybrid descriptor for noise-deterrent texture classification is presented in this paper. This work offers the ability to categorize a variety of textures under challenging image acquisition conditions. An image is initially resolved into its low-frequency components by applying wavelet decomposition. The resulting low-frequency components are further processed for feature extraction using completed joint-scale local binary patterns (CJLBP). Moreover, a second feature set is obtained by computing the low order derivatives of the original sample. The evaluated feature sets are integrated to obtain a final feature vector representation. The texture-discriminating performance of the hybrid descriptor is analysed using renowned datasets: Outex original, Outex extended and KTH-TIPS. Experimental results demonstrate a stable and robust performance of the descriptor under a variety of noisy conditions. An accuracy of 95.86%, 32.52% and 88.74% at noise variance of 0.025 is achieved for the given datasets, respectively. A comparison between performance parameters of the proposed work with its parent descriptors and recently published work is also demonstrated.

Breast abnormality detection using combined texture and vascular features

This work presents an integrated approach that combines texture and vascular features for distinguishing malignancy and benignity of breast abnormalities using thermal breast image. A local texture descriptor, called block variance (BV), is used here to extract the texture features. On the other hand, thermo-vascular pattern based features are identified by using a series of morphological operations. Then, these two feature sets are fused together to make a final feature vector. In this work, a five-layer feed forward, back propagation neural network (FBNN) is implemented as a classifier. The breast thermograms of DMR-IR database are used for the purpose of evaluation of the proposed system performance. Experimental results have shown that the proposed method detected malignant cases with 94% accuracy, while benign cases are detected with 100% accuracy. The overall system accuracy is obtained as 97.2%, which is comparatively better than other existing state-of-the-art methods.

Breast abnormality detection using combined texture and vascular features

This work presents an integrated approach that combines texture and vascular features for distinguishing malignancy and benignity of breast abnormalities using thermal breast image. A local texture descriptor, called block variance (BV), is used here to extract the texture features. On the other hand, thermo-vascular pattern based features are identified by using a series of morphological operations. Then, these two feature sets are fused together to make a final feature vector. In this work, a five-layer feed forward, back propagation neural network (FBNN) is implemented as a classifier. The breast thermograms of DMR-IR database are used for the purpose of evaluation of the proposed system performance. Experimental results have shown that the proposed method detected malignant cases with 94% accuracy, while benign cases are detected with 100% accuracy. The overall system accuracy is obtained as 97.2%, which is comparatively better than other existing state-of-the-art methods.

Moroccan Video Intelligent Transport System: Vehicle Type Classification Based on Three-Dimensional and Two-Dimensional Features

Vehicle type classification is a critical function in any intelligent transportation system (ITS). In this paper, we present a novel two-layer vehicle type classification framework based on the vehicle's 3D parameters and its local features. This framework is a part of the first Moroccan video intelligent transport system (MOVITS) that aims to control traffic and road code violations. In the first layer, the 3D features are extracted using the disparity map generated from stereo-images, and then, the width, height, and length of the vehicle are calculated based on the obtained list of 3D points. In the second layer, a gradient-based method is applied to extract the 2D features, and a dimensional reduction algorithm is performed to reduce its size. Both features are combined to construct the final feature vector that is used as an input for the classification. The Moroccan dataset and the BIT dataset were used to, respectively, validate the proposed framework and conduct a comparative study with the state-of-the-art algorithms. The experimental results demonstrate the efficiency of our approach against existing algorithms.

A Deep Feature Extraction Method for HEp-2 Cell Image Classification

The automated and accurate classification of the images portraying the Human Epithelial cells of type 2 (HEp-2) represents one of the most important steps in the diagnosis procedure of many autoimmune diseases. The extreme intra-class variations of the HEp-2 cell images datasets drastically complicates the classification task. We propose in this work a classification framework that, unlike most of the state-of-the-art methods, uses a deep learning-based feature extraction method in a strictly unsupervised way. We propose a deep learning-based hybrid feature learning with two levels of deep convolutional autoencoders. The first level takes the original cell images as the inputs and learns to reconstruct them, in order to capture the features related to the global shape of the cells, and the second network takes the gradients of the images, in order to encode the localized changes in intensity (gray variations) that characterize each cell type. A final feature vector is constructed by combining the latent representations extracted from the two networks, giving a highly discriminative feature representation. The created features will be fed to a nonlinear classifier whose output will represent the type of the cell image. We have tested the discriminability of the proposed features on two of the most popular HEp-2 cell classification datasets, the SNPHEp-2 and ICPR 2016 datasets. The results show that the proposed features manage to capture the distinctive characteristics of the different cell types while performing at least as well as the actual deep learning-based state-of-the-art methods in terms of discrimination.

An Automatic Bone Disorder Classification Using Hybrid Texture Feature Extraction with Bone Mineral Density

A novel approach has been proposed to classify bone disorders for classifying the radiographic bone image as normal, Osteopenia and Osteoporosis. The proposed system consists of three major stages to predict the accurate bone disorder classification. In the first stage, image preprocessing is performed where bilateral filtering is applied to remove noise and to enhance the image quality. Then, the image is fed to Otsu based segmentation approach for segmenting the abnormal area of the bone image. In the second stage, Discrete Wavelet Transform (DWT) is used to the segmented image. Once the image gets segmented then, the Gray-Level Co-occurrence Matrix (GLCM) method is applied to extract the features in terms of statistical texture-based. Further the image which is applied to Principle Component Analysis (PCA) to reduce size of the feature vector. Besides, Bone Mineral Density (BMD) feature namely calcium volume is estimated from abnormal region in the segmented bone image and it is concatenated with the extracted texture features to obtain the final feature vectors. In the final stage, the Multi-class Support Vector Machine (MSVM) takes feature vectors as a inputto classify bone disorders. The simulation result demonstrates that the proposed system achieved the accuracy of 95.1% and sensitivity of 96.15%.

Gray-level co-occurrence matrix of Fourier synchro-squeezed transform for epileptic seizure detection

Epilepsy is a brain disorder that many persons of different ages in the world suffer from it. According to the world health organization, epilepsy is characterized by repetitive seizures and more electrical discharge in a group of brain neurons results in sudden physical actions. The aim of this paper is to introduce a new method to classify epileptic phases based on Fourier synchro-squeezed transform (FSST) of electroencephalogram (EEG) signals. FSST is a time-frequency (TF) analysis and provides sharper TF estimates than the conventional short-time Fourier transform (STFT). Absolute of FSST of EEG signal is computed and segmented into five non-overlapping frequency sub-bands as delta (δ), theta (θ), alpha (α), beta (β), and gamma (γ). Each sub-band is considered as a gray-scale image and then we propose to obtain the gray-level co-occurrence matrix (GLCM) of each sub-band as features. We concatenate the features of different sub-bands to obtain the final feature vector. After selecting informative features by infinite latent feature selection (ILFS) method, the support vector machine (SVM) and K-nearest neighbor (KNN) classifiers are used separately to classify EEG signals. We use the EEG signals from Bonn University database and different combinations of its sets are considered. Simulation results show that the proposed method efficiently classifies the EEG signals and can be used to determine the phase of epilepsy.

A Multimodal Emotion Recognition System Using Facial Landmark Analysis

This paper introduces a multimodal emotion recognition system based on two different modalities, i.e., affective speech and facial expression. For affective speech, the common low-level descriptors including prosodic and spectral audio features (i.e., energy, zero crossing rate, MFCC, LPC, PLP and temporal derivatives) are extracted, whereas a novel visual feature extraction method is proposed in the case of facial expression. This method exploits the displacement of specific landmarks across consecutive frames of an utterance for feature extraction. To this end, the time series of temporal variations for each landmark is analyzed individually for extracting primary visual features, and then, the extracted features of all landmarks are concatenated for constructing the final feature vector. The analysis of displacement signal of landmarks is performed by the discrete wavelet transform which is a widely used mathematical transform in signal processing applications. In order to reduce the complexity of derived models and improve the efficiency, a variety of dimensionality-reduction schemes are applied. Furthermore, to exploit the advantages of multimodal emotion recognition systems, the feature-level fusion of the audio and the proposed visual features is examined. Results of experiments conducted on three SAVEE, RML and eNTERFACE05 databases show the efficiency of proposed visual feature extraction method in terms of performance criteria.

GPSuc: Global Prediction of Generic and Species-specific Succinylation Sites by aggregating multiple sequence features.

Lysine succinylation is one of the dominant post-translational modification of the protein that contributes to many biological processes including cell cycle, growth and signal transduction pathways. Identification of succinylation sites is an important step for understanding the function of proteins. The complicated sequence patterns of protein succinylation revealed by proteomic studies highlight the necessity of developing effective species-specific in silico strategies for global prediction succinylation sites. Here we have developed the generic and nine species-specific succinylation site classifiers through aggregating multiple complementary features. We optimized the consecutive features using the Wilcoxon-rank feature selection scheme. The final feature vectors were trained by a random forest (RF) classifier. With an integration of RF scores via logistic regression, the resulting predictor termed GPSuc achieved better performance than other existing generic and species-specific succinylation site predictors. To reveal the mechanism of succinylation and assist hypothesis-driven experimental design, our predictor serves as a valuable resource. To provide a promising performance in large-scale datasets, a web application was developed at http://kurata14.bio.kyutech.ac.jp/GPSuc/.