Compact Feature Vector Research Articles

AdaStreamLite

Streaming speech recognition aims to transcribe speech to text in a streaming manner, providing real-time speech interaction for smartphone users. However, it is not trivial to develop a high-performance streaming speech recognition system purely running on mobile platforms, due to the complex real-world acoustic environments and the limited computational resources of smartphones. Most existing solutions lack the generalization to unseen environments and have difficulty to work with streaming speech. In this paper, we design AdaStreamLite, an environment-adaptive streaming speech recognition tool for smartphones. AdaStreamLite interacts with its surroundings to capture the characteristics of the current acoustic environment to improve the robustness against ambient noise in a lightweight manner. We design an environment representation extractor to model acoustic environments with compact feature vectors, and construct a representation lookup table to improve the generalization of AdaStreamLite to unseen environments. We train our system using large speech datasets publicly available covering different languages. We conduct experiments in a large range of real acoustic environments with different smartphones. The results show that AdaStreamLite outperforms the state-of-the-art methods in terms of recognition accuracy, computational resource consumption and robustness against unseen environments.

ViralVectors: compact and scalable alignment-free virome feature generation.

The amount of sequencing data for SARS-CoV-2 is several orders of magnitude larger than any virus. This will continue to grow geometrically for SARS-CoV-2, and other viruses, as many countries heavily finance genomic surveillance efforts. Hence, we need methods for processing large amounts of sequence data to allow for effective yet timely decision-making. Such data will come from heterogeneous sources: aligned, unaligned, or even unassembled raw nucleotide or amino acid sequencing reads pertaining to the whole genome or regions (e.g., spike) of interest. In this work, we propose ViralVectors, a compact feature vector generation from virome sequencing data that allows effective downstream analysis. Such generation is based on minimizers, a type of lightweight "signature" of a sequence, used traditionally in assembly and read mapping - to our knowledge, the first use minimizers in this way. We validate our approach on different types of sequencing data: (a) 2.5M SARS-CoV-2 spike sequences (to show scalability); (b) 3K Coronaviridae spike sequences (to show robustness to more genomic variability); and (c) 4K raw WGS reads sets taken from nasal-swab PCR tests (to show the ability to process unassembled reads). Our results show that ViralVectors outperforms current benchmarks in most classification and clustering tasks. Graphical Abstract showing the all steps of proposed approach. We start by collecting the sequence-based data. Then Data cleaning and preprocessing is applied. After that, we generate the feature embeddings using minimizer based approach. Then Classification and clustering algorithms are applied on the resultant data and predictions are made on the test set.

Evolutionary Computation in Action: Hyperdimensional Deep Embedding Spaces of Gigapixel Pathology Images

One of the main obstacles of adopting digital pathology is the challenge of efficient processing of hyperdimensional digitized biopsy samples, called whole slide images (WSIs). Exploiting deep learning and introducing compact WSI representations are urgently needed to accelerate image analysis and facilitate the visualization and interpretability of pathology results in a postpandemic world. In this paper, we introduce a new evolutionary approach for WSI representation based on large-scale multi-objective optimization (LSMOP) of deep embeddings. We start with patch-based sampling to feed KimiaNet , a histopathology-specialized deep network, and to extract a multitude of feature vectors. Coarse multi-objective feature selection uses the reduced search space strategy guided by the classification accuracy and the number of features. In the second stage, the frequent features histogram (FFH), a novel WSI representation, is constructed by multiple runs of coarse LSMOP. Fine evolutionary feature selection is then applied to find a compact (short-length) feature vector based on the FFH and contributes to a more robust deep-learning approach to digital pathology supported by the stochastic power of evolutionary algorithms. We validate the proposed schemes using The Cancer Genome Atlas (TCGA) images in terms of WSI representation, classification accuracy, and feature quality. Furthermore, a novel decision space for multicriteria decision making in the LSMOP field is introduced. Finally, a patch-level visualization approach is proposed to increase the interpretability of deep features. The proposed evolutionary algorithm finds a very compact feature vector to represent a WSI (almost 14,000 times smaller than the original feature vectors) with 8% higher accuracy compared to the codes provided by the state-of-the-art methods.

Multi-field relation mining for malicious HTTP traffic detection based on attention and cross network

This paper introduces AutoHTTP, a novel end-to-end trainable framework for detecting malicious HTTP traffic. It can automatically analyze plain-text network traffic data without any manual labor and present an interpretable detection report for better human understanding. The purpose of the framework is to detect malicious HTTP traffic by mining multi-field inexplicit semantic characteristics and correlation. To conquer the problems in reality, we first divide the multi-field plain-texts (e.g. user-agent, URL, method) into two types: R-field and S-field. Then, an elementary feature extraction module is proposed to turn these fields into a compact field representation. Finally, the field interactions and significant parts of different fields are simultaneously extracted by feeding the compact feature vector into a newly proposed attention and cross network, which couples two important components, the attention portion and the cross part. We show that the network offers strong interpretability and reliable results for further analysis. Extensive experiments on CTU-13, CICAndMal, and ISCX-URL datasets demonstrate that our approach outperforms existing methods based on manually-designed features and other auto-designed features.

Evolutionary deep feature selection for compact representation of gigapixel images in digital pathology

Despite the recent progress in Deep Neural Networks (DNNs) to characterize histopathology images, compactly representing a gigapixel whole-slide image (WSI) via salient features to enable computational pathology is still an urgent need and a significant challenge. In this paper, we propose a novel WSI characterization approach to represent, search and classify biopsy specimens using a compact feature vector (CFV) extracted from a multitude of deep feature vectors. Since the non-optimal design and training of deep networks may result in many irrelevant and redundant features and also cause computational bottlenecks, we proposed a low-cost stochastic method to optimize the output of pre-trained deep networks using evolutionary algorithms to generate a very small set of features to accurately represent each tissue/biopsy. The performance of the proposed method has been assessed using WSIs from the publicly available TCGA image data. In addition to acquiring a very compact representation (i.e., 11,000 times smaller than the initial set of features), the optimized features achieved 93% classification accuracy resulting in 11% improvement compared to the published benchmarks. The experimental results reveal that the proposed method can reliably select salient features of the biopsy sample. Furthermore, the proposed approach holds the potential to immensely facilitate the adoption of digital pathology by enabling a new generation of WSI representation for efficient storage and more user-friendly visualization.

3D shearlet-based descriptors combined with deep features for the classification of Alzheimer's disease based on MRI data

Alzheimer's disease (AD) is a neurodegenerative disease that afflicts millions of people worldwide. Early detection of AD is critical, as drug trials show a promising advantage to those patients with early diagnoses. In this study, magnetic resonance imaging (MRI) datasets from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and The Open Access Series of Imaging Studies are used. Our method for performing the classification of AD is to combine a set of shearlet-based descriptors with deep features. A major challenge in classifying such MRI datasets is the high dimensionality of feature vectors because of the large number of slices of each MRI sample. Given the volumetric nature of the MRI data, we propose using the 3D shearlet transform (3D-ST), but we obtain the average of all directionalities, which reduces the dimensionality. On the other hand, we propose to leverage the capabilities of convolutional neural networks (CNN) to learn feature maps from stacked MRI slices, which generate a very compact feature vector for each MRI sample. The 3D-ST and CNN feature vectors are combined for the classification of AD. After the concatenation of the feature vectors, they are used to train a classifier. Alternatively, a custom CNN model is utilized, in which the descriptors are further processed end to end to obtain the classification model. Our experimental results show that the fusion of shearlet-based descriptors and deep features improves classification performance, especially on the ADNI dataset.

Robust video content analysis schemes for human action recognition.

Action recognition is a challenging time series classification task that has received much attention in the recent past due to its importance in critical applications, such as surveillance, visual behavior study, topic discovery, security, and content retrieval. The main objective of the research is to develop a robust and high-performance human action recognition techniques. A combination of local and holistic feature extraction methods used through analyzing the most effective features to extract to reach the objective, followed by using simple and high-performance machine learning algorithms. This paper presents three robust action recognition techniques based on a series of image analysis methods to detect activities in different scenes. The general scheme architecture consists of shot boundary detection, shot frame rate re-sampling, and compact feature vector extraction. This process is achieved by emphasizing variations and extracting strong patterns in feature vectors before classification. The proposed schemes are tested on datasets with cluttered backgrounds, low- or high-resolution videos, different viewpoints, and different camera motion conditions, namely, the Hollywood-2, KTH, UCF11 (YouTube actions), and Weizmann datasets. The proposed schemes resulted in highly accurate video analysis results compared to those of other works based on four widely used datasets. The First, Second, and Third Schemes provides recognition accuracies of 57.8%, 73.6%, and 52.0% on Hollywood2, 94.5%, 97.0%, and 59.3% on KTH, 94.5%, 95.6%, and 94.2% on UCF11, and 98.9%, 97.8% and 100% on Weizmann. Each of the proposed schemes provides high recognition accuracy compared to other state-of-art methods. Especially, the Second Scheme as it gives excellent comparable results to other benchmarked approaches.

Letter-Level Online Writer Identification

Writer identification (writer-id), an important field in biometrics, aims to identify a writer by their handwriting. Identification in existing writer-id studies requires a complete document or text, limiting the scalability and flexibility of writer-id in realistic applications. To make the application of writer-id more practical (e.g., on mobile devices), we focus on a novel problem, letter-level online writer-id, which requires only a few trajectories of written letters as identification cues. Unlike text-\(\backslash \) document-based writer-id which has rich context for identification, there are much fewer clues to recognize an author from only a few single letters. A main challenge is that a person often writes a letter in different styles from time to time. We refer to this problem as the variance of online writing styles (Var-O-Styles). We address the Var-O-Styles in a capture-normalize-aggregate fashion: Firstly, we extract different features of a letter trajectory by a carefully designed multi-branch encoder, in an attempt to capture different online writing styles. Then we convert all these style features to a reference style feature domain by a novel normalization layer. Finally, we aggregate the normalized features by a hierarchical attention pooling (HAP), which fuses all the input letters with multiple writing styles into a compact feature vector. In addition, we also contribute a large-scale LEtter-level online wRiter IDentification dataset (LERID) for evaluation. Extensive comparative experiments demonstrate the effectiveness of the proposed framework.

A compact and scalable representation of network traffic dynamics using shapes and its applications

The biggest challenge of analysing network traffic dynamics of large-scale networks is its complexity and pattern interpretability. In this work, we present a new computationally efficient method, inspired by human vision, to reduce the dimensions of a large-scale network and describe the traffic conditions with a compact, scalable and interpretable custom feature vector. This is done by extracting pockets of congestion that encompass connected 3D subnetworks as 3D shapes. We then parameterize these 3D shapes as 2D projections and construct parsimonious feature vectors from these projections. There are various applications of these feature vectors such as revealing the day-to-day regularity of the congestion patterns and building a classification model that allows us to predict travel time from any origin to any destination in the network. We demonstrate that our method achieves a 44% accuracy improvement when compared against the consensus method for travel prediction of an urban network of Amsterdam. Our method also outperforms historical average methods, especially for days with severe congestion. Furthermore, we demonstrate the scalability of the approach by applying the method on the entire Dutch highway network and show that the feature vector was able to encapsulate the network dynamics with a 93% prediction accuracy. There are many paths to further refine and improve the method. The compact form of the feature vector allows us to efficiently enrich it with more information such as context, weather and event without increasing the computational complexity.

Multi-Scale Deep Representation Aggregation for Vein Recognition

The recent success of Deep Convolutional Neural Network (DCNN) for various computer vision tasks such as image recognition has already demonstrated its robust feature representation ability. However, the limitation of training database on small scale vein recognition tasks restricts its performance because the recognition result of DCNN depends heavily on the number of trainsets. This motivates the design of a Multi-Scale Deep Representation Aggregation (MSDRA) model based on a pre-trained DCNN for vein recognition. First, the multi-scale feature maps are extracted by a pre-trained DCNN model. Second, a local mean threshold approach is designed to preliminarily remove the noisy information of multi-scale feature maps and generate the selected feature maps. Third, we propose an Unsupervised Vein Information Mining (UVIM) method to localize vein information of selected feature maps for generating a binary vein information mask, and then the vein information mask is utilized to keep useful deep representation and discard the background information. Finally, the discriminative multi-scale deep representations, which are generated by using the vein information mask to aggregate multi-scale feature maps, are concatenated into the final compact feature vectors, and then a Support Vector Machine (SVM) is introduced for final recognition. Our proposed model outperforms the state-of-the-art methods on two benchmark vein databases. Moreover, an additional experiment using the subset of PolyU Palmprint database illustrates the system’s generalization ability and robustness.

SDAE-GAN: Enable high-dimensional pathological images in liver cancer survival prediction with a policy gradient based data augmentation method.

High-dimensional pathological images produced by Immunohistochemistry (IHC) methods consist of many pathological indexes, which play critical roles in cancer treatment planning. However, these indexes currently cannot be utilized in survival prediction because joining them with patients' clinicopathological features (e.g., age and tumor size) is challenging due to their high dimension and sparse features. To address this problem, we propose a novel two-stage survival prediction model named ICSPM to join the IHC images and clinicopathological features. For the first stage, our proposed SDAE-GAN compresses high-dimensional IHC images to flat, compact and representative feature vectors by compressing and reconstructing them. For the first time, SDAE-GAN integrates dense blocks, the stacked auto-encoder and the GAN architecture to maximize the ability to detect patterns in IHC images. In addition, we propose a novel policy gradient based data augmentation method to involve the diversity in IHC images without breaking patterns inside them. For the second stage, ICSPM adopts a DenseNet to join feature vectors and clinicopathological features for survival prediction. Experimental results demonstrate that ICSPM reached a state-of-the-art prediction accuracy of 0.72 on the five-year survival. ICSPM is the first work to enable high-dimensional IHC images in cancer survival prediction. We prove that high-dimensional IHC images and clinicopathological features provide valuable and complementary information in survival prediction.

Texture Classification Using Pair-wise Difference Pooling Based Bilinear Convolutional Neural Networks.

Texture is normally represented by aggregating local features based on the assumption of spatial homogeneity. Effective texture features are always the research focus even though both hand-crafted and deep learning approaches have been extensively investigated. Motivated by the success of Bilinear Convolutional Neural Networks (BCNNs) in fine-grained image recognition, we propose to incorporate the BCNN with the Pair-wise Difference Pooling (i.e. BCNN-PDP) for texture classification. The BCNN-PDP is built on top of a set of feature maps extracted at a convolutional layer of the pre-trained CNN. Compared with the outer product used by the original BCNN feature set, the pair-wise difference not only captures the pair-wise relationship between two sets of features but also encodes the difference between each pair of features. Considering the importance of the gradient data to the representation of image structures, we further generalise the BCNN-PDP feature set to two sets of feature maps computed from the original image and its gradient magnitude map respectively, i.e. the Fused BCNN-PDP (F-BCNN-PDP) feature set. In addition, the BCNN-PDP can be applied to two different CNNs and is referred to as the Asymmetric BCNN-PDP (A-BCNN-PDP). The three PDP-based BCNN feature sets can also be extracted at multiple scales. Since the dimensionality of the BCNN feature vectors is very high, we propose a new yet simple Block-wise PCA (BPCA) method in order to derive more compact feature vectors. The proposed methods are tested on seven different datasets along with 21 baseline feature sets. The results show that the proposed feature sets are superior, or at least comparable, to their counterparts across different datasets.

Statistical tree-based feature vector for content-based image retrieval

The efficiency of any content-based image retrieval system depends on the extracted feature vectors of individual images stored in the database. Generation of a compact feature vector with good discriminative power is a real challenge in the image retrieval system. This paper presents the experimentation carried out to generate compact feature vectors for a colour image retrieval system based on image content. It has two stages of operation. In first stage the energy compaction property of image transforms is used whereas in the second stage, the statistical tree approach is used for feature vector generation. Performance of image retrieval is tested using image feature database as per various performance evaluation parameters such as precision recall crossover point (PRCP) along with newly proposed conflicting string of images (CSI). With different colour spaces, image transforms and statistical measures, proposed approach achieves the reduction in the feature vector size with better discriminative power.

A novel method for digital image copy-move forgery detection and localization using evolving cellular automata and local binary patterns

Copy-Move Forgery Detection (CMFD) methods aim to forensically analyze a digital image for a possible content duplication manipulation. In the past, many block-based algorithms have been proposed for detection and localization of CMF. However, the existing solutions show limited efficacy for images compressed in JPEG and lack robustness against post-processing attacks such as noise addition, blurring, etc. To address this problem, we propose a new block-based passive method for detection and localization of CMF in this paper. Passive methods, as opposed to active methods, are used to authenticate the image content in the absence of any pre-embedded information such as watermarks. In our proposed scheme, a suspicious input image to be analyzed is first low pass filtered and converted to Local Binary Patterns (LBP) image. The LBP texture image is then divided into overlapping blocks. Next, a compact five-dimensional feature vector is extracted from each block by employing thresholding and Cellular Automata. The set of feature vectors is sorted lexicographically to bring the copy-pasted blocks nearer to each other. Finally, the feature matching step is used to reveal the duplicate blocks. Our experimental results indicate that the proposed method performs exceptionally well relative to other state-of-art-methods, under different image manipulation scenarios.

Multi-sensor fusion based on multiple classifier systems for human activity identification

Multimodal sensors in healthcare applications have been increasingly researched because it facilitates automatic and comprehensive monitoring of human behaviors, high-intensity sports management, energy expenditure estimation, and postural detection. Recent studies have shown the importance of multi-sensor fusion to achieve robustness, high-performance generalization, provide diversity and tackle challenging issue that maybe difficult with single sensor values. The aim of this study is to propose an innovative multi-sensor fusion framework to improve human activity detection performances and reduce misrecognition rate. The study proposes a multi-view ensemble algorithm to integrate predicted values of different motion sensors. To this end, computationally efficient classification algorithms such as decision tree, logistic regression and k-Nearest Neighbors were used to implement diverse, flexible and dynamic human activity detection systems. To provide compact feature vector representation, we studied hybrid bio-inspired evolutionary search algorithm and correlation-based feature selection method and evaluate their impact on extracted feature vectors from individual sensor modality. Furthermore, we utilized Synthetic Over-sampling minority Techniques (SMOTE) algorithm to reduce the impact of class imbalance and improve performance results. With the above methods, this paper provides unified framework to resolve major challenges in human activity identification. The performance results obtained using two publicly available datasets showed significant improvement over baseline methods in the detection of specific activity details and reduced error rate. The performance results of our evaluation showed 3% to 24% improvement in accuracy, recall, precision, F-measure and detection ability (AUC) compared to single sensors and feature-level fusion. The benefit of the proposed multi-sensor fusion is the ability to utilize distinct feature characteristics of individual sensor and multiple classifier systems to improve recognition accuracy. In addition, the study suggests a promising potential of hybrid feature selection approach, diversity-based multiple classifier systems to improve mobile and wearable sensor-based human activity detection and health monitoring system.

An effective image retrieval system using machine learning and fuzzy c- means clustering approach

Recently, Content-based medical image retrieval (CBMIR) systems enable fast diagnosis via the assessment of the visual information in medical application. Most of the state-of-the-art CBMIR systems facing few issues: computationally expensive due to the usage of high dimensional feature vectors and complex classifier/clustering approaches. The reasons behind this are, inability to properly handle the “semantic gap” and the high intra-class versus inter-class variability problem of the medical image database (like radiographic image database). This yields a crucial demand for developing computationally efficient and highly effective retrieval system. For this purpose, the present study proposed an efficient retrieval system which has a four-fold: First, pre-processing and Feature extraction of input image using canonical correlation analysis (CCA). By this approach extracted the feature in both pixel and feature domains and examined more rigorously. Second, applied Fuzzy C means clustering of pixel intensity values as features based on the singular value decomposition. Through this can grouping, the image based on the pixel intensity value. Third, deep convolutional neural network with SVM classifier which makes implementable and requires only a compact feature vector representation of the stored database image with their class levels during retrieval. Finally evaluated the performance based on the measure of Mean Average Precision, Correct rate (CR), Error rate (ER), Accuracy. The classification results and learned features are used for the purpose of retrieving the medical images in a database. The proposed retrieval system performs better than the traditional approach in terms of measuring average value of precision, recall, f-measure and accuracy 95.9%, 94.96%, 95.37% and 95.798% respectively. The suggested approach is best suited towards retrieving the medical images for various part of the body.

Multi-Weighted Co-Occurrence Descriptor Encoding for Vein Recognition

Despite being highly secure, vein recognition suffers from the high inter-class similarity and intra-class variation resulting from the uncontrolled image capture, making the design of discriminative and robust representation very important. The recent success of convolutional neural network (CNN) for various image understanding tasks makes it a promising method for feature extraction. However, limited variability in small-scale datasets leads to systems derived from the direct training or fine-tuning not transferable and unreliable for practical biometric applications. This motivates the design of a multi-weighted co-occurrence descriptor encoding (MWCDE) model for vein recognition. Instead of directly conducting a feed-forward operation with a pre-trained CNN for obtaining the semantic features from the fully connected layers, co-occurrence features among convolutional filters are modeled first in MWCDE by a simple convolution between an indicator filter in a higher layer with a to-be-reweighted filter in a lower layer, and a redundancy-driven indicator filter selection algorithm is designed for filtering out some ambiguous representations. Second, another hard feature weighting strategy with a binary masking scheme is proposed for discarding noisy background and feature redundancy. The selected high-order descriptors are then embedded and aggregated into the compact feature vectors with a saliency driven spatial weighted Fisher vector algorithm, followed by the introduction of a generalized support vector machine for recognition. Extensive experiments with three benchmark vein datasets demonstrate that the proposed framework can achieve state-of-the-art results, and an additional experiment with the PolyU multispectral palmprint database illustrates its generalization ability. Code is available at (https://github.com/RobinCSIRO/MWCDE-for-Vein-Recognition).

A Novel Sketch-Based Three-Dimensional Shape Retrieval Method Using Multi-View Convolutional Neural Network

Retrieving 3D models by adopting hand-drawn sketches to be the input has turned out to be a popular study topic. Most current methods are based on manually selected features and the best view produced for 3D model calculations. However, there are many problems with these methods such as distortion. For the purpose of dealing with such issues, this paper proposes a novel feature representation method to select the projection view and adapt the maxout network to the extended Siamese network architecture. In addition, the strategy is able to handle the over-fitting issue of convolutional neural networks (CNN) and mitigate the discrepancies between the 3D shape domain and the sketch. A pre-trained AlexNet was used to sketch the extract features. For 3D shapes, multiple 2D views were compiled into compact feature vectors using pre-trained multi-view CNNs. Then the Siamese convolutional neural networks were learnt for transforming the two domains’ original characteristics into nonlinear feature space, which mitigated the domain discrepancy and kept the discriminations. Two large data sets were used for experiments, and the experimental results show that the method is superior to the prior art methods in accuracy.

Statistical tree-based feature vector for content-based image retrieval

The efficiency of any content-based image retrieval system depends on the extracted feature vectors of individual images stored in the database. Generation of a compact feature vector with good discriminative power is a real challenge in the image retrieval system. This paper presents the experimentation carried out to generate compact feature vectors for a colour image retrieval system based on image content. It has two stages of operation. In first stage the energy compaction property of image transforms is used whereas in the second stage, the statistical tree approach is used for feature vector generation. Performance of image retrieval is tested using image feature database as per various performance evaluation parameters such as precision recall crossover point (PRCP) along with newly proposed conflicting string of images (CSI). With different colour spaces, image transforms and statistical measures, proposed approach achieves the reduction in the feature vector size with better discriminative power.

Acoustic Scene Classification Using Efficient Summary Statistics and Multiple Spectro-Temporal Descriptor Fusion

This paper presents a novel approach for acoustic scene classification based on efficient acoustic feature extraction using spectro-temporal descriptors fusion. Grounded on the finding in neuroscience—“auditory system summarizes the temporal details of sounds using time-averaged statistics to understand acoustic scenes”, we devise an efficient computational framework for sound scene classification by using multipe time-frequency descriptors fusion with discriminant information enhancement. To characterize rich information of sound, i.e., local structures on the time-frequency plane, we adopt 2-dimensional local descriptors. A more critical issue raised in how to logically ‘summarize’ those local details into a compact feature vector for scene classification. Although ‘time-averaged statistics’ is suggested by the psychological investigation, directly computing time average of local acoustic features is not a logical way, since arithmetic mean is vulnerable to extreme values which are anticipated to be generated by interference sounds which are irrelevant to the scene category. To tackle this problem, we develop time-frame weighting approach to enhance sound textures as well as to suppress scene-irrelevant events. Subsequently, robust acoustic feature for scene classification can be efficiently characterized. The proposed method had been validated by using Rouen dataset which consists of 19 acoustic scene categories with 3029 real samples. Extensive results demonstrated the effectiveness of the proposed scheme.