Bag Of Visual Words Research Articles

A Semi-Direct Monocular Visual SLAM Algorithm in Complex Environments

A novel monocular visual simultaneous localization and mapping (SLAM) algorithm built on the semi-direct method is proposed to deal with some problems in complex environments, such as low-texture, moving objects and perceptual aliasing. The proposed algorithm takes advantage of direct and feature-based methods. On one hand, a direct method is used to track the camera poses and solve the feature alignment. On the other hand, ORB features in keyframes are extracted and matched for optimization and loop closure. To improve the localization accuracy in dynamic environments, a motion detection module that is robust to illumination change is adopted. In addition, for the sake of resolving the loop closure detection problem in perceptual aliasing scenes, this paper fuses the spatial information between two visual words into the bag of visual words (BoVW) model and employs an improved pyramid term frequency-inverse document frequency (TF-IDF) scoring match scheme. Experimental results prove that the proposed algorithm behaves better performance than ORB-SLAM with regard to overall accuracy and speed in complex environments.

Comparison of segmentation-free and segmentation-dependent computer-aided diagnosis of breast masses on a public mammography dataset

PurposeTo compare machine learning methods for classifying mass lesions on mammography images that use predefined image features computed over lesion segmentations to those that leverage segmentation-free representation learning on a standard, public evaluation dataset. MethodsWe apply several classification algorithms to the public Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM), in which each image contains a mass lesion. Segmentation-free representation learning techniques for classifying lesions as benign or malignant include both a Bag-of-Visual-Words (BoVW) method and a Convolutional Neural Network (CNN). We compare classification performance of these techniques to that obtained using two different segmentation-dependent approaches from the literature that rely on specific combinations of end classifiers (e.g. linear discriminant analysis, neural networks) and predefined features computed over the lesion segmentation (e.g. spiculation measure, morphological characteristics, intensity metrics). ResultsWe report area under the receiver operating characteristic curve (AZ) values for malignancy classification on CBIS-DDSM for each technique. We find average AZ values of 0.73 for a segmentation-free BoVW method, 0.86 for a segmentation-free CNN method, 0.75 for a segmentation-dependent linear discriminant analysis of Rubber-Band Straightening Transform features, and 0.58 for a hybrid rule-based neural network classification using a small number of hand-designed features. ConclusionsWe find that malignancy classification performance on the CBIS-DDSM dataset using segmentation-free BoVW features is comparable to that of the best segmentation-dependent methods we study, but also observe that a common segmentation-free CNN model substantially and significantly outperforms each of these (p < 0.05). These results reinforce recent findings suggesting that representation learning techniques such as BoVW and CNNs are advantageous for mammogram analysis because they do not require lesion segmentation, the quality and specific characteristics of which can vary substantially across datasets. We further observe that segmentation-dependent methods achieve performance levels on CBIS-DDSM inferior to those achieved on the original evaluation datasets reported in the literature. Each of these findings reinforces the need for standardization of datasets, segmentation techniques, and model implementations in performance assessments of automated classifiers for medical imaging.

Scene Based Classification of Aerial Images using Convolution Neural Networks

The advent of computer vision and evolution of high-end computing in remote sensing images have embellish various researchers for unprecedented development in remotely sensed aerial images. The requirement to extract essential information stimulated anatomization of aerial images for its usefulness. Deep learning provides state of the art solutions for widely explored visual recognition system and has emerged as an evolutionary area, being applicable to large scale image processing applications. Convolutional Neural Networks (CNNs), an essential component of deep learning algorithms consists of increasing the depth and connections in the processing layers to learn various features of data at different abstract levels. . In this paper, we present an outlook for classifying and extracting the features of aerial images using CNN. We propose a CNN architecture based on various parameters and layers for classification. CNN has been evaluated on two publicly available aerial data sets: UCMerced Land Use and RSSCN7. Experimental results show that the proposed CNN architecture is competent and efficient in terms of accuracy as performance evaluation parameter in comparison with conventional classifiers like Bag of Visual Words (BOVW).

The importance of feature aggregation in radiomics: a head and neck cancer study

In standard radiomics studies the features extracted from clinical images are mostly quantified with simple statistics such as the average or variance per Region of Interest (ROI). Such approaches may smooth out any intra-region heterogeneity and thus hide some tumor aggressiveness that may hamper predictions. In this paper we study the importance of feature aggregation within the standard radiomics workflow, which allows to take into account intra-region variations. Feature aggregation methods transform a collection of voxel values from feature response maps (over a ROI) into one or several scalar values that are usable for statistical or machine learning algorithms. This important step has been little investigated within the radiomics workflows, so far. In this paper, we compare several aggregation methods with standard radiomics approaches in order to assess the improvements in prediction capabilities. We evaluate the performance using an aggregation function based on Bags of Visual Words (BoVW), which allows for the preservation of piece-wise homogeneous information within heterogeneous regions and compared with standard methods. The different models are compared on a cohort of 214 head and neck cancer patients coming from 4 medical centers. Radiomics features were extracted from manually delineated tumors in clinical PET-FDG and CT images were analyzed. We compared the performance of standard radiomics models, the volume of the ROI alone and the BoVW model for survival analysis. The average concordance index was estimated with a five fold cross-validation. The performance was significantly better using the BoVW model 0.627 (95% CI: 0.616–0.637) as compared to standard radiomics0.505 (95% CI: 0.499–0.511), mean-var. 0.543 (95% CI: 0.536–0.549), mean0.547 (95% CI: 0.541–0.554), var.0.530 (95% CI: 0.524–0.536) or volume 0.577 (95% CI: 0.571–0.582). We conclude that classical aggregation methods are not optimal in case of heterogeneous tumors. We also showed that the BoVW model is a better alternative to extract consistent features in the presence of lesions composed of heterogeneous tissue.

Fusing Visual Attention CNN and Bag of Visual Words for Cross-Corpus Speech Emotion Recognition.

Speech emotion recognition (SER) classifies emotions using low-level features or a spectrogram of an utterance. When SER methods are trained and tested using different datasets, they have shown performance reduction. Cross-corpus SER research identifies speech emotion using different corpora and languages. Recent cross-corpus SER research has been conducted to improve generalization. To improve the cross-corpus SER performance, we pretrained the log-mel spectrograms of the source dataset using our designed visual attention convolutional neural network (VACNN), which has a 2D CNN base model with channel- and spatial-wise visual attention modules. To train the target dataset, we extracted the feature vector using a bag of visual words (BOVW) to assist the fine-tuned model. Because visual words represent local features in the image, the BOVW helps VACNN to learn global and local features in the log-mel spectrogram by constructing a frequency histogram of visual words. The proposed method shows an overall accuracy of 83.33%, 86.92%, and 75.00% in the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), the Berlin Database of Emotional Speech (EmoDB), and Surrey Audio-Visual Expressed Emotion (SAVEE), respectively. Experimental results on RAVDESS, EmoDB, SAVEE demonstrate improvements of 7.73%, 15.12%, and 2.34% compared to existing state-of-the-art cross-corpus SER approaches.

Image Classification and Detection of Insulators using Bag of Visual Words and Speeded up Robust Features

Electrical substation online monitoring in computer vision technology is based on image processing algorithm to perform visual analysis. This paper presents classification of ceramicand glass insulators through Bag of Visual Words and detection of these insulators by Point Feature Matching. The training image datasets are used for categorization by forming a visual vocabulary while a new unlabeled image from testing image dataset is classify using nearest neighbor classification method for features descriptor. For detection we use Speeded up Robust Features for detecting position of insulator present in cluttered scene image. Matching process is done between test and reference image and decision is made based on similar features. We conducted experiment on insulators to verify the superiority of our proposed method. The proposed method can be used in security, surveillance and inspection system

An Improved Plant Identification System by Fuzzy c-means Bag of Visual Words Model and Sparse Coding

An Improved Plant Identification System by Fuzzy c-means Bag of Visual Words Model and Sparse Coding

Hybrid Learning of Hand-Crafted and Deep-Activated Features Using Particle Swarm Optimization and Optimized Support Vector Machine for Tuberculosis Screening

Tuberculosis (TB) is a leading infectious killer, especially for people with Human Immunodeficiency Virus (HIV) and Acquired Immunodeficiency Syndrome (AIDS). Early diagnosis of TB is crucial for disease treatment and control. Radiology is a fundamental diagnostic tool used to screen or triage TB. Automated chest x-rays analysis can facilitate and expedite TB screening with fast and accurate reports of radiological findings and can rapidly screen large populations and alleviate a shortage of skilled experts in remote areas. We describe a hybrid feature-learning algorithm for automatic screening of TB in chest x-rays: it first segmented the lung regions using the DeepLabv3+ model. Then, six sets of hand-crafted features from statistical textures, local binary pattern, GIST, histogram of oriented gradients (HOG), pyramid histogram of oriented gradients and bags of visual words (BoVW), and nine sets of deep-activated features from AlexNet, GoogLeNet, InceptionV3, XceptionNet, ResNet-50, SqueezeNet, ShuffleNet, MobileNet, and DenseNet, were extracted. The dominant features of each feature set were selected using particle swarm optimization, and then separately input to an optimized support vector machine classifier to label ‘normal’ and ‘TB’ x-rays. GIST, HOG, BoVW from hand-crafted features, and MobileNet and DenseNet from deep-activated features performed better than the others. Finally, we combined these five best-performing feature sets to build a hybrid-learning algorithm. Using the Montgomery County (MC) and Shenzen datasets, we found that the hybrid features of GIST, HOG, BoVW, MobileNet and DenseNet, performed best, achieving an accuracy of 92.5% for the MC dataset and 95.5% for the Shenzen dataset.

Tuberculosis bacteria analysis in acid fast stained images of sputum smear

Tuberculosis (TB) basically originates due to bacteria, influences the developing nations and disrupts their economy severely. TB like diseases are with a high mortality rate worldwide, but early detection highly increases the chances of survival. This paper presents a novel method for TB bacteria segmentation and classification using microscopic images (MI). Manual identification of the bacterial cell is a very difficult process. The automation in TB bacteria detection is the objective of this article using MI processing. The proposed segmentation method first performs the image enhancement followed by bacteria region masking. Further, the marking of bacteria points is performed by the marked point process model. Finally, the complete bacteria are identified by the superellipse and supervised variational contour models. The features are extracted using bag of visual words and handcrafted work for the image classification. Simulation results confirm the superiority of the proposed method as compared with the state-of-the-art methods.

Auxiliary criterion conversion via spatiotemporal semantic encoding and feature entropy for action recognition

Video-based action recognition in realistic scenes is a core technology for human–computer interaction and smart surveillance. Although the trajectory features with the bag of visual words have confirmed promising performance, spatiotemporal interactive information cannot be effectively encoded which is valuable for classification. To address this issue, we propose a spatiotemporal semantic feature (ST-SF) and implement the conversion of it to the auxiliary criterion based on the information entropy theory. First, we present a text-based relevance analysis method to estimate the textual labels of objects most relevant to actions, which are employed to train the more targeted detectors based on the deep network. False detections are optimized by the inter-frame cooperativity and dynamic programming to construct the valid tubes. Then, we design the ST-SF to encode the interactive information, and the concept and calculation of feature entropy are defined based on the spatial distribution of ST-SFs on the training set. Finally, we achieve a two-stage classification strategy using the resulting decision gains. Experimental results on three publicly available datasets demonstrate that our method is robust and improves upon the state-of-the-art algorithms.

RETRACTED ARTICLE: Computer aided diagnosis of brain tumor using novel classification techniques

Brain cancer treatment mainly depends on the accurate detection of the tumor type, location, size and borders. Magnetic resonance images (MRI) can be used to analyze the properties of the desired region such as tissues and tumors with automated and semi-automated approaches. So, the extraction of MRI brain tumor image is a challenging task in medical image processing. The major problems associated with MRI analysis by a physician are time consuming and the accuracy depends on the expertise of the physician.This limitation can be overcome by the computer aided diagnosis (CAD) technology. In this paper, a CAD system is designed to detect brain tumors with computer assistance using T1 and T2 weighted MR images. The designed system classifies the tumor into benign or malignant from MR Image using a novel automated method which increases the performance and reduces the complexity involved in the tumor diagnosis. The CAD system has four stages such as image acquisition, segmentation, feature extraction and classification. Segmentation is done with the help of K-means clustering, which enhances the medical image and the clustering quality to avoid local optima and to find global optima. The feature extraction is performed by gray level co-occurrence matrix (GLCM). The tumor classification is done using support vector machine (SVM) and bag of visual words (BOVW) classifiers. The test result of the SVM classifier gives accuracy 95.0%, sensitivity 91.79% and specificity 94.75%. Whereas, the BOVW classifier yields results of accuracy 96.0%, sensitivity 90.0% and specificity 100%. This result shows that, the designed system classifies the tumor into benign or malignant with a good level of accuracy.

VisDroid: Android malware classification based on local and global image features, bag of visual words and machine learning techniques

In this paper, VisDroid, a novel generic image-based classification method has been suggested and developed for classifying the Android malware samples into its families. To this end, five grayscale image datasets each of which contains 4850 samples have been constructed based on different files from the contents of the Android malware samples sources. Two types of image-based features have been extracted and used to train six machine learning classifiers including Random Forest, K-nearest neighbour, Decision trees, Bagging, AdaBoost and Gradient Boost classifiers. The first type of the extracted features is local features including Scale-Invariant Feature Transform, Speeded Up Robust Features, Oriented FAST and Rotated BRIEF (ORB) and KAZE features. The second type of the extracted features is global features including Colour Histogram, Hu Moments and Haralick Texture. Furthermore, a hybridized ensemble voting classifier has been proposed to test the efficiency of using a number of machine learning classifiers trained using local and global features as voters to make a decision in an ensemble voting classifier. Moreover, two well-known deep learning model, i.e. Residual Neural Network and Inception-v3 have been tested using some of the constructed image datasets. Furthermore, when the results of the proposed model have been compared with the results of some state-of-art works it has been revealed that the proposed model outperforms the compared previous models in term of classification accuracy, computational time, generality and classification mode.

BSV-PAGS: Blockchain-based special vehicles priority access guarantee scheme

Traffic jams often occur, with the increasing number of private cars. Special vehicles (e.g. police cars, ambulances and fire trucks) often encounter traffic jams when carrying out emergency tasks, which may lead to unexpected consequences (e.g. causing serious damage to our routine and property). In order to ensure that special vehicles can quickly pass through crowded road and arrive at the accident scene in time, this paper proposes a blockchain-based special vehicles priority access guarantee scheme (BSV-PAGS). In BSV-PAGS, we introduce blockchain into the Internet of Things (IoT), and construct a decentralized distributed edge node network based on blockchian, which guarantees the information sharing among the edge nodes as well as processing the data efficiently. The intelligent terminal device will upload the special vehicle information to the edge node in the blockchain, and then the uploaded information will sync to each edge node through the information sharing smart contract, realizing the information sharing among the edge nodes. In addition, we use machine learning method (e.g. Bag of Visual Words and Support Vector Machine) to design and train special vehicle detectors to improve the recognition accuracy of special vehicles. We design a special vehicle priority access smart contract to achieve real-time scheduling of vehicles and ultimately ensure the priority of special vehicles. Then, we carried out a security analysis of BSV-PAGS. Finally, we test a large number of simulation experiments based on Ethereum clients. The experimental results show that our scheme is feasible and effective.

CNN Feature-Based Image Copy Detection with Contextual Hash Embedding

As one of the important techniques for protecting the copyrights of digital images, content-based image copy detection has attracted a lot of attention in the past few decades. The traditional content-based copy detection methods usually extract local hand-crafted features and then quantize these features to visual words by the bag-of-visual-words (BOW) model to build an inverted index file for rapid image matching. Recently, deep learning features, such as the features derived from convolutional neural networks (CNN), have been proven to outperform the hand-crafted features in many applications of computer vision. However, it is not feasible to directly apply the existing global CNN features for copy detection, since they are usually sensitive to partial content-discarded attacks, such as copping and occlusion. Thus, we propose a local CNN feature-based image copy detection method with contextual hash embedding. We first extract the local CNN features from images and then quantize them to visual words to construct an index file. Then, as the BOW quantization process decreases the discriminability of these features to some extent, a contextual hash sequence is captured from a relatively large region surrounding each CNN feature and then is embedded into the index file to improve the feature’s discriminability. Extensive experimental results demonstrate that the proposed method achieves a superior performance compared to the related works in the copy detection task.

Automatic Detection of Pornographic and Gambling Websites Based on Visual and Textual Content Using a Decision Mechanism.

Pornographic and gambling websites become increasingly stubborn via disguising, misleading, blocking, and bypassing, which hinder the construction of a safe and healthy network environment. However, most traditional approaches conduct the detection process through a single aspect of these sites, which would fail to handle the more intricate and challenging situations. To alleviate this problem, this study proposed an automatic detection system for porn and gambling websites based on visual and textual content using a decision mechanism (PG-VTDM). This system can be applied to the intelligent wireless router at home or school to realize the identification, blocking, and warning of ill-suited websites. First, Doc2Vec was employed to learn the textual features that can be used to represent the textual content in the hypertext markup language (HTML) source code of the websites. In addition, the traditional bag-of-visual-words (BoVW) was improved by introducing local spatial relationships of feature points for better representing the visual features of the website screenshot. Then, based on these two types of features, a text classifier and an image classifier were both trained. In the decision mechanism, a data fusion algorithm based on logistic regression (LR) was designed to obtain the final prediction result by measuring the contribution of the two classification results to the final category prediction. The efficiency of this proposed approach was substantiated via comparison experiments using gambling and porn website datasets crawled from the Internet. The proposed approach outperformed the approach based on a single feature and some state-of-the-art approaches, with accuracy, precision, and F-measure all over 99%.

BoVW model based on adaptive local and global visual words modeling and log-based relevance feedback for semantic retrieval of the images

The core of a content-based image retrieval (CBIR) system is based on an effective understanding of the visual contents of images due to which a CBIR system can be termed as accurate. One of the most prominent issues which affect the performance of a CBIR system is the semantic gap. It is a variance that exists between low-level patterns of an image and high-level abstractions as perceived by humans. A robust image visual representation and relevance feedback (RF) can bridge this gap by extracting distinctive local and global features from the image and by incorporating valuable information stored as feedback. To handle this issue, this article presents a novel adaptive complementary visual word integration method for a robust representation of the salient objects of the image using local and global features based on the bag-of-visual-words (BoVW) model. To analyze the performance of the proposed method, three integration methods based on the BoVW model are proposed in this article: (a) integration of complementary features before clustering (called as non-adaptive complementary feature integration), (b) integration of non-adaptive complementary features after clustering (called as a non-adaptive complementary visual words integration), and (c) integration of adaptive complementary feature weighting after clustering based on self-paced learning (called as a proposed method based on adaptive complementary visual words integration). The performance of the proposed method is further enhanced by incorporating a log-based RF (LRF) method in the proposed model. The qualitative and quantitative analysis of the proposed method is carried on four image datasets, which show that the proposed adaptive complementary visual words integration method outperforms as compared with the non-adaptive complementary feature integration, non-adaptive complementary visual words integration, and state-of-the-art CBIR methods in terms of performance evaluation metrics.

Robust Place Recognition Using Illumination-compensated Image-based Deep Convolutional Autoencoder Features

Place recognition is a method for determining whether a robot has previously visited the place it currently observes, thus helping the robot correct its accumulated position error. Ultimately, the robot will travel long distances more accurately. Conventional image-based place recognition uses features extracted from a bag-of-visual-words (BoVW) scheme or pre-trained deep neural network. However, the BoVW scheme does not cope well with environmental changes, and the pre-trained deep neural network is disadvantageous in that its computation time is high. Therefore, this paper proposes a novel place recognition scheme using an illumination-compensated image-based deep convolutional autoencoder (ICCAE) feature. Instead of reconstructing the raw image, the autoencoder designed to extract ICCAE features is trained to reconstruct the image, whose illumination component is compensated in the logarithm frequency domain. As a result, we can extract the ICCAE features based on a convolution layer that is robust to illumination and environmental changes. Additionally, ICCAE features can perform faster feature matching than the features extracted from existing deep networks. To evaluate the performance of ICCAE feature-based place recognition, experiments were conducted using a public dataset that includes various conditions.

Pooling region learning of visual word for image classification using bag-of-visual-words model.

In the problem where there is not enough data to use Deep Learning, Bag-of-Visual-Words (BoVW) is still a good alternative for image classification. In BoVW model, many pooling methods are proposed to incorporate the spatial information of local feature into the image representation vector, but none of the methods devote to making each visual word have its own pooling regions. The practice of designing the same pooling regions for all the words restrains the discriminability of image representation, since the spatial distributions of the local features indexed by different visual words are not same. In this paper, we propose to make each visual word have its own pooling regions, and raise a simple yet effective method for learning pooling region. Concretely, a kind of small window named observation window is used to obtain its responses to each word over the whole image region. The pooling regions of each word are organized by a kind of tree structure, in which each node indicates a pooling region. For each word, its pooling regions are learned by constructing a tree with its labelled coordinate data. The labelled coordinate data consist of the coordinates of responses and image class labels. The effectiveness of our method is validated by observing if there is an obvious classification accuracy improvement after applying our method. Our experimental results on four small datasets (i.e., Scene-15, Caltech-101, Caltech-256 and Corel-10) show that, the classification accuracy is improved by about 1% to 2.5%. We experimentally demonstrate that the practice of making each word have its own pooling regions is beneficial to image classification task, which is the significance of our work.

RETRACTED ARTICLE: A hybrid encoding strategy for classification of medical imaging modalities

In this work, an automated system is designed to identify and classify the modality of medical images. We considered six modalities in this work: X-ray (XR), computed tomography (CT), magnetic resonance imaging (MR), positron emission tomography (PET), ultrasound (US) and photographs (PX). The methodology is based on encoding scale invariant feature transform (SIFT) features using Bag of Visual Words (BoVW), vector of locally aggregated descriptors (VLAD) and Fisher vector (FV). The encoded features are fed to support vector machine (SVM) classifier for training. The classification accuracy of all the classifiers based on three encoding strategies is compared and analyzed. The hybrid model is then implemented by selecting the best performance from each case. The major contribution of this research work is the application of VLAD for modality classification task which has not been tried so far. Combining the best performance of three encoding strategies, the overall classification accuracy obtained with the proposed system is 90.7%. For identification task, the scores from all the three encoding strategies are combined and the recognition rate obtained is 77.7%.

Hierarchical learning using deep optimum-path forest

Bag-of-Visual Words (BoVW) and deep learning techniques have been widely used in several domains, which include computer-assisted medical diagnoses. In this work, we are interested in developing tools for the automatic identification of Parkinson’s disease using machine learning and the concept of BoVW. The proposed approach concerns a hierarchical-based learning technique to design visual dictionaries through the Deep Optimum-Path Forest classifier. The proposed method was evaluated in six datasets derived from data collected from individuals when performing handwriting exams. Experimental results showed the potential of the technique, with robust achievements.